--- /dev/null
+Return-Path: <sojkam1@fel.cvut.cz>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 9720840DDC9\r
+ for <notmuch@notmuchmail.org>; Sun, 14 Nov 2010 13:54:54 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -1.9\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5\r
+ tests=[BAYES_00=-1.9] autolearn=ham\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id B8M0rVVqMv-0 for <notmuch@notmuchmail.org>;\r
+ Sun, 14 Nov 2010 13:54:42 -0800 (PST)\r
+Received: from max.feld.cvut.cz (max.feld.cvut.cz [147.32.192.36])\r
+ by olra.theworths.org (Postfix) with ESMTP id B81C440DDF7\r
+ for <notmuch@notmuchmail.org>; Sun, 14 Nov 2010 13:54:41 -0800 (PST)\r
+Received: from localhost (unknown [192.168.200.4])\r
+ by max.feld.cvut.cz (Postfix) with ESMTP id 175533CFE74;\r
+ Sun, 14 Nov 2010 22:54:41 +0100 (CET)\r
+X-Virus-Scanned: IMAP AMAVIS\r
+Received: from max.feld.cvut.cz ([192.168.200.1])\r
+ by localhost (styx.feld.cvut.cz [192.168.200.4]) (amavisd-new,\r
+ port 10044)\r
+ with ESMTP id o6lj5FZRmmI4; Sun, 14 Nov 2010 22:54:39 +0100 (CET)\r
+Received: from imap.feld.cvut.cz (imap.feld.cvut.cz [147.32.192.34])\r
+ by max.feld.cvut.cz (Postfix) with ESMTP id 6FA0F19F331D;\r
+ Sun, 14 Nov 2010 22:54:39 +0100 (CET)\r
+Received: from steelpick.2x.cz (unknown [213.29.198.144])\r
+ (Authenticated sender: sojkam1)\r
+ by imap.feld.cvut.cz (Postfix) with ESMTPSA id 51E4DFA004;\r
+ Sun, 14 Nov 2010 22:54:39 +0100 (CET)\r
+Received: from wsh by steelpick.2x.cz with local (Exim 4.72)\r
+ (envelope-from <sojkam1@fel.cvut.cz>)\r
+ id 1PHkX0-0002Oy-W7; Sun, 14 Nov 2010 22:54:39 +0100\r
+From: Michal Sojka <sojkam1@fel.cvut.cz>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 1/5] test: Better handling of stdout and stderr\r
+Date: Sun, 14 Nov 2010 22:54:28 +0100\r
+Message-Id: <1289771672-9204-1-git-send-email-sojkam1@fel.cvut.cz>\r
+X-Mailer: git-send-email 1.7.2.3\r
+In-Reply-To: <87hbfjr2sr.fsf@steelpick.2x.cz>\r
+References: <87hbfjr2sr.fsf@steelpick.2x.cz>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 14 Nov 2010 21:54:54 -0000\r
+\r
+Git-style tests (test_expect_success etc.) suppress stdout and stderr\r
+unless -v is given. Notmuch-style tests (created by test_begin_subtest\r
+and test_expect_equal) do not have this behavior so implement it the\r
+same.\r
+\r
+Additionally, for both test styles, the test-lib.sh is changed so that\r
+the content of suppressed stdout and stderr is shown in case of failed\r
+test.\r
+\r
+Finally a test for this functionality is added to basic tests.\r
+---\r
+ test/basic | 21 ++++++++++++++++++++-\r
+ test/test-lib.sh | 21 +++++++++++++++------\r
+ test/test-verbose | 27 +++++++++++++++++++++++++++\r
+ test/test.expected-output/test-verbose-no | 20 ++++++++++++++++++++\r
+ test/test.expected-output/test-verbose-yes | 24 ++++++++++++++++++++++++\r
+ 5 files changed, 106 insertions(+), 7 deletions(-)\r
+ create mode 100755 test/test-verbose\r
+ create mode 100644 test/test.expected-output/test-verbose-no\r
+ create mode 100644 test/test.expected-output/test-verbose-yes\r
+\r
+diff --git a/test/basic b/test/basic\r
+index 725c753..309779c 100755\r
+--- a/test/basic\r
++++ b/test/basic\r
+@@ -52,9 +52,28 @@ test_expect_code 2 'failure to clean up causes the test to fail' '\r
+ # Ensure that all tests are being run\r
+ test_begin_subtest 'Ensure that all available tests will be run by notmuch-test'\r
+ tests_in_suite=$(grep TESTS= ../notmuch-test | sed -e "s/TESTS=\"\(.*\)\"/\1/" | tr " " "\n" | sort)\r
+-available=$(ls -1 ../ | grep -v -E "^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test|README|test-lib.sh|test-results|tmp.*|valgrind|corpus*|emacs.expected-output|smtp-dummy|smtp-dummy.c)" | sort)\r
++available=$(ls -1 ../ | grep -v -E "^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test|README|test-lib.sh|test-results|tmp.*|valgrind|corpus*|emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|test.expected-output)" | sort)\r
+ test_expect_equal "$tests_in_suite" "$available"\r
+ \r
++EXPECTED=../test.expected-output\r
++suppress_diff_date() {\r
++ sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \\r
++ -e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'\r
++}\r
++\r
++test_begin_subtest "Ensure that test output is suppressed unless the test fails"\r
++output=$(cd ..; ./test-verbose 2>&1 | suppress_diff_date)\r
++expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)\r
++test_expect_equal "$output" "$expected"\r
++\r
++test_begin_subtest "Ensure that -v does not suppress test output"\r
++output=$(cd ..; ./test-verbose -v 2>&1 | suppress_diff_date)\r
++expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)\r
++# Do not include the results of test-verbose in totals\r
++rm $TEST_DIRECTORY/test-results/test-verbose-*\r
++test_expect_equal "$output" "$expected"\r
++\r
++\r
+ ################################################################\r
+ # Test mail store prepared in test-lib.sh\r
+ \r
+diff --git a/test/test-lib.sh b/test/test-lib.sh\r
+index 418eaa7..68c9cf8 100644\r
+--- a/test/test-lib.sh\r
++++ b/test/test-lib.sh\r
+@@ -165,12 +165,6 @@ fi\r
+ echo $(basename "$0"): "Testing ${test_description}"\r
+ \r
+ exec 5>&1\r
+-if test "$verbose" = "t"\r
+-then\r
+- exec 4>&2 3>&1\r
+-else\r
+- exec 4>/dev/null 3>/dev/null\r
+-fi\r
+ \r
+ test_failure=0\r
+ test_count=0\r
+@@ -403,6 +397,11 @@ add_email_corpus ()\r
+ test_begin_subtest ()\r
+ {\r
+ test_subtest_name="$1"\r
++ # Remember stdout and stderr file descriptios and redirect test\r
++ # output to the previously prepared file descriptors 3 and 4 (see\r
++ # bellow)\r
++ if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi\r
++ exec 6>&1 7>&2 >&3 2>&4\r
+ }\r
+ \r
+ # Pass test if two arguments match\r
+@@ -413,6 +412,7 @@ test_begin_subtest ()\r
+ # name.\r
+ test_expect_equal ()\r
+ {\r
++ exec 1>&6 2>&7 # Restore stdout and stderr\r
+ test "$#" = 3 && { prereq=$1; shift; } || prereq=\r
+ test "$#" = 2 ||\r
+ error "bug in the test script: not 2 or 3 parameters to test_expect_equal"\r
+@@ -508,6 +508,7 @@ test_failure_ () {\r
+ echo " $1"\r
+ shift\r
+ echo "$@" | sed -e 's/^/ /'\r
++ if test "$verbose" != "t"; then cat test.output; fi\r
+ test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }\r
+ }\r
+ \r
+@@ -529,6 +530,7 @@ test_debug () {\r
+ \r
+ test_run_ () {\r
+ test_cleanup=:\r
++ if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi\r
+ eval >&3 2>&4 "$1"\r
+ eval_ret=$?\r
+ eval >&3 2>&4 "$test_cleanup"\r
+@@ -918,6 +920,13 @@ EOF\r
+ # in subprocesses like git equals our $PWD (for pathname comparisons).\r
+ cd -P "$test" || error "Cannot setup test environment"\r
+ \r
++if test "$verbose" = "t"\r
++then\r
++ exec 4>&2 3>&1\r
++else\r
++ exec 4>test.output 3>&4\r
++fi\r
++\r
+ this_test=${0##*/}\r
+ for skp in $NOTMUCH_SKIP_TESTS\r
+ do\r
+diff --git a/test/test-verbose b/test/test-verbose\r
+new file mode 100755\r
+index 0000000..f29a9c7\r
+--- /dev/null\r
++++ b/test/test-verbose\r
+@@ -0,0 +1,27 @@\r
++#!/bin/bash\r
++\r
++test_description='the verbosity options of the test framework itself.'\r
++\r
++. ./test-lib.sh\r
++\r
++test_expect_success 'print something in test_expect_success and pass' '\r
++ echo "hello stdout" &&\r
++ echo "hello stderr" >&2 &&\r
++ true\r
++'\r
++test_expect_success 'print something in test_expect_success and fail' '\r
++ echo "hello stdout" &&\r
++ echo "hello stderr" >&2 &&\r
++ false\r
++'\r
++test_begin_subtest 'print something between test_begin_subtest and test_expect_equal and pass'\r
++echo "hello stdout"\r
++echo "hello stderr" >&2\r
++test_expect_equal "a" "a"\r
++\r
++test_begin_subtest 'print something test_begin_subtest and test_expect_equal and fail'\r
++echo "hello stdout"\r
++echo "hello stderr" >&2\r
++test_expect_equal "a" "b"\r
++\r
++test_done\r
+diff --git a/test/test.expected-output/test-verbose-no b/test/test.expected-output/test-verbose-no\r
+new file mode 100644\r
+index 0000000..0bca754\r
+--- /dev/null\r
++++ b/test/test.expected-output/test-verbose-no\r
+@@ -0,0 +1,20 @@\r
++test-verbose: Testing the verbosity options of the test framework itself.\r
++ PASS print something in test_expect_success and pass\r
++ FAIL print something in test_expect_success and fail\r
++ \r
++ echo "hello stdout" &&\r
++ echo "hello stderr" >&2 &&\r
++ false\r
++ \r
++hello stdout\r
++hello stderr\r
++ PASS print something between test_begin_subtest and test_expect_equal and pass\r
++ FAIL print something test_begin_subtest and test_expect_equal and fail\r
++ --- test-verbose.4.expected 2010-11-14 21:41:12.738189710 +0000\r
++ +++ test-verbose.4.output 2010-11-14 21:41:12.738189710 +0000\r
++ @@ -1 +1 @@\r
++ -b\r
++ +a\r
++hello stdout\r
++hello stderr\r
++\r
+diff --git a/test/test.expected-output/test-verbose-yes b/test/test.expected-output/test-verbose-yes\r
+new file mode 100644\r
+index 0000000..ebe5187\r
+--- /dev/null\r
++++ b/test/test.expected-output/test-verbose-yes\r
+@@ -0,0 +1,24 @@\r
++test-verbose: Testing the verbosity options of the test framework itself.\r
++hello stdout\r
++hello stderr\r
++ PASS print something in test_expect_success and pass\r
++hello stdout\r
++hello stderr\r
++ FAIL print something in test_expect_success and fail\r
++ \r
++ echo "hello stdout" &&\r
++ echo "hello stderr" >&2 &&\r
++ false\r
++ \r
++hello stdout\r
++hello stderr\r
++ PASS print something between test_begin_subtest and test_expect_equal and pass\r
++hello stdout\r
++hello stderr\r
++ FAIL print something test_begin_subtest and test_expect_equal and fail\r
++ --- test-verbose.4.expected 2010-11-14 21:41:06.650023289 +0000\r
++ +++ test-verbose.4.output 2010-11-14 21:41:06.650023289 +0000\r
++ @@ -1 +1 @@\r
++ -b\r
++ +a\r
++\r
+-- \r
+1.7.2.3\r
+\r
projects / notmuch-archives.git / commitdiff