From: Austin Clements <amdragon@MIT.EDU>
To: notmuch@notmuchmail.org
Subject: [PATCH] test: Abort driver if a test script aborts
Date: Tue, 27 Nov 2012 23:13:16 -0500
Message-Id: <1354075997-29140-1-git-send-email-amdragon@mit.edu>
Cc: tomi.ollila@iki.fi
Precedence: list

Previously, if a test script aborted (e.g., because it passed too few
arguments to a test function), the test driver loop would simply
continue on to the next test script and the final results would
declare that everything passed (except that the test count would look
suspiciously low, but maybe you just misremembered how many tests
there were).

Now, if a test script exits with a non-zero status and did not produce
a final results file, we propagate that failure out of the driver loop
immediately.

To keep this simple, this patch removes the PID from the test-results
file name.  This PID was inherited from the git test system and seems
unnecessary, since the file name already includes the name of the test
script and the test-results directory is created anew for each run.
---
 test/basic        |    2 +-
 test/notmuch-test |    6 ++++++
 test/test-lib.sh  |    2 +-
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/test/basic b/test/basic
index 1b842d2..b7feb07 100755
--- a/test/basic
+++ b/test/basic
@@ -79,7 +79,7 @@ test_begin_subtest "Ensure that -v does not suppress test output"
 output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
-rm $TEST_DIRECTORY/test-results/test-verbose-*
+rm $TEST_DIRECTORY/test-results/test-verbose
 rm -r $TEST_DIRECTORY/tmp.test-verbose
 test_expect_equal "$output" "$expected"
 
diff --git a/test/notmuch-test b/test/notmuch-test
index 9a1b375..f275439 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -81,6 +81,12 @@ trap 'e=$?; kill $!; exit $e' HUP INT TERM
 for test in $TESTS; do
     $TEST_TIMEOUT_CMD ./$test "$@" &
     wait $!
+    # If the test failed without producing results, then it aborted,
+    # so we should abort, too.
+    RES=$?
+    if [[ $RES != 0 && ! -e "test-results/${test%.sh}" ]]; then
+        exit $RES
+    fi
 done
 trap - HUP INT TERM
 
diff --git a/test/test-lib.sh b/test/test-lib.sh
index e092231..77063a4 100644
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -920,7 +920,7 @@ test_done () {
 	GIT_EXIT_OK=t
 	test_results_dir="$TEST_DIRECTORY/test-results"
 	mkdir -p "$test_results_dir"
-	test_results_path="$test_results_dir/${0%.sh}-$$"
+	test_results_path="$test_results_dir/${0%.sh}"
 
 	echo "total $test_count" >> $test_results_path
 	echo "success $test_success" >> $test_results_path
-- 
1.7.10.4