From: Tomi Ollila <tomi.ollila@iki.fi>
To: Austin Clements <amdragon@MIT.EDU>, notmuch@notmuchmail.org
Subject: Re: [PATCH] test: Abort driver if a test script aborts
In-Reply-To: <1354075997-29140-1-git-send-email-amdragon@mit.edu>
References: <1354075997-29140-1-git-send-email-amdragon@mit.edu>
User-Agent: Notmuch/0.14+116~g29fcdb5 (http://notmuchmail.org) Emacs/24.2.1
	(x86_64-unknown-linux-gnu)
Date: Wed, 28 Nov 2012 09:25:07 +0200
Message-ID: <m28v9mxk8c.fsf@guru.guru-group.fi>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list

On Wed, Nov 28 2012, Austin Clements <amdragon@MIT.EDU> wrote:

> Previously, if a test script aborted (e.g., because it passed too few
> arguments to a test function), the test driver loop would simply
> continue on to the next test script and the final results would
> declare that everything passed (except that the test count would look
> suspiciously low, but maybe you just misremembered how many tests
> there were).
>
> Now, if a test script exits with a non-zero status and did not produce
> a final results file, we propagate that failure out of the driver loop
> immediately.
>
> To keep this simple, this patch removes the PID from the test-results
> file name.  This PID was inherited from the git test system and seems
> unnecessary, since the file name already includes the name of the test
> script and the test-results directory is created anew for each run.
> ---

+1

Tomi


>  test/basic        |    2 +-
>  test/notmuch-test |    6 ++++++
>  test/test-lib.sh  |    2 +-
>  3 files changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/test/basic b/test/basic
> index 1b842d2..b7feb07 100755
> --- a/test/basic
> +++ b/test/basic
> @@ -79,7 +79,7 @@ test_begin_subtest "Ensure that -v does not suppress test output"
>  output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)
>  expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
>  # Do not include the results of test-verbose in totals
> -rm $TEST_DIRECTORY/test-results/test-verbose-*
> +rm $TEST_DIRECTORY/test-results/test-verbose
>  rm -r $TEST_DIRECTORY/tmp.test-verbose
>  test_expect_equal "$output" "$expected"
>  
> diff --git a/test/notmuch-test b/test/notmuch-test
> index 9a1b375..f275439 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -81,6 +81,12 @@ trap 'e=$?; kill $!; exit $e' HUP INT TERM
>  for test in $TESTS; do
>      $TEST_TIMEOUT_CMD ./$test "$@" &
>      wait $!
> +    # If the test failed without producing results, then it aborted,
> +    # so we should abort, too.
> +    RES=$?
> +    if [[ $RES != 0 && ! -e "test-results/${test%.sh}" ]]; then
> +        exit $RES
> +    fi
>  done
>  trap - HUP INT TERM
>  
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index e092231..77063a4 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -920,7 +920,7 @@ test_done () {
>  	GIT_EXIT_OK=t
>  	test_results_dir="$TEST_DIRECTORY/test-results"
>  	mkdir -p "$test_results_dir"
> -	test_results_path="$test_results_dir/${0%.sh}-$$"
> +	test_results_path="$test_results_dir/${0%.sh}"
>  
>  	echo "total $test_count" >> $test_results_path
>  	echo "success $test_success" >> $test_results_path
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch