From: Mark Walters <markwalters1009@gmail.com>
To: Tomi Ollila <tomi.ollila@iki.fi>, notmuch@notmuchmail.org
Subject: Re: [PATCH 1/1] test: exit with nonzero value when not all tests
	completed successfully
In-Reply-To: <1378479935-26312-1-git-send-email-tomi.ollila@iki.fi>
References: <1378479935-26312-1-git-send-email-tomi.ollila@iki.fi>
User-Agent: Notmuch/0.16 (http://notmuchmail.org) Emacs/23.4.1
	(x86_64-pc-linux-gnu)
Date: Fri, 06 Sep 2013 18:15:44 +0100
Message-ID: <87d2olg9bz.fsf@qmul.ac.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: tomi.ollila@iki.fi
Precedence: list


Hi

On Fri, 06 Sep 2013, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> If any of the tests in our test system is not passing the execution
> of the test suite completes with nonzero exit value.
>
> It is better to rely on the exit value of the test system instead
> of some arbitrary strings in test output (or use both).

This is obviously a good thing to do: my only query is whether having a
single line setting ev would be clearer. Something along the lines of

if [ "$fixed" = "0 ] && [ "$failed" = "0 ] && [ "$broken" = "0 ] && ["$skipped" = "0 ]; then
 ev=1
else 
 ev=0
fi

But I am happy with it as is too. (I haven't actually tested it
currently).

Best wishes

Mark


> ---
>  test/aggregate-results.sh |    5 +++++
>  test/notmuch-test         |    3 +++
>  2 files changed, 8 insertions(+)
>
> diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh
> index 732d6ca..5193d7f 100755
> --- a/test/aggregate-results.sh
> +++ b/test/aggregate-results.sh
> @@ -49,10 +49,12 @@ pluralize () {
>  }
>  
>  echo "Notmuch test suite complete."
> +ev=1
>  if [ "$fixed" = "0" ] && [ "$failed" = "0" ]; then
>      tests=$(pluralize "test" $total)
>      printf "All $total $tests "
>      if [ "$broken" = "0" ]; then
> +	ev=0
>  	echo "passed."
>      else
>  	failures=$(pluralize "failure" $broken)
> @@ -76,6 +78,9 @@ fi
>  
>  skipped=$(($total - $fixed - $success - $failed - $broken))
>  if [ "$skipped" != "0" ]; then
> +    ev=1
>      tests=$(pluralize "test" $skipped)
>      echo "$skipped $tests skipped."
>  fi
> +
> +exit $ev
> diff --git a/test/notmuch-test b/test/notmuch-test
> index 6db7979..aa28bb0 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -97,6 +97,9 @@ trap - HUP INT TERM
>  
>  # Report results
>  ./aggregate-results.sh test-results/*
> +ev=$?
>  
>  # Clean up
>  rm -rf test-results corpus.mail
> +
> +exit $ev
> -- 
> 1.7.10.4
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch