From: Tomi Ollila <tomi.ollila@iki.fi>
Date: Sun, 8 Sep 2013 15:53:30 +0000 (+0300)
Subject: [PATCH v2] test: exit with nonzero value when not all tests completed successfully
X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=b5f7b8500d712d89a6a6319d55a5aacf11c749a2;p=notmuch-archives.git

[PATCH v2] test: exit with nonzero value when not all tests completed successfully
---

diff --git a/3c/a4abdb898a148226514197596cb3a6bf587ded b/3c/a4abdb898a148226514197596cb3a6bf587ded
new file mode 100644
index 000000000..cdb794829
--- /dev/null
+++ b/3c/a4abdb898a148226514197596cb3a6bf587ded
@@ -0,0 +1,100 @@
+Return-Path: <too@guru-group.fi>
+X-Original-To: notmuch@notmuchmail.org
+Delivered-To: notmuch@notmuchmail.org
+Received: from localhost (localhost [127.0.0.1])
+	by olra.theworths.org (Postfix) with ESMTP id 51757431FBC
+	for <notmuch@notmuchmail.org>; Sun,  8 Sep 2013 08:53:45 -0700 (PDT)
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
+X-Spam-Flag: NO
+X-Spam-Score: 0
+X-Spam-Level: 
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
+	autolearn=disabled
+Received: from olra.theworths.org ([127.0.0.1])
+	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
+	with ESMTP id HabxIQnpw6mA for <notmuch@notmuchmail.org>;
+	Sun,  8 Sep 2013 08:53:38 -0700 (PDT)
+Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])
+	by olra.theworths.org (Postfix) with ESMTP id AC749431FAE
+	for <notmuch@notmuchmail.org>; Sun,  8 Sep 2013 08:53:38 -0700 (PDT)
+Received: by guru.guru-group.fi (Postfix, from userid 501)
+	id 85213100093; Sun,  8 Sep 2013 18:53:32 +0300 (EEST)
+From: Tomi Ollila <tomi.ollila@iki.fi>
+To: notmuch@notmuchmail.org
+Subject: [PATCH v2] test: exit with nonzero value when not all tests completed
+	successfully
+Date: Sun,  8 Sep 2013 18:53:30 +0300
+Message-Id: <1378655610-21497-1-git-send-email-tomi.ollila@iki.fi>
+X-Mailer: git-send-email 1.8.0
+Cc: tomi.ollila@iki.fi
+X-BeenThere: notmuch@notmuchmail.org
+X-Mailman-Version: 2.1.13
+Precedence: list
+List-Id: "Use and development of the notmuch mail system."
+	<notmuch.notmuchmail.org>
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>
+List-Post: <mailto:notmuch@notmuchmail.org>
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
+X-List-Received-Date: Sun, 08 Sep 2013 15:53:45 -0000
+
+If any of the tests in our test system is not passing the execution
+of the test suite completes with nonzero exit value.
+
+It is better to rely on the exit value of the test system instead
+of some arbitrary strings in test output (or use both).
+---
+
+This is v2 of id:1378479935-26312-1-git-send-email-tomi.ollila@iki.fi
+thanks to Mark's observation in id:87d2olg9bz.fsf@qmul.ac.uk
+which widened my thought region :D
+
+Tomi
+
+ test/aggregate-results.sh | 9 +++++++++
+ test/notmuch-test         | 3 +++
+ 2 files changed, 12 insertions(+)
+
+diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh
+index 732d6ca..b016edb 100755
+--- a/test/aggregate-results.sh
++++ b/test/aggregate-results.sh
+@@ -1,5 +1,7 @@
+ #!/usr/bin/env bash
+ 
++set -eu
++
+ fixed=0
+ success=0
+ failed=0
+@@ -79,3 +81,10 @@ if [ "$skipped" != "0" ]; then
+     tests=$(pluralize "test" $skipped)
+     echo "$skipped $tests skipped."
+ fi
++
++if [ $success -gt 0 -a $fixed -eq 0 -a $failed -eq 0 -a $skipped -eq 0 ]
++then
++    exit 0
++else
++    exit 1
++fi
+diff --git a/test/notmuch-test b/test/notmuch-test
+index 6db7979..aa28bb0 100755
+--- a/test/notmuch-test
++++ b/test/notmuch-test
+@@ -97,6 +97,9 @@ trap - HUP INT TERM
+ 
+ # Report results
+ ./aggregate-results.sh test-results/*
++ev=$?
+ 
+ # Clean up
+ rm -rf test-results corpus.mail
++
++exit $ev
+-- 
+1.8.0
+