[PATCH] test suite: don't consider skipped individual tests as failing

diff --git a/8d/8785908b27fbeb43673addc83a307728867538 b/8d/8785908b27fbeb43673addc83a307728867538
new file mode 100644 (file)
index 0000000..4264667
--- /dev/null
+++ b/8d/8785908b27fbeb43673addc83a307728867538
@@ -0,0 +1,76 @@
+Return-Path: <bremner@tesseract.cs.unb.ca>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 6249C6DE0FB1\r
+ for <notmuch@notmuchmail.org>; Thu, 24 Sep 2015 04:30:33 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0.118\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0.118 tagged_above=-999 required=5 tests=[AWL=0.108, \r
+ T_HEADER_FROM_DIFFERENT_DOMAINS=0.01] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id fFiEc76wXxFm for <notmuch@notmuchmail.org>;\r
+ Thu, 24 Sep 2015 04:30:31 -0700 (PDT)\r
+X-Greylist: delayed 954 seconds by postgrey-1.35 at arlo;\r
+ Thu, 24 Sep 2015 04:30:31 PDT\r
+Received: from gitolite.debian.net (gitolite.debian.net [87.98.215.224])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 9634F6DE0B7C\r
+ for <notmuch@notmuchmail.org>; Thu, 24 Sep 2015 04:30:31 -0700 (PDT)\r
+Received: from remotemail by gitolite.debian.net with local (Exim 4.80)\r
+ (envelope-from <bremner@tesseract.cs.unb.ca>)\r
+ id 1Zf4TL-0006PP-Cs; Thu, 24 Sep 2015 11:13:55 +0000\r
+Received: (nullmailer pid 6555 invoked by uid 1000); Thu, 24 Sep 2015\r
+ 11:13:34 -0000\r
+From: David Bremner <david@tethera.net>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH] test suite: don't consider skipped individual tests as\r
+ failing\r
+Date: Thu, 24 Sep 2015 08:13:30 -0300\r
+Message-Id: <1443093210-6511-1-git-send-email-david@tethera.net>\r
+X-Mailer: git-send-email 2.5.1\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.18\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 24 Sep 2015 11:30:33 -0000\r
+\r
+It isn't completely clear what we want to do here, but\r
+\r
+1) We currently don't fail if we skip a whole test file (mainly because\r
+we neglect to count those skipped tests properly). This change at least\r
+makes the two kinds of skipping consistent.\r
+\r
+2) Automated build environments may have good reasons for building with\r
+a minimal set of prereqs, and we don't want to discourage running our\r
+test suite by breaking builds.\r
+---\r
+ test/aggregate-results.sh | 2 +-\r
+ 1 file changed, 1 insertion(+), 1 deletion(-)\r
+\r
+diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh\r
+index b016edb..b30da44 100755\r
+--- a/test/aggregate-results.sh\r
++++ b/test/aggregate-results.sh\r
+@@ -82,7 +82,7 @@ if [ "$skipped" != "0" ]; then\r
+     echo "$skipped $tests skipped."\r
+ fi\r
+ \r
+-if [ $success -gt 0 -a $fixed -eq 0 -a $failed -eq 0 -a $skipped -eq 0 ]\r
++if [ $success -gt 0 -a $fixed -eq 0 -a $failed -eq 0 ]\r
+ then\r
+     exit 0\r
+ else\r
+-- \r
+2.5.1\r
+\r