Improved diff-based failure reports from the test suite

diff --git a/5c/3495f1431dfcc6c3d866c5bf850c52fb99be6c b/5c/3495f1431dfcc6c3d866c5bf850c52fb99be6c
new file mode 100644 (file)
index 0000000..f9628c0
--- /dev/null
+++ b/5c/3495f1431dfcc6c3d866c5bf850c52fb99be6c
@@ -0,0 +1,91 @@
+Return-Path: <cworth@cworth.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id A412D4196F2\r
+       for <notmuch@notmuchmail.org>; Thu, 22 Apr 2010 14:14:39 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.89\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.89 tagged_above=-999 required=5\r
+       tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, T_MIME_NO_TEXT=0.01]\r
+       autolearn=ham\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id 8tkK1Lxf0kp4; Thu, 22 Apr 2010 14:14:38 -0700 (PDT)\r
+Received: from yoom.home.cworth.org (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 2837B431FC1;\r
+       Thu, 22 Apr 2010 14:14:38 -0700 (PDT)\r
+Received: by yoom.home.cworth.org (Postfix, from userid 1000)\r
+       id A9ECB568DE4; Thu, 22 Apr 2010 14:14:37 -0700 (PDT)\r
+From: Carl Worth <cworth@cworth.org>\r
+To: Michal Sojka <sojkam1@fel.cvut.cz>, Gregor Hoffleit <gregor@hoffleit.de>,\r
+       notmuch <notmuch@notmuchmail.org>\r
+Subject: Improved diff-based failure reports from the test suite\r
+In-Reply-To: <87vdbrerqs.fsf@steelpick.2x.cz>\r
+References: <1267697893-sup-4538@sam.mediasupervision.de>\r
+       <87r5mje55l.fsf@yoom.home.cworth.org>\r
+       <1271176622-sup-4680@sam.mediasupervision.de>\r
+       <87iq7tft7j.fsf@yoom.home.cworth.org>\r
+       <87ochlt8r9.fsf@steelpick.2x.cz>\r
+       <878w8ofpx9.fsf@yoom.home.cworth.org>\r
+       <87vdbrerqs.fsf@steelpick.2x.cz>\r
+Date: Thu, 22 Apr 2010 14:14:31 -0700\r
+Message-ID: <87k4rzqjew.fsf@yoom.home.cworth.org>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/signed; boundary="=-=-=";\r
+       micalg=pgp-sha1; protocol="application/pgp-signature"\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 22 Apr 2010 21:14:39 -0000\r
+\r
+--=-=-=\r
+Content-Transfer-Encoding: quoted-printable\r
+\r
+On Fri, 16 Apr 2010 10:17:15 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrot=\r
+e:\r
+> It's definitely better than before. The current implementation of\r
+> pass_if_equal has IMHO one drawback - if it compares multiline text and\r
+> there is a difference, it is quite hard to see where.\r
+>=20\r
+> In my tests for maildir synchronization I use this approach:\r
+>=20\r
+>   notmuch search tag:inbox | filter_output > actual &&\r
+>   diff -u - actual <<EOF\r
+>   thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; test message 3 (inbox)\r
+>   EOF\r
+>=20\r
+> Thanks to the usee of diff, I immediately see only the differences.\r
+\r
+I just added a very long multi-line test to the test suite.\r
+\r
+So I took the opportunity to fix it to output the failures with diff. It\r
+also saves both the complete expected and actual output (for any failed\r
+tests) to files of the form test-XXX.expected and test-XXX.output.\r
+\r
+Enjoy!\r
+\r
+=2DCarl\r
+\r
+--=-=-=\r
+Content-Type: application/pgp-signature\r
+\r
+-----BEGIN PGP SIGNATURE-----\r
+Version: GnuPG v1.4.10 (GNU/Linux)\r
+\r
+iD8DBQFL0Lw36JDdNq8qSWgRAmxyAJ9mJSYoUghzbQat3X2gLqv2oEdSPQCdGwMy\r
+RYjDTxkknMu+MUcl73/EiNc=\r
+=7zPR\r
+-----END PGP SIGNATURE-----\r
+--=-=-=--\r