Re: [PATCH v3 0/5] Git-based modularization of test suite

diff --git a/82/9034e7535ebc5b8ddf06273c638cd9aa901038 b/82/9034e7535ebc5b8ddf06273c638cd9aa901038
new file mode 100644 (file)
index 0000000..a82b148
--- /dev/null
+++ b/82/9034e7535ebc5b8ddf06273c638cd9aa901038
@@ -0,0 +1,102 @@
+Return-Path: <cworth@cworth.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id A125340D142\r
+       for <notmuch@notmuchmail.org>; Wed, 13 Oct 2010 08:42:07 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.89\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.89 tagged_above=-999 required=5\r
+       tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, T_MIME_NO_TEXT=0.01]\r
+       autolearn=ham\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id BeEFl7r2jpBg; Wed, 13 Oct 2010 08:41:55 -0700 (PDT)\r
+Received: from yoom.home.cworth.org (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 869C440D141;\r
+       Wed, 13 Oct 2010 08:41:55 -0700 (PDT)\r
+Received: by yoom.home.cworth.org (Postfix, from userid 1000)\r
+       id 237FD54C05C; Wed, 13 Oct 2010 08:41:55 -0700 (PDT)\r
+From: Carl Worth <cworth@cworth.org>\r
+To: Michal Sojka <sojkam1@fel.cvut.cz>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH v3 0/5] Git-based modularization of test suite\r
+In-Reply-To: <87pqve1cuw.fsf@steelpick.2x.cz>\r
+References: <1276152484-1164-1-git-send-email-sojkam1@fel.cvut.cz>\r
+       <874odkm4ey.fsf@yoom.home.cworth.org>\r
+       <87tyliun06.fsf@steelpick.2x.cz> <87pqve1cuw.fsf@steelpick.2x.cz>\r
+User-Agent: Notmuch/0.3.1-90-g8071c5c (http://notmuchmail.org) Emacs/23.2.1\r
+       (i486-pc-linux-gnu)\r
+Date: Wed, 13 Oct 2010 08:41:54 -0700\r
+Message-ID: <87d3rehzgd.fsf@yoom.home.cworth.org>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/signed; boundary="=-=-=";\r
+       micalg=pgp-sha1; protocol="application/pgp-signature"\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 13 Oct 2010 15:42:07 -0000\r
+\r
+--=-=-=\r
+Content-Transfer-Encoding: quoted-printable\r
+\r
+On Wed, 13 Oct 2010 14:44:23 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrot=\r
+e:\r
+> here is my experience with test_begin_subtest and test_expect_equal\r
+> functions in test-lib.sh. I see there two significant differences from\r
+> the git's test_expect_success and similar functions.\r
+>=20\r
+> First, when the test fails, you only see the differences in the real\r
+> output and the expected output. If you use test_expect_success you also\r
+> see the actual commands that were executed.\r
+\r
+Yes. For me, I never found the printing of the exact commands as\r
+something useful. We probably should fix our test suite to behave\r
+consistently regardless of which interface is used.\r
+\r
+> Second, when some command between test_begin_subtest and\r
+> test_expect_success fails (e.g. because of segfault) you may not detect\r
+> it. You can add "set -e" the test-lib.sh (see the patch bellow), but in\r
+> this case when a test fails you would see\r
+>=20\r
+>   FATAL: Unexpected exit with code 1\r
+>=20\r
+> without any additional information.\r
+\r
+What do you recommend here?\r
+\r
+Frankly, I'm fine if the test cases are easy to read, (and I think they\r
+are easier without having many commands chained by "&&" and concatenated\r
+into a single string). Then if any test fails, (whether a segfault or\r
+simply incorrect output), it shouldn't be hard to just open the test\r
+case and manually inspect the commands.\r
+\r
+But if you've got some idea for anything more useful, (some magic action\r
+with trap for example?), I'd be glad to hear it.\r
+\r
+=2DCarl\r
+\r
+=2D-=20\r
+carl.d.worth@intel.com\r
+\r
+--=-=-=\r
+Content-Type: application/pgp-signature\r
+\r
+-----BEGIN PGP SIGNATURE-----\r
+Version: GnuPG v1.4.10 (GNU/Linux)\r
+\r
+iD8DBQFMtdNC6JDdNq8qSWgRArxeAJ9sr3gVjZK4x4CnTW1NgxSZ4neKsQCfcSY9\r
+5wRtwuDgZ9z7BHNbvaFWcr0=\r
+=rP3a\r
+-----END PGP SIGNATURE-----\r
+--=-=-=--\r