Re: [PATCH v3 0/5] Git-based modularization of test suite

diff --git a/f0/33e9716eb6f8aa6d2b6d3b478feadcbe767436 b/f0/33e9716eb6f8aa6d2b6d3b478feadcbe767436
new file mode 100644 (file)
index 0000000..cc18da9
--- /dev/null
+++ b/f0/33e9716eb6f8aa6d2b6d3b478feadcbe767436
@@ -0,0 +1,114 @@
+Return-Path: <cworth@cworth.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id A2C3940BFD3\r
+       for <notmuch@notmuchmail.org>; Mon, 20 Sep 2010 15:33:28 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.89\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.89 tagged_above=-999 required=5\r
+       tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, T_MIME_NO_TEXT=0.01]\r
+       autolearn=ham\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id 8lb0jveAW9XF; Mon, 20 Sep 2010 15:33:16 -0700 (PDT)\r
+Received: from yoom.home.cworth.org (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 9347740BC9C;\r
+       Mon, 20 Sep 2010 15:33:16 -0700 (PDT)\r
+Received: by yoom.home.cworth.org (Postfix, from userid 1000)\r
+       id 3DFD4568E44; Mon, 20 Sep 2010 15:33:16 -0700 (PDT)\r
+From: Carl Worth <cworth@cworth.org>\r
+To: Michal Sojka <sojkam1@fel.cvut.cz>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH v3 0/5] Git-based modularization of test suite\r
+In-Reply-To: <1276152484-1164-1-git-send-email-sojkam1@fel.cvut.cz>\r
+References: <1276152484-1164-1-git-send-email-sojkam1@fel.cvut.cz>\r
+User-Agent: Notmuch/0.3.1-61-g9f0accb (http://notmuchmail.org) Emacs/23.2.1\r
+       (i486-pc-linux-gnu)\r
+Date: Mon, 20 Sep 2010 15:33:09 -0700\r
+Message-ID: <874odkm4ey.fsf@yoom.home.cworth.org>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/signed; boundary="=-=-=";\r
+       micalg=pgp-sha1; protocol="application/pgp-signature"\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 20 Sep 2010 22:33:28 -0000\r
+\r
+--=-=-=\r
+Content-Transfer-Encoding: quoted-printable\r
+\r
+> Michal Sojka (5):\r
+>   Copy test framework from Git\r
+>   Update test framework for use with notmuch\r
+>   test: Update helper functions\r
+>   Convert the actual tests to the new framework\r
+>   test: Set all times to UTC\r
+\r
+Hi Michal,\r
+\r
+Thanks for sending this, (and patiently sending in three versions of\r
+it).\r
+\r
+I've now pushed this out, along with some changes of my own on top of\r
+it. My changes make things in the test suite look more like they did\r
+before the git-based modularization, (both output from the test suite\r
+and how the tests themselves are written). I couldn't stand the approach\r
+of doing long chains of shell commands within strings for the tests, and\r
+I also didn't like the numeric filenames.\r
+\r
+Take a look at what I've done, and let me know if I defeated any\r
+intended benefits of the git-based modularization of the test suite.\r
+\r
+One problem with the current arrangement is that all tests now start\r
+with an empty database of messages. This means that the search tests\r
+aren't giving as much test coverage. And the dump/restore tests are\r
+actually testing nothing at all!\r
+\r
+I think what I'd prefer to have is for test-lib.sh to populate a\r
+non-empty database (it could even provide a function to reset the\r
+database to empty for any tests that actually want that).\r
+\r
+For efficiency, the non-empty database could be created just once at the\r
+first invocation and then could simply be copied into place for future\r
+invocations.\r
+\r
+If anyone wants to help fix that, that would be great.\r
+\r
+=2DCarl\r
+\r
+PS. The valgrind support[*] in the newly modularized test suite helped\r
+me find a use-after-free bug in the current code base, so that's already\r
+a useful result here. Thanks again!\r
+\r
+[*] The valgrind support isn't ideal yet since it's not actually\r
+flagging failures since the test suite isn't currently looking at return\r
+values of valgrind/notmuch. This is arguably my fault since I got rid of\r
+the ugly 'command && another_command && yet_another_command' syntax. I'd\r
+like to see a way to fix the valgrind support without introducing that\r
+awful syntax. I'm open to ideas.\r
+\r
+=2D-=20\r
+carl.d.worth@intel.com\r
+\r
+--=-=-=\r
+Content-Type: application/pgp-signature\r
+\r
+-----BEGIN PGP SIGNATURE-----\r
+Version: GnuPG v1.4.10 (GNU/Linux)\r
+\r
+iD8DBQFMl+Em6JDdNq8qSWgRAlA9AKCMGECE/qWTpXXJ743FWFvwaRU7xwCfe3fm\r
+oSFAWYkOPOvZ3ryr/EwrztQ=\r
+=HU0B\r
+-----END PGP SIGNATURE-----\r
+--=-=-=--\r