Re: [PATCH] test: optionally print subtest number

diff --git a/ce/6d49890ffb244a5a21f64fbdf6163b05e4c70a b/ce/6d49890ffb244a5a21f64fbdf6163b05e4c70a
new file mode 100644 (file)
index 0000000..d3f90b9
--- /dev/null
+++ b/ce/6d49890ffb244a5a21f64fbdf6163b05e4c70a
@@ -0,0 +1,113 @@
+Return-Path: <dmitry.kurochkin@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 0DD74429E26\r
+       for <notmuch@notmuchmail.org>; Tue, 13 Dec 2011 15:52:01 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id bBX9UopunUbJ for <notmuch@notmuchmail.org>;\r
+       Tue, 13 Dec 2011 15:51:59 -0800 (PST)\r
+Received: from mail-fx0-f53.google.com (mail-fx0-f53.google.com\r
+       [209.85.161.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 4973D429E25\r
+       for <notmuch@notmuchmail.org>; Tue, 13 Dec 2011 15:51:59 -0800 (PST)\r
+Received: by faaa5 with SMTP id a5so831508faa.26\r
+       for <notmuch@notmuchmail.org>; Tue, 13 Dec 2011 15:51:56 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;\r
+       h=from:to:subject:in-reply-to:references:user-agent:date:message-id\r
+       :mime-version:content-type;\r
+       bh=M/vSVYXT0lL6HVxpiQZLY2Rad1bSu23NK1bZQ4mt0pg=;\r
+       b=xEl6SlrEVxiQZXsf8/nOo4Gf3xI8eWwu2keLoFft/zuWkuHelSqAu7WombDQBS9Bfu\r
+       9pDOzMY+RFRjY86QOZ/HHAxyk1w7iTUghzqFcRG64lSihVIMIQ8rJvEM1rdU9vhKSZYR\r
+       weaoex0aocaReoCssvdDQQ50d47qqQEuz0+hM=\r
+Received: by 10.180.4.37 with SMTP id h5mr922126wih.45.1323820316532;\r
+       Tue, 13 Dec 2011 15:51:56 -0800 (PST)\r
+Received: from localhost ([91.144.186.21])\r
+       by mx.google.com with ESMTPS id dj9sm973169wib.6.2011.12.13.15.51.55\r
+       (version=TLSv1/SSLv3 cipher=OTHER);\r
+       Tue, 13 Dec 2011 15:51:56 -0800 (PST)\r
+From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>\r
+To: Jameson Graef Rollins <jrollins@finestructure.net>,\r
+       David Bremner <david@tethera.net>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] test: optionally print subtest number\r
+In-Reply-To: <87pqfs2gfa.fsf@servo.finestructure.net>\r
+References: <1323806207-31888-1-git-send-email-david@tethera.net>\r
+       <87zkew2jtu.fsf@servo.finestructure.net>\r
+       <87d3bsjc1z.fsf@zancas.localnet> <87hb14ukbc.fsf@gmail.com>\r
+       <87pqfs2gfa.fsf@servo.finestructure.net>\r
+User-Agent: Notmuch/0.10.2+96~g74e5ae5 (http://notmuchmail.org) Emacs/23.3.1\r
+       (x86_64-pc-linux-gnu)\r
+Date: Wed, 14 Dec 2011 03:51:19 +0400\r
+Message-ID: <87ehw8uj2g.fsf@gmail.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 13 Dec 2011 23:52:01 -0000\r
+\r
+On Tue, 13 Dec 2011 15:35:53 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:\r
+> On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:\r
+> > BTW I have some plans to introduce optional explicit test ids that can\r
+> > be used for inter-test dependencies.  E.g.:\r
+> > \r
+> >   test_begin_subtest test-id-1 "A subtest"\r
+> >   ....\r
+> >   ;; in another test requre that test-id-1 passed\r
+> >   test_require_subtest test-id-1\r
+> \r
+> Would the required test need to be listed twice, both on the\r
+> begin_subtest line *and* in the require_subtest line?\r
+> \r
+> And again, why would the test id have to be any different that the\r
+> existing test names?  The tests already have names, so I don't\r
+> understand why we would want to introduce some other kind of\r
+> identification.  Seems like it's just going to add extra confusion.\r
+> \r
+\r
+What you listed in the other email are test scripts, each with many\r
+subtests.  I was talking about dependencies between subtests, not test\r
+scripts.\r
+\r
+> And speaking of which, I sometimes worry that the test infrastructure\r
+> itself is getting too complicated.  Pretty soon we're going to need\r
+> tests for the tests.\r
+\r
+We already have them :)  Though, pretty limited.\r
+\r
+>  I don't necessarily see the need to all of these\r
+> extra features in the test suite, so I worry that it's just making\r
+> everything harder to debug.\r
+> \r
+\r
+I hope we can keep balance here.\r
+\r
+Without inter-subtest dependencies, we have unhealthy situation where\r
+some tests may be skipped because of missing prerequisites, but test\r
+that depend on them are failing.  The only alternative I see is to\r
+rewrite these tests to remove the dependencies.  But that would\r
+complicate test cases itself, so I believe inter-subtest dependencies is\r
+a better option.\r
+\r
+Regards,\r
+  Dmitry\r
+\r
+> jamie.\r