From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
To: Jameson Graef Rollins <jrollins@finestructure.net>,
	David Bremner <david@tethera.net>, notmuch@notmuchmail.org
Subject: Re: [PATCH] test: optionally print subtest number
In-Reply-To: <87pqfs2gfa.fsf@servo.finestructure.net>
References: <1323806207-31888-1-git-send-email-david@tethera.net>
	<87zkew2jtu.fsf@servo.finestructure.net>
	<87d3bsjc1z.fsf@zancas.localnet> <87hb14ukbc.fsf@gmail.com>
	<87pqfs2gfa.fsf@servo.finestructure.net>
User-Agent: Notmuch/0.10.2+96~g74e5ae5 (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-pc-linux-gnu)
Date: Wed, 14 Dec 2011 03:51:19 +0400
Message-ID: <87ehw8uj2g.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Tue, 13 Dec 2011 15:35:53 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Wed, 14 Dec 2011 03:24:23 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > BTW I have some plans to introduce optional explicit test ids that can
> > be used for inter-test dependencies.  E.g.:
> > 
> >   test_begin_subtest test-id-1 "A subtest"
> >   ....
> >   ;; in another test requre that test-id-1 passed
> >   test_require_subtest test-id-1
> 
> Would the required test need to be listed twice, both on the
> begin_subtest line *and* in the require_subtest line?
> 
> And again, why would the test id have to be any different that the
> existing test names?  The tests already have names, so I don't
> understand why we would want to introduce some other kind of
> identification.  Seems like it's just going to add extra confusion.
> 

What you listed in the other email are test scripts, each with many
subtests.  I was talking about dependencies between subtests, not test
scripts.

> And speaking of which, I sometimes worry that the test infrastructure
> itself is getting too complicated.  Pretty soon we're going to need
> tests for the tests.

We already have them :)  Though, pretty limited.

>  I don't necessarily see the need to all of these
> extra features in the test suite, so I worry that it's just making
> everything harder to debug.
> 

I hope we can keep balance here.

Without inter-subtest dependencies, we have unhealthy situation where
some tests may be skipped because of missing prerequisites, but test
that depend on them are failing.  The only alternative I see is to
rewrite these tests to remove the dependencies.  But that would
complicate test cases itself, so I believe inter-subtest dependencies is
a better option.

Regards,
  Dmitry

> jamie.