From: David Bremner <david@tethera.net>
To: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>, notmuch@notmuchmail.org
Subject: Re: [PATCH] test: reset known_broken status in test_expect_equal and
	test_expect_equal_file
In-Reply-To: <87r53mveq9.fsf@gmail.com>
References: <1309752441-10651-3-git-send-email-dmitry.kurochkin@gmail.com>
	<1315782714-32287-1-git-send-email-david@tethera.net>
	<87r53mveq9.fsf@gmail.com>
User-Agent: Notmuch/0.8 (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-pc-linux-gnu)
Date: Sun, 11 Sep 2011 20:51:47 -0300
Message-ID: <877h5ed4do.fsf@zancas.localnet>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Mon, 12 Sep 2011 03:30:54 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> Hi David.
> IMHO this is not a good idea, because:
> 
> 1. It introduces multiple places where the flag is reset.  If new
>    test_expect_* functions are added in the future, there would be more
>    of these.  So it brings us more complex code, code duplication, more
>    chances for bugs, etc.

Well, I'm not sure how to solve this without code duplication, since
there is no test_end_subtest. We could make one, but that seems pretty
intrusive.

> 2. No support for tests with multiple test_expect_* calls.  I do not
>    know if it is used or works now, but the patch certainly does not

Looking at the code for test_expect_equal_* (note that these two are
very different than test_expect_success and test_expect_failure), 
several things are reset already, so I don't think it will work even
before my patch to call those routines twice.

> 3. I thought that every test must start with a test_begin_subtest call.
>    So it is the right place to put all subtest initialization code to.
>    Is this not correct?  If it is correct, then I do not understand why
>    we should support buggy tests by hiding (some of) their bugs.  Why do
>    we need it?

I could not find anything in the docs (or test-lib.sh) that says
test_begin_subtest must be called before test_expect_success or
test_expect_failure. Indeed if test_begin_subtest is called first, the
extra message argument to those two does not really make sense.

What brought this to my attention is the atomicity test introduced by
amdragon, which does exactly

test_begin_subtest
test_expect_equal_failure
test_expect_success

d