From: Pieter Praet <pieter@praet.org>
To: Jameson Graef Rollins <jrollins@finestructure.net>,
 notmuch@notmuchmail.org
Subject: Re: [PATCH 1/4] test: add 'GnuPG' prereq to dependent 'crypto' tests
In-Reply-To: <87hb2n4k5c.fsf@servo.finestructure.net>
References: <1320176954-4897-1-git-send-email-pieter@praet.org>
	<1320176954-4897-2-git-send-email-pieter@praet.org>
	<87hb2n4k5c.fsf@servo.finestructure.net>
User-Agent: Notmuch/0.9+33~gadde72d (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-unknown-linux-gnu)
Date: Wed, 02 Nov 2011 18:20:09 +0100
Message-ID: <8739e6o306.fsf@praet.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Tue, 01 Nov 2011 14:17:51 -0700, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Tue,  1 Nov 2011 20:49:11 +0100, Pieter Praet <pieter@praet.org> wrote:
> > -test_expect_success 'emacs delivery of signed message' \
> > +test_expect_success GPG 'emacs delivery of signed message' \
> 
> Hi, Pieter and Thomas.  Thanks for all the work on this, but I have one
> issue.  Is there a way we can do this without adding a new argument to
> every test function?  For some reason I really don't like that solution.
> It seems too invasive.  Can't we have something that works more like
> test_subtest_known_broken, that modifies the test environment, rather
> than add an argument to every call of every testing function?
> 
> jamie.

I've been thinking the very same thing.


We could use `test_have_prereq' to get rid of the argument, e.g.:

  #+begin_src sh
    test_have_prereq EMACS && \
    test_begin_subtest "blah"
    echo "doing stuff..."
    test_expect_equal_file OUTPUT EXPECTED
  #+end_src

...but still, everything between `test_begin_subtest' and `test_expect_*'
would be executed, so to err on the safe side, we could wrap the full body
of every test, e.g.:

  #+begin_src sh
    test_begin_subtest "blah"
    test_have_prereq EMACS && {
    ...
    echo "doing stuff..."
    ...
    }
    test_expect_equal_file OUTPUT EXPECTED
  #+end_src


Or... (I've given this zero thought, so please take it with a bag of
salt) we could run all tests from inside a "controller" which `eval's
their contents line by line, and skips a test entirely whenever $? > 0.


All of this is (still) excessively invasive in some way or another though...

Suggestions?


Peace


-- 
Pieter