From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>
To: Tomi Ollila <tomi.ollila@iki.fi>, notmuch@notmuchmail.org
Subject: Re: [PATCH 4/9] test: add support for external executable
 dependencies
In-Reply-To: <yf662i4j6c4.fsf@taco2.nixu.fi>
References: <1321494986-18998-1-git-send-email-dmitry.kurochkin@gmail.com>
	<1321494986-18998-5-git-send-email-dmitry.kurochkin@gmail.com>
	<yf662i4j6c4.fsf@taco2.nixu.fi>
User-Agent: Notmuch/0.10+44~g067c44f (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-pc-linux-gnu)
Date: Tue, 29 Nov 2011 01:53:49 +0400
Message-ID: <87d3cbx6aa.fsf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Mon, 28 Nov 2011 23:16:27 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> On Thu, 17 Nov 2011 05:56:21 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > There is existing support for general prerequisites in the test suite.
> > But it is not very convenient to use: every test case has to keep
> > track for it's dependencies and they have to be explicitly listed.
> > 
> > The patch aims to add better support for a particular type of external
> > dependencies: external executables.  The main idea is to replace
> > missing external binaries with shell functions that have the same
> > name.  These functions always fail and keep track of missing
> > dependencies for a subtest.  The result reporting functions later can
> > check that an external binaries are missing and correctly report SKIP
> > result instead of FAIL.  The primary benefit is that the test cases do
> > not need to declare their dependencies or be changed in any way.
> > ---
> >  test/test-lib.sh |   49 +++++++++++++++++++++++++++++++++++++++++--------
> >  1 files changed, 41 insertions(+), 8 deletions(-)
> > 
> > diff --git a/test/test-lib.sh b/test/test-lib.sh
> > index f21e45e..ab8c6fd 100755
> > --- a/test/test-lib.sh
> > +++ b/test/test-lib.sh
> > @@ -526,40 +526,53 @@ notmuch_json_show_sanitize ()
> >  # - Implicitly by specifying the prerequisite tag in the calls to
> >  #   test_expect_{success,failure,code}.
> >  #
> >  # The single parameter is the prerequisite tag (a simple word, in all
> >  # capital letters by convention).
> >  
> >  test_set_prereq () {
> >  	satisfied="$satisfied$1 "
> >  }
> >  satisfied=" "
> >  
> >  test_have_prereq () {
> >  	case $satisfied in
> >  	*" $1 "*)
> >  		: yes, have it ;;
> >  	*)
> >  		! : nope ;;
> >  	esac
> >  }
> >  
> > +# declare prerequisite for the given external binary
> > +test_declare_external_prereq () {
> > +	binary="$1"
> > +	test "$#" = 2 && name=$2 || name="$binary(1)"
> > +
> > +        hash $binary 2>/dev/null || eval "
> > +$1 () {
> > +	echo -n \"\$test_subtest_missing_external_prereqs_\" | grep -e \" $name \" ||
> > +	test_subtest_missing_external_prereqs_=\"$test_subtest_missing_external_prereqs_ $name\"
> > +	false
> > +}"
> > +}
> > +
> 
> Does this work right ?

It does not.

Moreover, there is a missing backslash before
$test_subtest_missing_external_prereqs_ (and that is why I did not
notice the bug during my poor testing).

> ... the grep -e \" $name \"   -- part requires
> spaces on both side of, but next line does not write trailing space...
> ... and is there leading space (and also the latest
> $test_subtest_missing_external_prereqs_ (just before 'false') is evaluated) ?
> 
> This could perhaps be written like the above test_set_prereq &
> test_save_prereq:
> ...
>         hash $binary 2>/dev/null || eval "
> $binary () {
> 	test_missing_external_prereq_${binary}_=t
> 	case \$test_subtest_missing_external_prereqs_ in
>         	*' $name '*) ;;
>                 *) test_subtest_missing_external_prereqs_=\"\$test_subtest_missing_external_prereqs_$name \"
>         esac
> 	false
> }
> 
> and  test_subtest_missing_external_prereqs_=' '  done in test_reset_state_
> 

Well, this approach would obviously be better, at least because it does
work.  But IMO it is too complex, and essentially has the same problem
as the current code: it mixes the check with diagnostic message.

We already have a proper way to check if dependency is missing already:
test_missing_external_prereq_${binary}_.  We should check it and add the
binary to test_subtest_missing_external_prereqs_ if it is not set.  And
move setting of test_missing_external_prereq_${binary}_ below.

> (I took some code from current git head.... also perhaps instead of 
> setting test_missing_external_prereq_${binary}_=t, (bash) associative
> arrays could be taken into use -- the eval to read that information
> is a bit hairy -- well, at least that part works for sure :D )
> 

Yes!  I like using hash here.

> Hmm... how about this:
> 
>         hash $binary 2>/dev/null || eval "
> $binary () {
>         if [ -z \"\${test_missing_external_prereq_[$binary]}\" ]
>         then
>         	test_missing_external_prereq_[$binary]=t
>                 test_subtest_missing_external_prereqs_=\"\$test_subtest_missing_external_prereqs_ $name\"
>         fi
> 	false
> }
> 

There is some inconsistent indenting in the above code (mixed tabs and
spaces).  Also test_require_external_prereq() should be changed as well.

Otherwise I like it.  Would you please submit a patch?

Regards,
  Dmitry

> and test_subtest_missing_external_prereqs_ cleared in test_reset_state_ like now.
> 
> Tomi