From: Jameson Graef Rollins <jrollins@finestructure.net>
To: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>,
	Tomi Ollila <tomi.ollila@iki.fi>, notmuch@notmuchmail.org
Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible
In-Reply-To: <87k446n8ji.fsf@gmail.com>
References: <1328080794-24670-1-git-send-email-dmitry.kurochkin@gmail.com>
	<87r4yfszx9.fsf@servo.finestructure.net>
	<87pqdync64.fsf@gmail.com> <m262fqc0wv.fsf@guru.guru-group.fi>
	<87k446n8ji.fsf@gmail.com>
User-Agent: Notmuch/0.11+128~g6f388fa (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-pc-linux-gnu)
Date: Wed, 01 Feb 2012 09:24:32 -0800
Message-ID: <87ehuetqjz.fsf@servo.finestructure.net>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha256; protocol="application/pgp-signature"
Precedence: list

--=-=-=
Content-Transfer-Encoding: quoted-printable

On Wed, 01 Feb 2012 14:37:53 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmai=
l.com> wrote:
> On Wed, 01 Feb 2012 12:18:08 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrot=
e:
> >=20
> > There are at least these options here
> >=20
> > 1) go through all ~100 places where test_expect_equal_file is used
> >    and fix the call order: quick look tells that the offending uses
> >    are in dump-restore, hooks, search-limiting and symbol-hiding.
> >=20
> > 2) enforce "expected" filename has some format *and* fix all current
> >    uses of it. Add testbed_error () function which yells loudly ane exi=
ts...
> >=20
> > 3) guess which is output and which is expected from args so that=20
> >    machine helps tester here (for both diff output & copied files)a
> >=20
> > 4) just copy compared files to some directory, those are named as
> >    basename of the original -- diff order still inconsistent.
> >=20
> >=20
> > I'd just go with option 1 and fix new *violations* when stumble upon on=
e.
> >=20
>=20
> Option 1 does not solve the problem.  New violations would apper and
> need to be fixed again.  I am for option 2.

How is enforcing use of a particular filename better and more robust
than enforcing argument order?  You'll still have to force an arbitrary
heuristic.  And you'll still be vulnerable to people messing up the file
names (which actually seems easier to get wrong than messing up the
order).  And you'll have to have more code to parse the argument
strings.  And you'll still get inconsistent diffs.

If this is really a problem, I vote for 1.  In general, I am not in
favor of making the test suite more complicated than it needs to be.

jamie.

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBCAAGBQJPKXVQAAoJEO00zqvie6q8/L0P/3zAXYtLqisL3wl5Zk2urhcH
6kQI2EFKcdPqRO1aKPvjT7dlnkXHeztxovwAYNAl2fj7TRbnnIzsGmm1H49fDi1b
Sqa/aHwm9d8GFiHP981YV9cgLfyvHQdRjr7OXBangZTKzIoIs7s/uV9OAHXAGmN3
SbX4f5el9MVYMQXxbVs4B6yGP4qMi1LsIyFxoI3crrYqJIykbedOMfglFPzbb3vD
0ivtMphws9mutnDyXJNCF+HzRPDjnI7ac3/u2KqJ67++WOOJ76hfHRa0f4tcrGC8
XNakXDDiUhirvBn7+0usG+iwoEd8SwC1IsbTPrNywAE/WjCi78sjmRHHpBHr+qAE
T7tDudL3tctuI4c3MW95GbbVWSLGsOvWcXR64O12gEyhFs5P9wnNMbx6KhdG8WxM
vbwIixdSCU1CJUAKOFs9MVbBRbtddrvSwWlL9JEwig+/41mH5HiuvIngddKyOYkP
84NLTzY9cG9FJzrVs5ud++N3D5bjhPcZbeUQ/ZD0mjsq1AKtzY/fZqVytZywthCH
gVobPgz50mCNnkKrod9oMmHrqkVXBpTrguvRuO4P4Cek6aILxk92O7uxtZd3wQB6
BQFTOh8+331zIkoXuDkzSArMfrb8+IVKDdy1WB7zWiq9qFUOYXjzV/WNQqKGU8Rz
QttiXiBdZIDBkfbQb6Vw
=xYxA
-----END PGP SIGNATURE-----
--=-=-=--