From: Jameson Graef Rollins Date: Wed, 1 Feb 2012 17:24:32 +0000 (+1600) Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=3e09a23934c82cedc4341e2a3aed4eb66115962e;p=notmuch-archives.git Re: [PATCH] test: make test_expect_equal_file() arguments flexible --- diff --git a/bd/310ed43ee5116f296d0d9b1fb2565e7a321a85 b/bd/310ed43ee5116f296d0d9b1fb2565e7a321a85 new file mode 100644 index 000000000..cead4f7dd --- /dev/null +++ b/bd/310ed43ee5116f296d0d9b1fb2565e7a321a85 @@ -0,0 +1,125 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id BCCE1421192 + for ; Wed, 1 Feb 2012 09:24:46 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -2.29 +X-Spam-Level: +X-Spam-Status: No, score=-2.29 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_MED=-2.3, T_MIME_NO_TEXT=0.01] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id o5OOtEwJL9Yf for ; + Wed, 1 Feb 2012 09:24:46 -0800 (PST) +Received: from outgoing-mail.its.caltech.edu (outgoing-mail.its.caltech.edu + [131.215.239.19]) + by olra.theworths.org (Postfix) with ESMTP id 09A6E429E5B + for ; Wed, 1 Feb 2012 09:24:46 -0800 (PST) +Received: from earth-doxen.imss.caltech.edu (localhost [127.0.0.1]) + by earth-doxen-postvirus (Postfix) with ESMTP id EAFD266E0030; + Wed, 1 Feb 2012 09:24:43 -0800 (PST) +X-Spam-Scanned: at Caltech-IMSS on earth-doxen by amavisd-new +Received: from finestructure.net (cpe-76-174-137-84.socal.res.rr.com + [76.174.137.84]) (Authenticated sender: jrollins) + by earth-doxen-submit (Postfix) with ESMTP id 1580B66E01A1; + Wed, 1 Feb 2012 09:24:37 -0800 (PST) +Received: by finestructure.net (Postfix, from userid 1000) + id DEFCC291; Wed, 1 Feb 2012 09:24:35 -0800 (PST) +From: Jameson Graef Rollins +To: Dmitry Kurochkin , + Tomi Ollila , notmuch@notmuchmail.org +Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible +In-Reply-To: <87k446n8ji.fsf@gmail.com> +References: <1328080794-24670-1-git-send-email-dmitry.kurochkin@gmail.com> + <87r4yfszx9.fsf@servo.finestructure.net> + <87pqdync64.fsf@gmail.com> + <87k446n8ji.fsf@gmail.com> +User-Agent: Notmuch/0.11+128~g6f388fa (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Wed, 01 Feb 2012 09:24:32 -0800 +Message-ID: <87ehuetqjz.fsf@servo.finestructure.net> +MIME-Version: 1.0 +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha256; protocol="application/pgp-signature" +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 01 Feb 2012 17:24:46 -0000 + +--=-=-= +Content-Transfer-Encoding: quoted-printable + +On Wed, 01 Feb 2012 14:37:53 +0400, Dmitry Kurochkin wrote: +> On Wed, 01 Feb 2012 12:18:08 +0200, Tomi Ollila wrot= +e: +> >=20 +> > There are at least these options here +> >=20 +> > 1) go through all ~100 places where test_expect_equal_file is used +> > and fix the call order: quick look tells that the offending uses +> > are in dump-restore, hooks, search-limiting and symbol-hiding. +> >=20 +> > 2) enforce "expected" filename has some format *and* fix all current +> > uses of it. Add testbed_error () function which yells loudly ane exi= +ts... +> >=20 +> > 3) guess which is output and which is expected from args so that=20 +> > machine helps tester here (for both diff output & copied files)a +> >=20 +> > 4) just copy compared files to some directory, those are named as +> > basename of the original -- diff order still inconsistent. +> >=20 +> >=20 +> > I'd just go with option 1 and fix new *violations* when stumble upon on= +e. +> >=20 +>=20 +> Option 1 does not solve the problem. New violations would apper and +> need to be fixed again. I am for option 2. + +How is enforcing use of a particular filename better and more robust +than enforcing argument order? You'll still have to force an arbitrary +heuristic. And you'll still be vulnerable to people messing up the file +names (which actually seems easier to get wrong than messing up the +order). And you'll have to have more code to parse the argument +strings. And you'll still get inconsistent diffs. + +If this is really a problem, I vote for 1. In general, I am not in +favor of making the test suite more complicated than it needs to be. + +jamie. + +--=-=-= +Content-Type: application/pgp-signature + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1.4.11 (GNU/Linux) + +iQIcBAEBCAAGBQJPKXVQAAoJEO00zqvie6q8/L0P/3zAXYtLqisL3wl5Zk2urhcH +6kQI2EFKcdPqRO1aKPvjT7dlnkXHeztxovwAYNAl2fj7TRbnnIzsGmm1H49fDi1b +Sqa/aHwm9d8GFiHP981YV9cgLfyvHQdRjr7OXBangZTKzIoIs7s/uV9OAHXAGmN3 +SbX4f5el9MVYMQXxbVs4B6yGP4qMi1LsIyFxoI3crrYqJIykbedOMfglFPzbb3vD +0ivtMphws9mutnDyXJNCF+HzRPDjnI7ac3/u2KqJ67++WOOJ76hfHRa0f4tcrGC8 +XNakXDDiUhirvBn7+0usG+iwoEd8SwC1IsbTPrNywAE/WjCi78sjmRHHpBHr+qAE +T7tDudL3tctuI4c3MW95GbbVWSLGsOvWcXR64O12gEyhFs5P9wnNMbx6KhdG8WxM +vbwIixdSCU1CJUAKOFs9MVbBRbtddrvSwWlL9JEwig+/41mH5HiuvIngddKyOYkP +84NLTzY9cG9FJzrVs5ud++N3D5bjhPcZbeUQ/ZD0mjsq1AKtzY/fZqVytZywthCH +gVobPgz50mCNnkKrod9oMmHrqkVXBpTrguvRuO4P4Cek6aILxk92O7uxtZd3wQB6 +BQFTOh8+331zIkoXuDkzSArMfrb8+IVKDdy1WB7zWiq9qFUOYXjzV/WNQqKGU8Rz +QttiXiBdZIDBkfbQb6Vw +=xYxA +-----END PGP SIGNATURE----- +--=-=-=--