From f5899c24faa6b1e25ec42210b3608a11f30e2528 Mon Sep 17 00:00:00 2001 From: Dmitry Kurochkin Date: Thu, 2 Feb 2012 04:07:01 +0400 Subject: [PATCH] Re: [PATCH] test: make test_expect_equal_file() arguments flexible --- e3/16cbb0108e75d7211232037720e37a72adb38b | 149 ++++++++++++++++++++++ 1 file changed, 149 insertions(+) create mode 100644 e3/16cbb0108e75d7211232037720e37a72adb38b diff --git a/e3/16cbb0108e75d7211232037720e37a72adb38b b/e3/16cbb0108e75d7211232037720e37a72adb38b new file mode 100644 index 000000000..8937c7417 --- /dev/null +++ b/e3/16cbb0108e75d7211232037720e37a72adb38b @@ -0,0 +1,149 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id D61E7431FAF + for ; Wed, 1 Feb 2012 16:08:17 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 1.7 +X-Spam-Level: * +X-Spam-Status: No, score=1.7 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, FREEMAIL_REPLY=2.499, RCVD_IN_DNSWL_LOW=-0.7] + autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id OhNMhGAMg7Ip for ; + Wed, 1 Feb 2012 16:08:17 -0800 (PST) +Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com + [209.85.214.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id B5732431FAE + for ; Wed, 1 Feb 2012 16:08:16 -0800 (PST) +Received: by bke11 with SMTP id 11so1739408bke.26 + for ; Wed, 01 Feb 2012 16:08:15 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; + h=from:to:subject:in-reply-to:references:user-agent:date:message-id + :mime-version:content-type; + bh=uMCnPsmkImyhcjRnIl8tGx7+O4FQJUrQ9pC1f3WIcqs=; + b=Qnf9/yKafIkuaoqaHxOzTgsLUCLGQCVFKEiKTGJwRZRyomIxSOfiR2VqiCAl5r29gE + 9SdDUjYoLw/SYLmZF5rDD7ZMBSGnb7vMLlKCf4OC8VI6+VJ8GM2Zgf1DcZWvTOtjs180 + tkEKToAUnpEnJFsMcF/yBYHzg8Ti9xP2VmW9A= +Received: by 10.204.152.208 with SMTP id h16mr348389bkw.6.1328141295506; + Wed, 01 Feb 2012 16:08:15 -0800 (PST) +Received: from localhost ([91.144.186.21]) + by mx.google.com with ESMTPS id ew13sm1635099bkb.1.2012.02.01.16.08.14 + (version=TLSv1/SSLv3 cipher=OTHER); + Wed, 01 Feb 2012 16:08:14 -0800 (PST) +From: Dmitry Kurochkin +To: Jameson Graef Rollins , + Tomi Ollila , notmuch@notmuchmail.org +Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible +In-Reply-To: <8739aum87u.fsf@gmail.com> +References: <1328080794-24670-1-git-send-email-dmitry.kurochkin@gmail.com> + <87r4yfszx9.fsf@servo.finestructure.net> + <87pqdync64.fsf@gmail.com> + <87k446n8ji.fsf@gmail.com> <87ehuetqjz.fsf@servo.finestructure.net> + <8739aum87u.fsf@gmail.com> +User-Agent: Notmuch/0.11+139~gd9b7cab (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Thu, 02 Feb 2012 04:07:01 +0400 +Message-ID: <87wr86ksii.fsf@gmail.com> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 02 Feb 2012 00:08:18 -0000 + +On Thu, 02 Feb 2012 03:42:29 +0400, Dmitry Kurochkin wrote: +> Hi Jameson. +> +> On Wed, 01 Feb 2012 09:24:32 -0800, Jameson Graef Rollins wrote: +> > On Wed, 01 Feb 2012 14:37:53 +0400, Dmitry Kurochkin wrote: +> > > On Wed, 01 Feb 2012 12:18:08 +0200, Tomi Ollila wrote: +> > > > +> > > > There are at least these options here +> > > > +> > > > 1) go through all ~100 places where test_expect_equal_file is used +> > > > and fix the call order: quick look tells that the offending uses +> > > > are in dump-restore, hooks, search-limiting and symbol-hiding. +> > > > +> > > > 2) enforce "expected" filename has some format *and* fix all current +> > > > uses of it. Add testbed_error () function which yells loudly ane exits... +> > > > +> > > > 3) guess which is output and which is expected from args so that +> > > > machine helps tester here (for both diff output & copied files)a +> > > > +> > > > 4) just copy compared files to some directory, those are named as +> > > > basename of the original -- diff order still inconsistent. +> > > > +> > > > +> > > > I'd just go with option 1 and fix new *violations* when stumble upon one. +> > > > +> > > +> > > Option 1 does not solve the problem. New violations would apper and +> > > need to be fixed again. I am for option 2. +> > +> > How is enforcing use of a particular filename better and more robust +> > than enforcing argument order? +> +> Filename check is a way to make sure the argument order is correct. +> +> > You'll still have to force an arbitrary +> > heuristic. And you'll still be vulnerable to people messing up the file +> > names (which actually seems easier to get wrong than messing up the +> > order). +> +> Do you mean that people would start writing tests with filenames like: +> +> test_expect_equal_file EXPECTED1 EXPECTED2 +> +> ? That is possible, of course. But do you seriously believe that +> deliberately changing file names in a way that violates common sense and +> is inconsistent with all other code is "easier" than writing "EXPECTED +> OUTPUT" in the wrong order? I do not think so. And it would definately +> be easier to catch during review. +> +> > And you'll have to have more code to parse the argument +> > strings. +> +> That is one case statement, with one non-empty case, with one error +> call. +> +> > And you'll still get inconsistent diffs. +> > +> +> No, you do not. That is the point. +> +> > If this is really a problem, I vote for 1. In general, I am not in +> > favor of making the test suite more complicated than it needs to be. +> > +> +> Ok. I plan to send a patch soon. If my arguments do not convince +> enough people, I will move along. +> + +The new patches, based on idea suggested by Tomi [1], are here: + + id:"1328141050-30356-1-git-send-email-dmitry.kurochkin@gmail.com" + +Regards, + Dmitry + +[1] id:"m28vknaq5l.fsf@guru.guru-group.fi" + +> Regards, +> Dmitry +> +> > jamie. -- 2.26.2