From: Dmitry Kurochkin Date: Wed, 1 Feb 2012 23:42:29 +0000 (+0400) Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=69ded891c2e6646d02fa0f5919919964d37353c4;p=notmuch-archives.git Re: [PATCH] test: make test_expect_equal_file() arguments flexible --- diff --git a/c6/641de3c5921b5bdb1b54133ba6cafa0f177519 b/c6/641de3c5921b5bdb1b54133ba6cafa0f177519 new file mode 100644 index 000000000..1100ae877 --- /dev/null +++ b/c6/641de3c5921b5bdb1b54133ba6cafa0f177519 @@ -0,0 +1,136 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 072E2431FAF + for ; Wed, 1 Feb 2012 15:43:46 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.799 +X-Spam-Level: +X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id tubsxMjO-Pn1 for ; + Wed, 1 Feb 2012 15:43:45 -0800 (PST) +Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com + [209.85.214.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id B7C1E431FAE + for ; Wed, 1 Feb 2012 15:43:44 -0800 (PST) +Received: by bke11 with SMTP id 11so1724156bke.26 + for ; Wed, 01 Feb 2012 15:43:43 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; + h=from:to:subject:in-reply-to:references:user-agent:date:message-id + :mime-version:content-type; + bh=PdOAIh0skTBPmBwiGSZoatA8TzcoVcMeTX71Uq/+gLs=; + b=DAExZ9Q0pczZnWWfvqTYVNk6upq2TYlMRNGQ+4pBs8R0yjKeMqzIOIR7E14AtgwF5i + mp873sYthNCo11hm7h9IBqTceSctvQ+JmRimvot2/x3SR0pExqv/O4URYzI0uaIIem8J + RnJoCZfCtW+EtduBHmTNpQcp4uYP6p9aZOeJk= +Received: by 10.204.152.219 with SMTP id h27mr294469bkw.40.1328139823347; + Wed, 01 Feb 2012 15:43:43 -0800 (PST) +Received: from localhost ([91.144.186.21]) + by mx.google.com with ESMTPS id ci12sm1381438bkb.13.2012.02.01.15.43.41 + (version=TLSv1/SSLv3 cipher=OTHER); + Wed, 01 Feb 2012 15:43:42 -0800 (PST) +From: Dmitry Kurochkin +To: Jameson Graef Rollins , + Tomi Ollila , notmuch@notmuchmail.org +Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible +In-Reply-To: <87ehuetqjz.fsf@servo.finestructure.net> +References: <1328080794-24670-1-git-send-email-dmitry.kurochkin@gmail.com> + <87r4yfszx9.fsf@servo.finestructure.net> + <87pqdync64.fsf@gmail.com> + <87k446n8ji.fsf@gmail.com> <87ehuetqjz.fsf@servo.finestructure.net> +User-Agent: Notmuch/0.11+139~gd9b7cab (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Thu, 02 Feb 2012 03:42:29 +0400 +Message-ID: <8739aum87u.fsf@gmail.com> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 01 Feb 2012 23:43:46 -0000 + +Hi Jameson. + +On Wed, 01 Feb 2012 09:24:32 -0800, Jameson Graef Rollins wrote: +> On Wed, 01 Feb 2012 14:37:53 +0400, Dmitry Kurochkin wrote: +> > On Wed, 01 Feb 2012 12:18:08 +0200, Tomi Ollila wrote: +> > > +> > > There are at least these options here +> > > +> > > 1) go through all ~100 places where test_expect_equal_file is used +> > > and fix the call order: quick look tells that the offending uses +> > > are in dump-restore, hooks, search-limiting and symbol-hiding. +> > > +> > > 2) enforce "expected" filename has some format *and* fix all current +> > > uses of it. Add testbed_error () function which yells loudly ane exits... +> > > +> > > 3) guess which is output and which is expected from args so that +> > > machine helps tester here (for both diff output & copied files)a +> > > +> > > 4) just copy compared files to some directory, those are named as +> > > basename of the original -- diff order still inconsistent. +> > > +> > > +> > > I'd just go with option 1 and fix new *violations* when stumble upon one. +> > > +> > +> > Option 1 does not solve the problem. New violations would apper and +> > need to be fixed again. I am for option 2. +> +> How is enforcing use of a particular filename better and more robust +> than enforcing argument order? + +Filename check is a way to make sure the argument order is correct. + +> You'll still have to force an arbitrary +> heuristic. And you'll still be vulnerable to people messing up the file +> names (which actually seems easier to get wrong than messing up the +> order). + +Do you mean that people would start writing tests with filenames like: + + test_expect_equal_file EXPECTED1 EXPECTED2 + +? That is possible, of course. But do you seriously believe that +deliberately changing file names in a way that violates common sense and +is inconsistent with all other code is "easier" than writing "EXPECTED +OUTPUT" in the wrong order? I do not think so. And it would definately +be easier to catch during review. + +> And you'll have to have more code to parse the argument +> strings. + +That is one case statement, with one non-empty case, with one error +call. + +> And you'll still get inconsistent diffs. +> + +No, you do not. That is the point. + +> If this is really a problem, I vote for 1. In general, I am not in +> favor of making the test suite more complicated than it needs to be. +> + +Ok. I plan to send a patch soon. If my arguments do not convince +enough people, I will move along. + +Regards, + Dmitry + +> jamie.