From: Dmitry Kurochkin Date: Wed, 1 Feb 2012 09:19:31 +0000 (+0400) Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=288f627957fc7a565de36a471f9672e538ac2921;p=notmuch-archives.git Re: [PATCH] test: make test_expect_equal_file() arguments flexible --- diff --git a/2e/36b8a69d5c020ad3793ba55702b9407bb0f1a4 b/2e/36b8a69d5c020ad3793ba55702b9407bb0f1a4 new file mode 100644 index 000000000..49bfb3700 --- /dev/null +++ b/2e/36b8a69d5c020ad3793ba55702b9407bb0f1a4 @@ -0,0 +1,104 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id C1BDC431E64 + for ; Wed, 1 Feb 2012 01:20:46 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.799 +X-Spam-Level: +X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 1azBxmEmJtKT for ; + Wed, 1 Feb 2012 01:20:46 -0800 (PST) +Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com + [209.85.214.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 05BE7431FAF + for ; Wed, 1 Feb 2012 01:20:45 -0800 (PST) +Received: by bke11 with SMTP id 11so901115bke.26 + for ; Wed, 01 Feb 2012 01:20:44 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; + h=from:to:subject:in-reply-to:references:user-agent:date:message-id + :mime-version:content-type; + bh=H600vKT2fagxgcwkG1GZQp28qtdZNK1F9D0NHGxh/Uc=; + b=slKUfpPAUIdGL+iVqJFzU2/0AH2+yBbG/QtySXn8HJkP8SbogdjhlbhxID+Q3cDu/x + tcAvl6fPd38xCglPCeBWrR9KT5f4r7PMblOJv38aw8l8g6lFgRqHQUrgKjfOQT4cWktf + iNy+1yG9B7f6YK37OFIBt/1AFNbrCwLyH2nL0= +Received: by 10.205.137.4 with SMTP id im4mr12725334bkc.136.1328088044615; + Wed, 01 Feb 2012 01:20:44 -0800 (PST) +Received: from localhost ([91.144.186.21]) + by mx.google.com with ESMTPS id x20sm37315973bka.9.2012.02.01.01.20.43 + (version=TLSv1/SSLv3 cipher=OTHER); + Wed, 01 Feb 2012 01:20:43 -0800 (PST) +From: Dmitry Kurochkin +To: Jameson Graef Rollins , + notmuch@notmuchmail.org +Subject: Re: [PATCH] test: make test_expect_equal_file() arguments flexible +In-Reply-To: <87r4yfszx9.fsf@servo.finestructure.net> +References: <1328080794-24670-1-git-send-email-dmitry.kurochkin@gmail.com> + <87r4yfszx9.fsf@servo.finestructure.net> +User-Agent: Notmuch/0.11+139~gd9b7cab (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Wed, 01 Feb 2012 13:19:31 +0400 +Message-ID: <87pqdync64.fsf@gmail.com> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 01 Feb 2012 09:20:46 -0000 + +On Wed, 01 Feb 2012 00:47:30 -0800, Jameson Graef Rollins wrote: +> On Wed, 1 Feb 2012 11:19:54 +0400, Dmitry Kurochkin wrote: +> > The down side of this approach is that diff argument order depends on +> > test_expect_equal_file() argument order. So sometimes we get diff +> > from expected to actual results, and sometimes the other way around. +> > But the files are always named correctly. +> +> Actually, I think this last point is the most important thing to retain. +> Consistency in the diffs makes reading test results much more efficient. +> The order I don't much care about. But seeing as we have been +> consistent with a particular order for a while, it seems like more +> effort than it's worth to change it. +> + +It is not true that we are consistent with test_expect_equal_file() +argument order. If we were, I would not bother. The problem is we are +not. I remember that we already fixed argument order for +test_expect_equal() and/or test_expect_equal_file(). If we do not solve +this problem, we should make it a tradition. + +Consistent diff would be good. But IMO the current situation is worse: +we are *supposed* to have consistent diff output, but in reality we have +messed diff output. + +Also please consider the following points: + +* Usually one is looking at a single failing test. So it is not like + you have a series of inconsistent diffs. + +* I personally can not remember the argument and diff order. So each + time I need to understand the diff, I look at the beginning to see + which side is where anyway. + +So IMHO diff order is not that important. But I would like to see a +better solution. Perhaps Tomi's proposal would be the one. + +Regards, + Dmitry + +> jamie.