[PATCH 1/2] test: add check for <expected> filename argument for test_expect_equal_file

diff --git a/1e/9116be2ac705126b3f4600d02915789a872f34 b/1e/9116be2ac705126b3f4600d02915789a872f34
new file mode 100644 (file)
index 0000000..59daa99
--- /dev/null
+++ b/1e/9116be2ac705126b3f4600d02915789a872f34
@@ -0,0 +1,117 @@
+Return-Path: <dmitry.kurochkin@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 94FEE431FAF\r
+       for <notmuch@notmuchmail.org>; Wed,  1 Feb 2012 16:05:29 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id rSNzXOSjC65R for <notmuch@notmuchmail.org>;\r
+       Wed,  1 Feb 2012 16:05:29 -0800 (PST)\r
+Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com\r
+       [209.85.214.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id E7AE9431FAE\r
+       for <notmuch@notmuchmail.org>; Wed,  1 Feb 2012 16:05:28 -0800 (PST)\r
+Received: by mail-bk0-f53.google.com with SMTP id 11so1737569bke.26\r
+       for <notmuch@notmuchmail.org>; Wed, 01 Feb 2012 16:05:28 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;\r
+       h=from:to:subject:date:message-id:x-mailer:in-reply-to:references;\r
+       bh=i76hdNGeWU6YSpkK7FOJa2vXgmsRE1aioGTE8R2OmLk=;\r
+       b=xV5EMcEpfd20UATDj0ylKwZ4exwGk8Ws1VfxVsh6+j37Kwk2pmZusagKKsKhu4fgnM\r
+       ThUewoz/DELnaBEmCxBp6gAuXQERTyL/iaL9btYY0O47p+956BppJQtfDqLnfIxlHtge\r
+       BbBn8yaxV8AnWT2gdZxlB+6fLOdDgKLkm7PNo=\r
+Received: by 10.204.136.220 with SMTP id s28mr233826bkt.59.1328141128681;\r
+       Wed, 01 Feb 2012 16:05:28 -0800 (PST)\r
+Received: from localhost ([91.144.186.21])\r
+       by mx.google.com with ESMTPS id bw9sm1562946bkb.8.2012.02.01.16.05.27\r
+       (version=TLSv1/SSLv3 cipher=OTHER);\r
+       Wed, 01 Feb 2012 16:05:28 -0800 (PST)\r
+From: Dmitry Kurochkin <dmitry.kurochkin@gmail.com>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 1/2] test: add check for <expected> filename argument for\r
+       test_expect_equal_file\r
+Date: Thu,  2 Feb 2012 04:04:09 +0400\r
+Message-Id: <1328141050-30356-2-git-send-email-dmitry.kurochkin@gmail.com>\r
+X-Mailer: git-send-email 1.7.9\r
+In-Reply-To: <1328141050-30356-1-git-send-email-dmitry.kurochkin@gmail.com>\r
+References: <1328141050-30356-1-git-send-email-dmitry.kurochkin@gmail.com>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 02 Feb 2012 00:05:29 -0000\r
+\r
+Test_expect_equal_file() function treats the first argument as "actual\r
+output file" and the second argument as "expected output file".  When\r
+the test fails, the files are copied for later inspection.  The first\r
+files is copied to "$testname.output" and the second file to\r
+"$testname.expected".  The argument order for test_expect_equal_file()\r
+is often wrong which results in confusing diff output and incorrectly\r
+named files.\r
+\r
+The patch requires the <expected> filename argument for\r
+test_expect_equal_file() function to contain 'EXPECT' or 'expect'\r
+substring.  Otherwise test fails with appropriate error.  This allows\r
+us to easily catch argument order errors during test development.\r
+\r
+Most tests already comply with the required format.  But there are\r
+some tests which are broken by this change.  Notably, the tests which\r
+have wrong argument order for test_expect_equal_file() fail after the\r
+change, which kind of proofs the idea.\r
+---\r
+ test/README      |    3 ++-\r
+ test/test-lib.sh |    8 ++++++++\r
+ 2 files changed, 10 insertions(+), 1 deletions(-)\r
+\r
+diff --git a/test/README b/test/README\r
+index 43656a3..0105019 100644\r
+--- a/test/README\r
++++ b/test/README\r
+@@ -181,7 +181,8 @@ library for your script to use.\r
+    Identical to test_exepect_equal, except that <output> and\r
+    <expected> are files instead of strings.  This is a much more\r
+    robust method to compare formatted textual information, since it\r
+-   also notices whitespace and closing newline differences.\r
++   also notices whitespace and closing newline differences.  The\r
++   <expected> file name must contain 'expect' or 'EXPECT' substring.\r
+ \r
+  test_debug <script>\r
+ \r
+diff --git a/test/test-lib.sh b/test/test-lib.sh\r
+index 8158328..482f47b 100644\r
+--- a/test/test-lib.sh\r
++++ b/test/test-lib.sh\r
+@@ -490,6 +490,14 @@ test_expect_equal_file ()\r
+ \r
+       output="$1"\r
+       expected="$2"\r
++      case "$expected" in\r
++      *expect*|*EXPECT*)\r
++              ;;\r
++      *)\r
++              error "bug in the test script: test_expect_equal_file parameter '$expected' does not contain 'expect' substring"\r
++              ;;\r
++      esac\r
++\r
+       if ! test_skip "$test_subtest_name"\r
+       then\r
+               if diff -q "$expected" "$output" >/dev/null ; then\r
+-- \r
+1.7.9\r
+\r