Re: [PATCH 3/3] test: implement and document NOTMUCH_TEST_QUIET variable usage
authorAustin Clements <amdragon@MIT.EDU>
Wed, 4 Dec 2013 20:15:07 +0000 (15:15 +1900)
committerW. Trevor King <wking@tremily.us>
Fri, 7 Nov 2014 17:58:37 +0000 (09:58 -0800)
ac/7e103d438da3295e761b2d2efe534170f1958e [new file with mode: 0644]

diff --git a/ac/7e103d438da3295e761b2d2efe534170f1958e b/ac/7e103d438da3295e761b2d2efe534170f1958e
new file mode 100644 (file)
index 0000000..bc304dd
--- /dev/null
@@ -0,0 +1,204 @@
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 20147431FD2\r
+       for <notmuch@notmuchmail.org>; Wed,  4 Dec 2013 12:15:21 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id yTSzDuYJ03tz for <notmuch@notmuchmail.org>;\r
+       Wed,  4 Dec 2013 12:15:13 -0800 (PST)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu\r
+       [18.9.25.15])\r
+       (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 1F89D431FD0\r
+       for <notmuch@notmuchmail.org>; Wed,  4 Dec 2013 12:15:13 -0800 (PST)\r
+X-AuditID: 1209190f-b7fb86d000000c36-e5-529f8d5014b2\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+       (using TLS with cipher AES256-SHA (256/256 bits))\r
+       (Client did not present a certificate)\r
+       by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id C5.4B.03126.05D8F925; Wed,  4 Dec 2013 15:15:12 -0500 (EST)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+       by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id rB4KFAb2009161; \r
+       Wed, 4 Dec 2013 15:15:11 -0500\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+       (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id rB4KF7u4012916\r
+       (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+       Wed, 4 Dec 2013 15:15:10 -0500\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+       (envelope-from <amdragon@mit.edu>)\r
+       id 1VoIqh-0001pe-DK; Wed, 04 Dec 2013 15:15:07 -0500\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Tomi Ollila <tomi.ollila@iki.fi>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH 3/3] test: implement and document NOTMUCH_TEST_QUIET\r
+       variable usage\r
+In-Reply-To: <87vbz437yu.fsf@awakening.csail.mit.edu>\r
+References: <1384288868-23903-1-git-send-email-tomi.ollila@iki.fi>\r
+       <1385399299-12936-1-git-send-email-tomi.ollila@iki.fi>\r
+       <1385399299-12936-3-git-send-email-tomi.ollila@iki.fi>\r
+       <87vbz437yu.fsf@awakening.csail.mit.edu>\r
+User-Agent: Notmuch/0.16+154~g96c0ce2 (http://notmuchmail.org) Emacs/23.4.1\r
+       (i486-pc-linux-gnu)\r
+Date: Wed, 04 Dec 2013 15:15:07 -0500\r
+Message-ID: <87siu82x10.fsf@awakening.csail.mit.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFnrNIsWRmVeSWpSXmKPExsUixG6nrhvQOz/IYN02DovrN2cyW7xZOY/V\r
+       gcnj8NeFLB7PVt1iDmCK4rJJSc3JLEst0rdL4Mp42bOcseCMcsXGewoNjF9kuhg5OSQETCRO\r
+       3vvGDmGLSVy4t56ti5GLQ0hgNpPE/onrmCCcDYwSGxfvhcqcYpI48P88VGYJo8SKwxvA+tkE\r
+       NCS27V/OCGKLCNhKXFlykrWLkYODGWhu+7wQkLCwQLjE0r7jYCWcAqYSvSvXsoDYQgI3GSX+\r
+       rdQBsUUF4iWmLdzJDGKzCKhK/Nz2khlkDC/QqT9uxIOEeQUEJU7OfALWyiygJXHj30umCYyC\r
+       s5CkZiFJLWBkWsUom5JbpZubmJlTnJqsW5ycmJeXWqRropebWaKXmlK6iREUppyS/DsYvx1U\r
+       OsQowMGoxMPrkDI/SIg1say4MvcQoyQHk5Ior349UIgvKT+lMiOxOCO+qDQntfgQowQHs5II\r
+       778aoBxvSmJlVWpRPkxKmoNFSZz3Jod9kJBAemJJanZqakFqEUxWhoNDSYI3sQeoUbAoNT21\r
+       Ii0zpwQhzcTBCTKcB2j4qW6Q4cUFibnFmekQ+VOMilLivHUgzQIgiYzSPLheWBp5xSgO9Iow\r
+       bytIFQ8wBcF1vwIazAQ0uPnBPJDBJYkIKakGxgBFwW/2wmu6nx59HbysIX2L0owvMw2udyR1\r
+       R/mIN+YEb+mc1PVU7vbfQ1ffB+SnPKl7EmpSylNxY5kmd8KbuLPde89Pnm85Z9Gsx9nfedba\r
+       8TQ5/aq03DV9i27lZjt90zeJySc/r5ew9ZlmdjF16q4TyYtki+IZxFa+UhKxdy77u+7zxris\r
+       vUosxRmJhlrMRcWJAMA6etH+AgAA\r
+Cc: tomi.ollila@iki.fi\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 04 Dec 2013 20:15:21 -0000\r
+\r
+On Wed, 04 Dec 2013, Austin Clements <amdragon@MIT.EDU> wrote:\r
+> I just tried to use this and realized it hadn't been pushed yet.\r
+>\r
+> This series LGTM except one minor nit below and the fact that it\r
+> introduces a lot of tab-indented code in sections of test-lib.sh that\r
+> appear to be space-indented.  Given that test-lib.sh is already a mess\r
+> of indentation styles, I don't know if we care, but it would be nice if\r
+> its entropy were at least non-increasing.\r
+\r
+Ignore this comment.  Apparently I was confused by show-mode\r
+transforming leading tabs into spaces.\r
+\r
+> On Mon, 25 Nov 2013, Tomi Ollila <tomi.ollila@iki.fi> wrote:\r
+>> When NOTMUCH_TEST_QUIET environment variable is set to non-null value\r
+>> messages when new test script starts and when test PASSes are disabled.\r
+>> This eases picking the cases when tests FAIL (as those are still printed).\r
+>> ---\r
+>>  test/README      |  8 ++++++++\r
+>>  test/basic       |  4 ++--\r
+>>  test/test-lib.sh | 11 ++++++++++-\r
+>>  3 files changed, 20 insertions(+), 3 deletions(-)\r
+>>\r
+>> diff --git a/test/README b/test/README\r
+>> index d12cff2..79a9b1b 100644\r
+>> --- a/test/README\r
+>> +++ b/test/README\r
+>> @@ -74,10 +74,18 @@ the tests in one of the following ways.\r
+>>  \r
+>>     TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient make test\r
+>>     TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient ./emacs\r
+>>     make test TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient\r
+>>  \r
+>> +Quiet Execution\r
+>> +---------------\r
+>> +\r
+>> +Normally, when new script starts and when test PASSes you get a message\r
+>> +printed on screen. This printing can be disabled by setting the\r
+>> +NOTMUCH_TEST_QUIET variable to a non-null value. Message on test\r
+>> +failures and skips are still printed.\r
+>> +\r
+>>  Skipping Tests\r
+>>  --------------\r
+>>  If, for any reason, you need to skip one or more tests, you can do so\r
+>>  by setting the NOTMUCH_SKIP_TESTS variable to the name of one or more\r
+>>  sections of tests.\r
+>> diff --git a/test/basic b/test/basic\r
+>> index 64eb7d7..f7eed32 100755\r
+>> --- a/test/basic\r
+>> +++ b/test/basic\r
+>> @@ -72,16 +72,16 @@ suppress_diff_date() {\r
+>>      sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \\r
+>>     -e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'\r
+>>  }\r
+>>  \r
+>>  test_begin_subtest "Ensure that test output is suppressed unless the test fails"\r
+>> -output=$(cd $TEST_DIRECTORY; ./test-verbose 2>&1 | suppress_diff_date)\r
+>> +output=$(cd $TEST_DIRECTORY; NOTMUCH_TEST_QUIET= ./test-verbose 2>&1 | suppress_diff_date)\r
+>>  expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)\r
+>>  test_expect_equal "$output" "$expected"\r
+>>  \r
+>>  test_begin_subtest "Ensure that -v does not suppress test output"\r
+>> -output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)\r
+>> +output=$(cd $TEST_DIRECTORY; NOTMUCH_TEST_QUIET= ./test-verbose -v 2>&1 | suppress_diff_date)\r
+>>  expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)\r
+>>  # Do not include the results of test-verbose in totals\r
+>>  rm $TEST_DIRECTORY/test-results/test-verbose\r
+>>  rm -r $TEST_DIRECTORY/tmp.test-verbose\r
+>>  test_expect_equal "$output" "$expected"\r
+>> diff --git a/test/test-lib.sh b/test/test-lib.sh\r
+>> index 34e0db6..9d4a807 100644\r
+>> --- a/test/test-lib.sh\r
+>> +++ b/test/test-lib.sh\r
+>> @@ -196,11 +196,14 @@ print_test_description ()\r
+>>     test -z "$test_description_printed" || return 0\r
+>>     echo\r
+>>     echo $this_test: "Testing ${test_description}"\r
+>>     test_description_printed=1\r
+>>  }\r
+>> -print_test_description\r
+>> +if [ -z "$NOTMUCH_TEST_QUIET" ]\r
+>> +then\r
+>> +   print_test_description\r
+>> +fi\r
+>>  \r
+>>  exec 5>&1\r
+>>  \r
+>>  test_failure=0\r
+>>  test_count=0\r
+>> @@ -715,20 +718,26 @@ test_ok_ () {\r
+>>     if test "$test_subtest_known_broken_" = "t"; then\r
+>>             test_known_broken_ok_\r
+>>             return\r
+>>     fi\r
+>>     test_success=$(($test_success + 1))\r
+>> +   if test -n "$NOTMUCH_TEST_QUIET"; then\r
+>> +           return 0\r
+>> +   fi\r
+>>     say_color pass "%-6s" "PASS"\r
+>>     echo " $test_subtest_name"\r
+>>  }\r
+>>  \r
+>>  test_failure_ () {\r
+>>     if test "$test_subtest_known_broken_" = "t"; then\r
+>>             test_known_broken_failure_ "$@"\r
+>>             return\r
+>>     fi\r
+>>     test_failure=$(($test_failure + 1))\r
+>> +   if test -n "$NOTMUCH_TEST_QUIET"; then\r
+>\r
+> Strictly speaking, this test isn't necessary, right?\r
+>\r
+>> +           print_test_description\r
+>> +   fi\r
+>>     test_failure_message_ "FAIL" "$test_subtest_name" "$@"\r
+>>     test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }\r
+>>     return 1\r
+>>  }\r
+>>  \r
+>> -- \r
+>> 1.8.4.2\r