--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id DD988431FD7\r
+ for <notmuch@notmuchmail.org>; Tue, 12 Nov 2013 15:03:08 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id thGm7jeXgGLB for <notmuch@notmuchmail.org>;\r
+ Tue, 12 Nov 2013 15:03:03 -0800 (PST)\r
+Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu\r
+ [18.9.25.12])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id E9A74431FCF\r
+ for <notmuch@notmuchmail.org>; Tue, 12 Nov 2013 15:03:02 -0800 (PST)\r
+X-AuditID: 1209190c-b7f7f6d000000bbd-e8-5282b3a6c922\r
+Received: from mailhub-auth-3.mit.edu ( [18.9.21.43])\r
+ (using TLS with cipher AES256-SHA (256/256 bits))\r
+ (Client did not present a certificate)\r
+ by dmz-mailsec-scanner-1.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 22.0B.03005.6A3B2825; Tue, 12 Nov 2013 18:03:02 -0500 (EST)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+ by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id rACN31Xb027243; \r
+ Tue, 12 Nov 2013 18:03:01 -0500\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id rACN2xvS028631\r
+ (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+ Tue, 12 Nov 2013 18:03:00 -0500\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1VgMz5-000784-AB; Tue, 12 Nov 2013 18:02:59 -0500\r
+Date: Tue, 12 Nov 2013 18:02:59 -0500\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Tomi Ollila <tomi.ollila@iki.fi>\r
+Subject: Re: [PATCH 3/3] test: implement and document NOTMUCH_TEST_QUIET\r
+ variable usage\r
+Message-ID: <20131112230259.GD13399@mit.edu>\r
+References: <1384288868-23903-1-git-send-email-tomi.ollila@iki.fi>\r
+ <1384288868-23903-3-git-send-email-tomi.ollila@iki.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <1384288868-23903-3-git-send-email-tomi.ollila@iki.fi>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFmphleLIzCtJLcpLzFFi42IR4hTV1l22uSnI4OkLYYvrN2cyW7xZOY/V\r
+ gcnj8NeFLB7PVt1iDmCK4rJJSc3JLEst0rdL4Mr42/iasWCXfsWWzauYGxh71LoYOTkkBEwk\r
+ dn15wgxhi0lcuLeerYuRi0NIYDaTxNmpDewQzkZGiTdXzjFBOKeZJG7de8kC4SxhlPg+/SA7\r
+ SD+LgKpEw9JbjCA2m4CGxLb9y8FsEQEViQdt61lBbGYBaYlvv5uZQGxhgUiJN3c3gu3mFdCR\r
+ mDypmwXEFhKolpgw9SAjRFxQ4uTMJywQvVoSN/69BOrlAJuz/B8HSJhTwFli/qH3YOWiQKum\r
+ nNzGNoFRaBaS7llIumchdC9gZF7FKJuSW6Wbm5iZU5yarFucnJiXl1qka6iXm1mil5pSuokR\r
+ HNiSPDsY3xxUOsQowMGoxMP7ILIpSIg1say4MvcQoyQHk5Iob8RaoBBfUn5KZUZicUZ8UWlO\r
+ avEhRgkOZiURXuk1QDnelMTKqtSifJiUNAeLkjjvTQ77ICGB9MSS1OzU1ILUIpisDAeHkgTv\r
+ qU1AjYJFqempFWmZOSUIaSYOTpDhPEDDm0BqeIsLEnOLM9Mh8qcYFaXEeR+BJARAEhmleXC9\r
+ sMTzilEc6BVh3h0gVTzApAXX/QpoMBPQYItisMEliQgpqQZG6xfN++aHXr29luNhi7bQJWmh\r
+ 9RErmIrZ7D52mvW/nJY3xZ1ly+QOSea7Beci1SVY79w/2po4OWext28fZ2yz8U5hr0l7PZ53\r
+ Me5v+p4UIv5FPM2s5aLbkk1GLzXfvLAucAra9eXeCpHtpcUaMpbHl30RLzobvKZb+LvsvybX\r
+ 5QquTy91mT1QYinOSDTUYi4qTgQAI7vsExcDAAA=\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 12 Nov 2013 23:03:09 -0000\r
+\r
+Quoth Tomi Ollila on Nov 12 at 10:41 pm:\r
+> When NOTMUCH_TEST_QUIET environment variable is set to non-null value\r
+> messages when new test script starts and when test PASSes are disabled.\r
+> This eases picking the cases when tests FAIL (as those are still printed).\r
+> ---\r
+> test/README | 8 ++++++++\r
+> test/basic | 12 ++++++++++--\r
+> test/test-lib.sh | 11 ++++++++++-\r
+> test/test.expected-output/test-quiet-verbose-no | 20 ++++++++++++++++++++\r
+> test/test.expected-output/test-quiet-verbose-yes | 24 ++++++++++++++++++++++++\r
+> 5 files changed, 72 insertions(+), 3 deletions(-)\r
+> create mode 100644 test/test.expected-output/test-quiet-verbose-no\r
+> create mode 100644 test/test.expected-output/test-quiet-verbose-yes\r
+> \r
+> diff --git a/test/README b/test/README\r
+> index d12cff2..79a9b1b 100644\r
+> --- a/test/README\r
+> +++ b/test/README\r
+> @@ -76,6 +76,14 @@ the tests in one of the following ways.\r
+> TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient ./emacs\r
+> make test TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient\r
+> \r
+> +Quiet Execution\r
+> +---------------\r
+> +\r
+> +Normally, when new script starts and when test PASSes you get a message\r
+> +printed on screen. This printing can be disabled by setting the\r
+> +NOTMUCH_TEST_QUIET variable to a non-null value. Message on test\r
+> +failures and skips are still printed.\r
+> +\r
+> Skipping Tests\r
+> --------------\r
+> If, for any reason, you need to skip one or more tests, you can do so\r
+> diff --git a/test/basic b/test/basic\r
+> index 64eb7d7..3b7668b 100755\r
+> --- a/test/basic\r
+> +++ b/test/basic\r
+> @@ -73,14 +73,22 @@ suppress_diff_date() {\r
+> -e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'\r
+> }\r
+> \r
+> +if [ -z "$NOTMUCH_TEST_QUIET" ]\r
+> +then\r
+> + test_verbose_no=$EXPECTED/test-verbose-no\r
+> + test_verbose_yes=$EXPECTED/test-verbose-yes\r
+> +else\r
+> + test_verbose_no=$EXPECTED/test-quiet-verbose-no\r
+> + test_verbose_yes=$EXPECTED/test-quiet-verbose-yes\r
+> +fi\r
+> test_begin_subtest "Ensure that test output is suppressed unless the test fails"\r
+> output=$(cd $TEST_DIRECTORY; ./test-verbose 2>&1 | suppress_diff_date)\r
+> -expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)\r
+> +expected=$(cat ${test_verbose_no} | suppress_diff_date)\r
+> test_expect_equal "$output" "$expected"\r
+> \r
+> test_begin_subtest "Ensure that -v does not suppress test output"\r
+> output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)\r
+> -expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)\r
+> +expected=$(cat ${test_verbose_yes} | suppress_diff_date)\r
+> # Do not include the results of test-verbose in totals\r
+> rm $TEST_DIRECTORY/test-results/test-verbose\r
+> rm -r $TEST_DIRECTORY/tmp.test-verbose\r
+> diff --git a/test/test-lib.sh b/test/test-lib.sh\r
+> index e022e46..4b342ac 100644\r
+> --- a/test/test-lib.sh\r
+> +++ b/test/test-lib.sh\r
+> @@ -195,7 +195,10 @@ print_test_description ()\r
+> echo\r
+> echo $this_test: "Testing ${test_description}"\r
+> }\r
+> -print_test_description\r
+> +if [ -z "$NOTMUCH_TEST_QUIET" ]\r
+> +then\r
+> + print_test_description\r
+> +fi\r
+> \r
+> exec 5>&1\r
+> \r
+> @@ -703,6 +706,9 @@ test_ok_ () {\r
+> return\r
+> fi\r
+> test_success=$(($test_success + 1))\r
+> + if test -n "$NOTMUCH_TEST_QUIET"; then\r
+> + return 0\r
+> + fi\r
+> say_color pass "%-6s" "PASS"\r
+> echo " $test_subtest_name"\r
+> }\r
+> @@ -713,6 +719,9 @@ test_failure_ () {\r
+> return\r
+> fi\r
+> test_failure=$(($test_failure + 1))\r
+> + if test -n "$NOTMUCH_TEST_QUIET"; then\r
+> + print_test_description\r
+\r
+This prints the test description for *every* failing test. Was that\r
+intentional? I would think that, ideally, it would be only printed\r
+before the first failing subtest in a test (maybe by setting a\r
+variable in print_test_description on the first call and making it\r
+return immediately if this variable is set? Then you wouldn't even\r
+need the condition here, just the call to print_test_description.)\r
+\r
+> + fi\r
+> test_failure_message_ "FAIL" "$test_subtest_name" "$@"\r
+> test "$immediate" = "" || { GIT_EXIT_OK=t; exit 1; }\r
+> return 1\r
+\r
+Stylistic nit: The three if's above use two different styles ([ vs\r
+test and hanging 'then'). OTOH, maybe this is consistent with\r
+test-lib's inconsistent style.\r
+\r
+> diff --git a/test/test.expected-output/test-quiet-verbose-no b/test/test.expected-output/test-quiet-verbose-no\r
+> new file mode 100644\r
+> index 0000000..74840b9\r
+> --- /dev/null\r
+> +++ b/test/test.expected-output/test-quiet-verbose-no\r
+> @@ -0,0 +1,20 @@\r
+> +\r
+> +test-verbose: Testing the verbosity options of the test framework itself.\r
+> + FAIL print something in test_expect_success and fail\r
+> + \r
+> + echo "hello stdout" &&\r
+> + echo "hello stderr" >&2 &&\r
+> + false\r
+> + \r
+> +hello stdout\r
+> +hello stderr\r
+> +\r
+> +test-verbose: Testing the verbosity options of the test framework itself.\r
+> + FAIL print something test_begin_subtest and test_expect_equal and fail\r
+> + --- test-verbose.4.expected\r
+> + +++ test-verbose.4.output\r
+> + @@ -1 +1 @@\r
+> + -b\r
+> + +a\r
+> +hello stdout\r
+> +hello stderr\r
+> diff --git a/test/test.expected-output/test-quiet-verbose-yes b/test/test.expected-output/test-quiet-verbose-yes\r
+> new file mode 100644\r
+> index 0000000..51e759d\r
+> --- /dev/null\r
+> +++ b/test/test.expected-output/test-quiet-verbose-yes\r
+> @@ -0,0 +1,24 @@\r
+> +hello stdout\r
+> +hello stderr\r
+> +hello stdout\r
+> +hello stderr\r
+> +\r
+> +test-verbose: Testing the verbosity options of the test framework itself.\r
+> + FAIL print something in test_expect_success and fail\r
+> + \r
+> + echo "hello stdout" &&\r
+> + echo "hello stderr" >&2 &&\r
+> + false\r
+> + \r
+> +hello stdout\r
+> +hello stderr\r
+> +hello stdout\r
+> +hello stderr\r
+> +\r
+> +test-verbose: Testing the verbosity options of the test framework itself.\r
+> + FAIL print something test_begin_subtest and test_expect_equal and fail\r
+> + --- test-verbose.4.expected\r
+> + +++ test-verbose.4.output\r
+> + @@ -1 +1 @@\r
+> + -b\r
+> + +a\r
projects / notmuch-archives.git / commitdiff