Return-Path: X-Original-To: notmuch@notmuchmail.org Delivered-To: notmuch@notmuchmail.org Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id AAEEF429E32 for ; Tue, 28 Jun 2011 15:11:37 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.799 X-Spam-Level: X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YnEUTjE86r7J for ; Tue, 28 Jun 2011 15:11:36 -0700 (PDT) Received: from mail-iw0-f181.google.com (mail-iw0-f181.google.com [209.85.214.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 1F288429E55 for ; Tue, 28 Jun 2011 15:11:36 -0700 (PDT) Received: by iwn37 with SMTP id 37so580741iwn.26 for ; Tue, 28 Jun 2011 15:11:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=from:to:cc:subject:in-reply-to:references:user-agent:date :message-id:mime-version:content-type; bh=gluxTS6CLFklOomDOrP87inZpN/PCS/2wGcs3uyeslc=; b=sMiGI68vodiJfaQ8rgZ3U1BFUA+JsUPFWpgaFN4xbQX/sddDIs1u27b9aexE67exbC f0Bt+PKTskLebaHdUmIjMHUGtPpMPfxr21h5JGgtbAnQ/d2iYlsLTyaK4czC5r4wni1Y zuBsKEbWbuDV0x1w+QGHc0WszDxb1smFKr4g0= Received: by 10.231.166.142 with SMTP id m14mr56341iby.116.1309299095302; Tue, 28 Jun 2011 15:11:35 -0700 (PDT) Received: from localhost ([74.205.145.146]) by mx.google.com with ESMTPS id gb8sm274825ibb.26.2011.06.28.15.11.33 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 28 Jun 2011 15:11:34 -0700 (PDT) From: Mark Anderson To: Carl Worth , Mark Anderson , Austin Clements Subject: [PATCH v3] test:Improve test behaviors when --root is used In-Reply-To: <87oc1hir73.fsf@gmail.com> References: <87vcvrw91f.fsf@gmail.com> <3wd39iuubbs.fsf@testarossa.amd.com> <87sjqt8zvp.fsf@yoom.home.cworth.org> <87oc1hir73.fsf@gmail.com> User-Agent: Notmuch/0.5-283-gb744eac (http://notmuchmail.org) Emacs/23.2.1 (i686-pc-linux-gnu) Date: Tue, 28 Jun 2011 16:11:32 -0600 Message-ID: <87liwlip2j.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "notmuch@notmuchmail.org" X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Jun 2011 22:11:37 -0000 Change add_email_corpus, emacs_deliver_message and tests to use $TEST_DIRECTORY instead of '..'. This improves the behavior of the usage of --root=, as the assumption of what '..' means will usually be incorrect. Document -root option in README and update valgrind to work with -root. --- This patch actually fixes what Austin pointed out. test/README | 9 +++++++++ test/basic | 10 +++++----- test/crypto | 2 +- test/emacs | 4 ++-- test/symbol-hiding | 4 ++-- test/test-lib.sh | 18 +++++++++--------- 6 files changed, 28 insertions(+), 19 deletions(-) diff --git a/test/README b/test/README index be75e0e..8fbf78d 100644 --- a/test/README +++ b/test/README @@ -41,6 +41,15 @@ The following command-line options are available when running tests: As the names depend on the tests' file names, it is safe to run the tests with this option in parallel. +--root=:: + This runs the testsuites specified under a seperate directory. + However, caution is advised, as not all tests are maintained + with this relocation in mind, so some tests may behave + differently. + + Pointing this argument at a tmpfs filesystem can improve the + speed of the test suite for some users. + When invoking the test suite via "make test" any of the above options can be specified as follows: diff --git a/test/basic b/test/basic index d6e8c10..33bf711 100755 --- a/test/basic +++ b/test/basic @@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to fail' ' # Ensure that all tests are being run test_begin_subtest 'Ensure that all available tests will be run by notmuch-test' -eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test ../notmuch-test) +eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test $TEST_DIRECTORY/notmuch-test) tests_in_suite=$(for i in $TESTS; do echo $i; done | sort) -available=$(ls -1 ../ | \ +available=$(ls -1 $TEST_DIRECTORY/ | \ sed -r -e "/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d" \ -e "/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d" \ -e "/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d" \ @@ -63,19 +63,19 @@ available=$(ls -1 ../ | \ | sort) test_expect_equal "$tests_in_suite" "$available" -EXPECTED=../test.expected-output +EXPECTED=$TEST_DIRECTORY/test.expected-output suppress_diff_date() { sed -e 's/$.*\-\-\- test-verbose\.4\.\expected$.*/\1/' \ -e 's/$.*\+\+\+ test-verbose\.4\.\output$.*/\1/' } test_begin_subtest "Ensure that test output is suppressed unless the test fails" -output=$(cd ..; ./test-verbose 2>&1 | suppress_diff_date) +output=$(cd $TEST_DIRECTORY; ./test-verbose 2>&1 | suppress_diff_date) expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date) test_expect_equal "$output" "$expected" test_begin_subtest "Ensure that -v does not suppress test output" -output=$(cd ..; ./test-verbose -v 2>&1 | suppress_diff_date) +output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date) expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date) # Do not include the results of test-verbose in totals rm $TEST_DIRECTORY/test-results/test-verbose-* diff --git a/test/crypto b/test/crypto index 01daffe..7eb3559 100755 --- a/test/crypto +++ b/test/crypto @@ -12,7 +12,7 @@ add_gnupg_home () local output [ -d ${GNUPGHOME} ] && return mkdir -m 0700 "$GNUPGHOME" - gpg --no-tty --import <../gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1 + gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1 test_debug "cat $GNUPGHOME/import.log" if (gpg --quick-random --version >/dev/null 2>&1) ; then echo quick-random >> "$GNUPGHOME"/gpg.conf diff --git a/test/emacs b/test/emacs index 6f82b08..f91078e 100755 --- a/test/emacs +++ b/test/emacs @@ -2,7 +2,7 @@ test_description="emacs interface" . test-lib.sh -EXPECTED=../emacs.expected-output +EXPECTED=$TEST_DIRECTORY/emacs.expected-output add_email_corpus @@ -81,7 +81,7 @@ mkdir -p mail/sent/cur mkdir -p mail/sent/new mkdir -p mail/sent/tmp -../smtp-dummy sent_message & +$TEST_DIRECTORY/smtp-dummy sent_message & smtp_dummy_pid=$! test_emacs "(setq message-send-mail-function 'message-smtpmail-send-it) (setq smtpmail-smtp-server \"localhost\") (setq smtpmail-smtp-service \"25025\") (notmuch-hello) (notmuch-mua-mail) (message-goto-to) (insert \"user@example.com\nDate: Fri, 29 Mar 1974 10:00:00 -0000\") (message-goto-subject) (insert \"Testing message sent via SMTP\") (message-goto-body) (insert \"This is a test that messages are sent via SMTP\") (message-send-and-exit)" >/dev/null 2>&1 wait ${smtp_dummy_pid} diff --git a/test/symbol-hiding b/test/symbol-hiding index bb55524..d0b31ae 100755 --- a/test/symbol-hiding +++ b/test/symbol-hiding @@ -12,13 +12,13 @@ test_description='exception symbol hiding' . ./test-lib.sh run_test(){ - result=$(LD_LIBRARY_PATH=../../lib ./symbol-test 2>&1) + result=$(LD_LIBRARY_PATH=$TEST_DIRECTORY/../lib ./symbol-test 2>&1) } output="A Xapian exception occurred opening database: Couldn't stat 'fakedb/.notmuch/xapian' caught No chert database found at path \`./nonexistant'" -g++ -o symbol-test -I../../lib ../symbol-test.cc -L../../lib -lnotmuch -lxapian +g++ -o symbol-test -I$TEST_DIRECTORY/../lib $TEST_DIRECTORY/symbol-test.cc -L$TEST_DIRECTORY/../lib -lnotmuch -lxapian mkdir -p fakedb/.notmuch test_expect_success 'running test' run_test test_begin_subtest 'checking output' diff --git a/test/test-lib.sh b/test/test-lib.sh index cc20f41..352be39 100755 --- a/test/test-lib.sh +++ b/test/test-lib.sh @@ -392,7 +392,7 @@ emacs_deliver_message () shift 2 # before we can send a message, we have to prepare the FCC maildir mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} - ../smtp-dummy sent_message & + $TEST_DIRECTORY/smtp-dummy sent_message & smtp_dummy_pid=$! test_emacs "(setq message-send-mail-function 'message-smtpmail-send-it) (setq smtpmail-smtp-server \"localhost\") (setq smtpmail-smtp-service \"25025\") (notmuch-hello) (notmuch-mua-mail) (message-goto-to) (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") (message-goto-subject) (insert \"${subject}\") (message-goto-body) (insert \"${body}\") $@ (message-send-and-exit)" >/dev/null 2>&1 wait ${smtp_dummy_pid} @@ -408,12 +408,12 @@ emacs_deliver_message () add_email_corpus () { rm -rf ${MAIL_DIR} - if [ -d ../corpus.mail ]; then - cp -a ../corpus.mail ${MAIL_DIR} + if [ -d $TEST_DIRECTORY/corpus.mail ]; then + cp -a $TEST_DIRECTORY/corpus.mail ${MAIL_DIR} else - cp -a ../corpus ${MAIL_DIR} + cp -a $TEST_DIRECTORY/corpus ${MAIL_DIR} notmuch new >/dev/null - cp -a ${MAIL_DIR} ../corpus.mail + cp -a ${MAIL_DIR} $TEST_DIRECTORY/corpus.mail fi } @@ -860,8 +860,8 @@ fi # set-frame-width 80 columns (avoids crazy 10-column default of --batch) emacs \$BATCH --no-init-file --no-site-file \ - --directory ../../emacs --load notmuch.el \ - --directory .. --load test-lib.el \ + --directory $TEST_DIRECTORY/../emacs --load notmuch.el \ + --directory $TEST_DIRECTORY --load test-lib.el \ --eval "(defun notmuch-test-wait () (while (get-buffer-process (current-buffer)) (sleep-for 0.1)))" \ @@ -925,11 +925,11 @@ then test ! -d "$symlink_target" && test "#!" != "$(head -c 2 < "$symlink_target")" then - symlink_target=../valgrind.sh + symlink_target=$TEST_DIRECTORY/valgrind.sh fi case "$base" in *.sh|*.perl) - symlink_target=../unprocessed-script + symlink_target=$TEST_DIRECTORY/unprocessed-script esac # create the link, or replace it if it is out of date make_symlink "$symlink_target" "$GIT_VALGRIND/bin/$base" || exit -- 1.7.4.1