From: Pieter Praet <pieter@praet.org>
To: Thomas Jost <schnouki@schnouki.net>, notmuch@notmuchmail.org
Subject: Re: [PATCH 0/6] Rebase of Pieter's "set test prereqs"
In-Reply-To: <1321454035-22023-1-git-send-email-schnouki@schnouki.net>
References: <1320176954-4897-1-git-send-email-pieter@praet.org>
	<1321454035-22023-1-git-send-email-schnouki@schnouki.net>
User-Agent: Notmuch/0.9+76~g2fd88e6 (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-unknown-linux-gnu)
Date: Wed, 16 Nov 2011 21:50:17 +0100
Message-ID: <8739dneqra.fsf@praet.org>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
Precedence: list

--=-=-=

On Wed, 16 Nov 2011 15:33:49 +0100, Thomas Jost <schnouki@schnouki.net> wrote:
> Hello list,
> 
> This is another rebased version of Pieter's series to add GPG and Emacs as test
> prereqs, plus some additions on my own. (Rebased and posted as requested by
> Pieter [1].)
> 

Thanks Thomas!

Although... you may have misread (or maybe I mistyped :), but what I
actually intended [1] was for you to rebase *only* your fixes on top of
my rebased series (e.g. see "tjost-fixes.patch" in att), so you could
receive proper credit for cleaning up my mess.

Also, while my apprehension [2,3] re the inclusion of the SCREEN/DTACH
prereq in patches #4,5,6 didn't have much merit (it's an all-or-nothing
affair anyways), the issue [3] in patch #5 @ "Reply within emacs" still
stands: `sed' will run unconditionally, and treat "EMACS" as an input
file.  (see "sed-prereq-fix.patch" in att).


Thanks again!

> Changes as compared to Pieter's patches (including parts from [2]):
> - prereqs are not tested using test_expect_success as they were in Pieter's
>   original patches, but using a new function called test_set_bin_prereq. I wrote
>   this before the gdb prereq was added, hence the different way to set it.
> 
> - some fixes in Pieter's patches so that it actually works when gpg is not
>   installed. Can't exactly remember what (...but you can just check his original
>   patches), but in the end it was working fine in a chroot without gpg.
> 
> - since Emacs is now run using dtach, the emacs prereq also depends on dtach.
>   The presence of emacs and dtach is also checked in the test_emacs() function
>   of the test suite.
> 
> - testing for prereqs is now done using the "hash" built-in instead of "which",
>   as suggested in [3].
> 
> Tested with and without dtach. A previous version of this series was also
> without emacs/gpg in a chroot, but not this one :)
> 
> [1] id:"874ny4fcdz.fsf@praet.org"
> [2] id:"1317660447-27520-1-git-send-email-schnouki@schnouki.net"
> [3] id:"87zkgemodd.fsf@praet.org"
> 
> Pieter Praet (4):
>   test: add 'GnuPG' prereq to dependent 'crypto' tests
>   test: add 'Emacs' prereq to dependent 'crypto' tests
>   test: add 'Emacs' prereq to dependent 'emacs' tests
>   test: add 'Emacs' prereq to dependent 'emacs-large-search-buffer'
>     tests
> 
> Thomas Jost (2):
>   test: define a helper function for defining prereqs on executables
>   test: check if emacs and dtach are available in test_emacs()
> 
>  test/crypto                    |   46 +++++++++++++++-------
>  test/emacs                     |   82 +++++++++++++++++++++------------------
>  test/emacs-large-search-buffer |    9 +++-
>  test/test-lib.sh               |   17 ++++++++
>  4 files changed, 99 insertions(+), 55 deletions(-)
> 
> -- 
> 1.7.7.3
> 
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline; filename=sed-prereq-fix.patch

diff --git a/test/emacs b/test/emacs
index ea641af..79edf01 100755
--- a/test/emacs
+++ b/test/emacs
@@ -244,7 +244,8 @@ test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
 	    (notmuch-test-wait)
 	    (notmuch-search-reply-to-thread)
 	    (test-output)'
-sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' EMACS OUTPUT
+test_have_prereq EMACS \
+    && sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' OUTPUT
 cat <<EOF >EXPECTED
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: user@example.com

--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline; filename=tjost-fixes.patch

diff --git a/test/crypto b/test/crypto
index 4a00c00..085070b 100755
--- a/test/crypto
+++ b/test/crypto
@@ -8,12 +8,11 @@ test_description='PGP/MIME signature verification and decryption'
 . ./test-lib.sh
 
 # Emacs is a prereq.
-test_expect_success "prereq: Emacs is present" "which emacs" \
-    && test_set_prereq EMACS
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
 
 # GnuPG is a prereq.
-test_expect_success "prereq: GnuPG is present" "which gpg" \
-    && test_set_prereq GPG
+test_set_bin_prereq gpg "GnuPG" GPG
 
 # Some tests have multiple prereqs, but the test_expect_* functions
 # accept only a single argument as prereq tag, and using test_have_prereq
@@ -28,6 +27,7 @@ add_gnupg_home ()
     local output
     [ -d ${GNUPGHOME} ] && return
     mkdir -m 0700 "$GNUPGHOME"
+    test_have_prereq GPG || return
     gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1
     test_debug "cat $GNUPGHOME/import.log"
     if (gpg --quick-random --version >/dev/null 2>&1) ; then
@@ -41,7 +41,7 @@ add_gnupg_home ()
 
 add_gnupg_home
 # get key fingerprint
-FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
+test_have_prereq GPG && FINGERPRINT=$(gpg --no-tty --list-secret-keys --with-colons --fingerprint | grep '^fpr:' | cut -d: -f10)
 
 # for some reason this is needed for emacs_deliver_message to work,
 # although I can't figure out why
@@ -80,7 +80,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -115,7 +115,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -148,7 +148,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -191,7 +191,7 @@ Non-text part: application/octet-stream
 
part}
 
body}
 
message}'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -226,7 +226,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "application/octet-stream",
  "filename": "TESTATTACHMENT"}]}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -237,7 +237,7 @@ output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted m
 expected='{"id": 4,
  "content-type": "text/plain",
  "content": "This is a test encrypted message.\n"}'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -247,7 +247,7 @@ notmuch show \
     --part=5 \
     --decrypt \
     subject:"test encrypted message 001" >OUTPUT
-test_expect_equal_file GPG OUTPUT TESTATTACHMENT
+test_expect_equal_file EMACS+GPG OUTPUT TESTATTACHMENT
 
 test_begin_subtest "decryption failure with missing key"
 mv "${GNUPGHOME}"{,.bak}
@@ -274,7 +274,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/octet-stream"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 mv "${GNUPGHOME}"{.bak,}
@@ -314,7 +314,7 @@ expected='[[[{"id": "XXXXX",
  "content-type": "text/plain",
  "content": "This is another test encrypted message.\n"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -326,7 +326,7 @@ Subject: Re: test encrypted message 002
 
 On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
 > This is another test encrypted message.'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
@@ -367,7 +367,7 @@ expected='[[[{"id": "XXXXX",
  {"id": 3,
  "content-type": "application/pgp-signature"}]}]},
  []]]]'
-test_expect_equal GPG \
+test_expect_equal EMACS+GPG \
     "$output" \
     "$expected"
 
diff --git a/test/emacs b/test/emacs
index 41b6ee9..ea641af 100755
--- a/test/emacs
+++ b/test/emacs
@@ -4,8 +4,8 @@ test_description="emacs interface"
 . test-lib.sh
 
 # Emacs is a prereq.
-test_expect_success "prereq: Emacs is present" "which emacs" \
-    && test_set_prereq EMACS
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
 
 
 EXPECTED=$TEST_DIRECTORY/emacs.expected-output
@@ -124,7 +124,7 @@ output=$(notmuch search 'id:"123..456@example"' | notmuch_search_sanitize)
 test_expect_equal EMACS "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Message with .. in Message-Id (inbox search-add show-add)"
 
 test_begin_subtest "Sending a message via (fake) SMTP"
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'Testing message sent via SMTP' \
     'This is a test that messages are sent via SMTP' \
     '(message-goto-to)
@@ -150,7 +150,7 @@ test_expect_equal_file EMACS OUTPUT EXPECTED
 test_begin_subtest "Verify that sent messages are saved/searchable (via FCC)"
 notmuch new > /dev/null
 output=$(notmuch search 'subject:"testing message sent via SMTP"' | notmuch_search_sanitize)
-test_expect_equal "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)"
+test_expect_equal EMACS "$output" "thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)"
 
 test_begin_subtest "notmuch-fcc-dirs set to nil"
 test_emacs "(let ((notmuch-fcc-dirs nil))
@@ -244,7 +244,7 @@ test_emacs '(notmuch-search "subject:\"testing message sent via SMTP\"")
 	    (notmuch-test-wait)
 	    (notmuch-search-reply-to-thread)
 	    (test-output)'
-sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' OUTPUT
+sed -i -e 's/^In-Reply-To: <.*>$/In-Reply-To: <XXX>/' EMACS OUTPUT
 cat <<EOF >EXPECTED
 From: Notmuch Test Suite <test_suite@notmuchmail.org>
 To: user@example.com
@@ -375,7 +375,7 @@ test_emacs '(notmuch-show "id:\"bought\"")
         (reverse-region (point-min) (point-max))
 	    (test-output)'
 sed -i -e 's/^.*tmp.emacs\/mail.*$/FILENAME/' OUTPUT
-test_expect_equal_file OUTPUT $EXPECTED/emacs-stashing
+test_expect_equal_file EMACS OUTPUT $EXPECTED/emacs-stashing
 
 test_begin_subtest "Stashing in notmuch-search"
 test_emacs '(notmuch-search "id:\"bought\"")
@@ -386,12 +386,13 @@ test_emacs '(notmuch-search "id:\"bought\"")
         (yank)
 	    (test-output)'
 sed -i -e 's/^thread:.*$/thread:XXX/' OUTPUT
-test_expect_equal $(cat OUTPUT) "thread:XXX"
+output=$(cat OUTPUT)
+test_expect_equal EMACS "$output" "thread:XXX"
 
 test_begin_subtest 'Hiding message following HTML part'
 test_subtest_known_broken
 id='html-message@notmuchmail.org'
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'HTML message' \
     '' \
     "(message-goto-eoh)
@@ -399,7 +400,7 @@ emacs_deliver_message \
      (message-goto-body)
      (mml-insert-part \"text/html\")
      (insert \"<body>This is a test HTML message</body>\")"
-emacs_deliver_message \
+test_have_prereq EMACS && emacs_deliver_message \
     'Reply to HTML message' \
     'This is a reply to the test HTML message' \
     "(message-goto-eoh)
@@ -426,7 +427,7 @@ test_emacs "(notmuch-search \"$message1 or $message2\")
 	    (test-output)"
 test_emacs "(notmuch-show \"$message2\")
 	    (test-output \"EXPECTED\")"
-test_expect_equal_file OUTPUT EXPECTED
+test_expect_equal_file EMACS OUTPUT EXPECTED
 
 test_begin_subtest "Refresh show buffer"
 test_emacs '(notmuch-show "id:f35dbb950911171438k5df6eb56k77b6c0944e2e79ae@mail.gmail.com")
diff --git a/test/emacs-large-search-buffer b/test/emacs-large-search-buffer
index 5757380..63eb87c 100755
--- a/test/emacs-large-search-buffer
+++ b/test/emacs-large-search-buffer
@@ -3,8 +3,8 @@ test_description="Emacs with large search results buffer"
 . test-lib.sh
 
 # Emacs is a prereq.
-test_expect_success "prereq: Emacs is present" "which emacs" \
-    && test_set_prereq EMACS
+test_set_bin_prereq dtach "dtach" DTACH
+test_have_prereq DTACH && test_set_bin_prereq emacs "Emacs" EMACS
 
 
 x=xxxxxxxxxx # 10
diff --git a/test/test-lib.sh b/test/test-lib.sh
index 1ea7fa9..ca1f412 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -543,6 +543,19 @@ test_have_prereq () {
 	esac
 }
 
+test_set_bin_prereq () {
+	bin=$1
+	name=$2
+	prereq=$3
+	if hash $bin &>/dev/null
+	then
+		test_set_prereq $prereq
+	else
+		say_color info "%-6s" "INFO"
+		echo " Missing test prerequisite: $name"
+	fi
+}
+
 # You are not expected to call test_ok_ and test_failure_ directly, use
 # the text_expect_* functions instead.
 
@@ -839,6 +852,10 @@ EOF
 }
 
 test_emacs () {
+	# First check that emacs and dtach are available
+	hash emacs &>/dev/null || return
+	hash dtach &>/dev/null || return
+
 	if [ -z "$EMACS_SERVER" ]; then
 		EMACS_SERVER="notmuch-test-suite-$$"
 		# start a detached session with an emacs server

--=-=-=



Peace

-- 
Pieter

[1] id:"874ny4fcdz.fsf@praet.org"
[2] id:"87ipn3obvi.fsf@praet.org"
[3] id:"87fwi7obso.fsf@praet.org"

--=-=-=--