--- /dev/null
+Return-Path: <bremner@tethera.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 21329431FB6\r
+ for <notmuch@notmuchmail.org>; Sun, 25 Nov 2012 07:02:47 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+ autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id zJXOu7xviP0c for <notmuch@notmuchmail.org>;\r
+ Sun, 25 Nov 2012 07:02:43 -0800 (PST)\r
+Received: from tesseract.cs.unb.ca (tesseract.cs.unb.ca [131.202.240.238])\r
+ (using TLSv1 with cipher AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 06627431FC4\r
+ for <notmuch@notmuchmail.org>; Sun, 25 Nov 2012 07:02:41 -0800 (PST)\r
+Received: from fctnnbsc30w-156034089108.dhcp-dynamic.fibreop.nb.bellaliant.net\r
+ ([156.34.89.108] helo=zancas.localnet)\r
+ by tesseract.cs.unb.ca with esmtpsa\r
+ (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72)\r
+ (envelope-from <bremner@tethera.net>)\r
+ id 1TcdjD-0007HR-Du; Sun, 25 Nov 2012 11:02:40 -0400\r
+Received: from bremner by zancas.localnet with local (Exim 4.80)\r
+ (envelope-from <bremner@tethera.net>)\r
+ id 1Tcdj7-00034t-UQ; Sun, 25 Nov 2012 11:02:33 -0400\r
+From: david@tethera.net\r
+To: notmuch@notmuchmail.org\r
+Subject: [Patch v4 2/2] test: initial performance testing infrastructure\r
+Date: Sun, 25 Nov 2012 11:02:24 -0400\r
+Message-Id: <1353855745-11697-3-git-send-email-david@tethera.net>\r
+X-Mailer: git-send-email 1.7.10.4\r
+In-Reply-To: <1353855745-11697-1-git-send-email-david@tethera.net>\r
+References: <1353855745-11697-1-git-send-email-david@tethera.net>\r
+X-Spam_bar: -\r
+Cc: David Bremner <bremner@debian.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 25 Nov 2012 15:02:47 -0000\r
+\r
+From: David Bremner <bremner@debian.org>\r
+\r
+This is not near as fancy as as the unit tests, on the theory that\r
+the code should typically be crashing when performance tuning.\r
+Nonetheless, there is plenty of room for improvement. Several more of\r
+the pieces of the test infrastructure (e.g. the option parsing) could\r
+be factored out into test/test-lib-common.sh\r
+---\r
+ Makefile | 3 +-\r
+ performance-test/.gitignore | 1 +\r
+ performance-test/Makefile | 7 +++\r
+ performance-test/Makefile.local | 32 ++++++++++\r
+ performance-test/README | 50 +++++++++++++++\r
+ performance-test/basic | 15 +++++\r
+ performance-test/download/.gitignore | 2 +\r
+ .../download/notmuch-email-corpus-0.2.tar.xz.asc | 9 +++\r
+ performance-test/notmuch-perf-test | 25 ++++++++\r
+ performance-test/perf-test-lib.sh | 65 ++++++++++++++++++++\r
+ performance-test/version.sh | 3 +\r
+ 11 files changed, 211 insertions(+), 1 deletion(-)\r
+ create mode 100644 performance-test/.gitignore\r
+ create mode 100644 performance-test/Makefile\r
+ create mode 100644 performance-test/Makefile.local\r
+ create mode 100644 performance-test/README\r
+ create mode 100755 performance-test/basic\r
+ create mode 100644 performance-test/download/.gitignore\r
+ create mode 100644 performance-test/download/notmuch-email-corpus-0.2.tar.xz.asc\r
+ create mode 100755 performance-test/notmuch-perf-test\r
+ create mode 100644 performance-test/perf-test-lib.sh\r
+ create mode 100644 performance-test/version.sh\r
+\r
+diff --git a/Makefile b/Makefile\r
+index bb9c316..5decbea 100644\r
+--- a/Makefile\r
++++ b/Makefile\r
+@@ -3,7 +3,8 @@\r
+ all:\r
+ \r
+ # List all subdirectories here. Each contains its own Makefile.local\r
+-subdirs = compat completion emacs lib man parse-time-string util test\r
++subdirs := compat completion emacs lib man parse-time-string\r
++subdirs := $(subdirs) performance-test util test\r
+ \r
+ # We make all targets depend on the Makefiles themselves.\r
+ global_deps = Makefile Makefile.config Makefile.local \\r
+diff --git a/performance-test/.gitignore b/performance-test/.gitignore\r
+new file mode 100644\r
+index 0000000..53f2697\r
+--- /dev/null\r
++++ b/performance-test/.gitignore\r
+@@ -0,0 +1 @@\r
++tmp.*/\r
+diff --git a/performance-test/Makefile b/performance-test/Makefile\r
+new file mode 100644\r
+index 0000000..de492a7\r
+--- /dev/null\r
++++ b/performance-test/Makefile\r
+@@ -0,0 +1,7 @@\r
++# See Makefile.local for the list of files to be compiled in this\r
++# directory.\r
++all:\r
++ $(MAKE) -C .. all\r
++\r
++.DEFAULT:\r
++ $(MAKE) -C .. $@\r
+diff --git a/performance-test/Makefile.local b/performance-test/Makefile.local\r
+new file mode 100644\r
+index 0000000..1114ec1\r
+--- /dev/null\r
++++ b/performance-test/Makefile.local\r
+@@ -0,0 +1,32 @@\r
++# -*- makefile -*-\r
++\r
++dir := performance-test\r
++\r
++include $(dir)/version.sh\r
++\r
++CORPUS_NAME := notmuch-email-corpus-$(PERFTEST_VERSION).tar.xz\r
++TXZFILE := ${dir}/download/${CORPUS_NAME}\r
++SIGFILE := ${TXZFILE}.asc\r
++TEST_SCRIPT := ${dir}/notmuch-perf-test\r
++DEFAULT_URL := http://notmuchmail.org/releases/${CORPUS_NAME}\r
++\r
++perf-test: setup-perf-test all\r
++ $(TEST_SCRIPT) $(OPTIONS)\r
++\r
++.PHONY: download-corpus setup-perf-test\r
++\r
++# Note that this intentionally does not depend on download-corpus.\r
++setup-perf-test: $(TXZFILE)\r
++ gpg --verify $(SIGFILE)\r
++\r
++$(TXZFILE):\r
++ @printf "\nPlease download ${TXZFILE}.\n\n"\r
++ @printf "\t%% make download-corpus\n\n"\r
++ @echo or see http://notmuchmail.org/corpus for download locations\r
++ @echo\r
++ @false\r
++\r
++download-corpus:\r
++ wget -O ${TXZFILE} ${DEFAULT_URL}\r
++\r
++CLEAN := $(CLEAN) $(dir)/tmp.*\r
+diff --git a/performance-test/README b/performance-test/README\r
+new file mode 100644\r
+index 0000000..239d2fb\r
+--- /dev/null\r
++++ b/performance-test/README\r
+@@ -0,0 +1,50 @@\r
++Pre-requisites\r
++--------------\r
++\r
++In addition to having notmuch, you need:\r
++\r
++- gpg\r
++- gnu tar\r
++- gnu time\r
++- xz. Some speedup can be gotten by installing "pixz", but this is\r
++ probably only worthwhile if you are debugging the tests.\r
++\r
++Getting set up to run tests:\r
++----------------------------\r
++\r
++First, you need to get the corpus.\r
++\r
++It should work to run\r
++\r
++ % make download-corpus\r
++\r
++In case that fails or is too slow, check\r
++\r
++ http://notmuchmail.org/corpus\r
++\r
++for a list of mirrors.\r
++\r
++Running tests\r
++-------------\r
++\r
++The easiest way to run performance tests is to say "make perf-test", (or\r
++simply run the notmuch-perf-test script). Either command will run all\r
++available performance tests.\r
++\r
++Alternately, you can run a specific subset of tests by simply invoking\r
++one of the executable scripts in this directory, (such as ./basic).\r
++\r
++\r
++Writing tests\r
++-------------\r
++\r
++Have a look at "basic" for an example.\r
++\r
++add_email_corpus takes arguments "--small" and "--medium" for when you\r
++want smaller corpuses to check.\r
++\r
++time_done does the cleanup; comment it out or define "$debug" to leave\r
++the temporary files around.\r
++\r
++Currently there is no option processing (e.g. --debug) in the\r
++performance tests.\r
+diff --git a/performance-test/basic b/performance-test/basic\r
+new file mode 100755\r
+index 0000000..9d015ee\r
+--- /dev/null\r
++++ b/performance-test/basic\r
+@@ -0,0 +1,15 @@\r
++#!/bin/bash\r
++\r
++. ./perf-test-lib.sh\r
++\r
++add_email_corpus\r
++\r
++print_header\r
++\r
++time_run 'initial notmuch new' 'notmuch new'\r
++time_run 'second notmuch new' 'notmuch new'\r
++time_run 'dump *' 'notmuch dump > tags.out'\r
++time_run 'restore *' 'notmuch restore < tags.out'\r
++time_run 'tag * +new_tag' "notmuch tag +new_tag '*'"\r
++\r
++time_done\r
+diff --git a/performance-test/download/.gitignore b/performance-test/download/.gitignore\r
+new file mode 100644\r
+index 0000000..7b09234\r
+--- /dev/null\r
++++ b/performance-test/download/.gitignore\r
+@@ -0,0 +1,2 @@\r
++*.tar.gz\r
++*.tar.xz\r
+diff --git a/performance-test/download/notmuch-email-corpus-0.2.tar.xz.asc b/performance-test/download/notmuch-email-corpus-0.2.tar.xz.asc\r
+new file mode 100644\r
+index 0000000..c8b4b3d\r
+--- /dev/null\r
++++ b/performance-test/download/notmuch-email-corpus-0.2.tar.xz.asc\r
+@@ -0,0 +1,9 @@\r
++-----BEGIN PGP SIGNATURE-----\r
++Version: GnuPG v1.4.12 (GNU/Linux)\r
++\r
++iJwEAAECAAYFAlCsvx0ACgkQTiiN/0Um85kZAwP9GgOQ22jK8mr5X4pT/mB8EjSH\r
++QbndlxxbRrP0ChTqjBQoD3IsTHjNL7W572BfXb/MNo94R/iIQ7yTHCDVNuwBhvKd\r
++7qgIuW2FUS1uTfJRP5KBNf8JPuin+6wqGe8/+y/iOs+XJSdiYg1ElS49Ntnpg0yl\r
++btImgEcxTxQ2qfzDS1g=\r
++=iuZR\r
++-----END PGP SIGNATURE-----\r
+diff --git a/performance-test/notmuch-perf-test b/performance-test/notmuch-perf-test\r
+new file mode 100755\r
+index 0000000..1bea345\r
+--- /dev/null\r
++++ b/performance-test/notmuch-perf-test\r
+@@ -0,0 +1,25 @@\r
++#!/usr/bin/env bash\r
++\r
++# Run tests\r
++#\r
++# Copyright (c) 2005 Junio C Hamano\r
++#\r
++# Adapted from a Makefile to a shell script by Carl Worth (2010)\r
++\r
++if [ ${BASH_VERSINFO[0]} -lt 4 ]; then\r
++ echo "Error: The notmuch test suite requires a bash version >= 4.0"\r
++ echo "due to use of associative arrays within the test suite."\r
++ echo "Please try again with a newer bash (or help us fix the"\r
++ echo "test suite to be more portable). Thanks."\r
++ exit 1\r
++fi\r
++\r
++cd $(dirname "$0")\r
++\r
++TESTS="\r
++ basic\r
++"\r
++\r
++for test in $TESTS; do\r
++ ./$test "$@"\r
++done\r
+diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh\r
+new file mode 100644\r
+index 0000000..11ccc77\r
+--- /dev/null\r
++++ b/performance-test/perf-test-lib.sh\r
+@@ -0,0 +1,65 @@\r
++. ./version.sh\r
++\r
++. ../test/test-lib-common.sh\r
++\r
++set -e\r
++\r
++if ! test -x ../notmuch\r
++then\r
++ echo >&2 'You do not seem to have built notmuch yet.'\r
++ exit 1\r
++fi\r
++\r
++add_email_corpus ()\r
++{\r
++ rm -rf ${MAIL_DIR}\r
++\r
++ arg=""\r
++ case "$1" in\r
++ --small)\r
++ arg="/enron/bailey-s"\r
++ ;;\r
++ --medium)\r
++ arg="/notmuch-archive"\r
++ ;;\r
++ esac\r
++\r
++ if command -v pixz > /dev/null; then\r
++ XZ=pixz\r
++ else\r
++ XZ=xz\r
++ fi\r
++\r
++ printf "Unpacking corpus\n"\r
++ tar --checkpoint=.5000 --extract --strip-components=1 \\r
++ --directory ${TMP_DIRECTORY} \\r
++ --use-compress-program ${XZ} \\r
++ --file ../download/notmuch-email-corpus-${PERFTEST_VERSION}.tar.xz \\r
++ notmuch-email-corpus/mail"$arg"\r
++\r
++ printf "\n"\r
++}\r
++\r
++print_header () {\r
++ printf " Wall(s)\tUsr(s)\tSys(s)\tRes(K)\tIn\tOut\n"\r
++}\r
++\r
++time_run () {\r
++ printf "%-22s" "$1"\r
++ if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi\r
++ if ! eval >&3 "/usr/bin/time -f '%e\t%U\t%S\t%M\t%I\t%O' $2" ; then\r
++ test_failure=$(($test_failure + 1))\r
++ fi\r
++}\r
++\r
++time_done () {\r
++ if [ "$test_failure" = "0" ]; then\r
++ rm -rf "$remove_tmp"\r
++ exit 0\r
++ else\r
++ exit 1\r
++ fi\r
++}\r
++\r
++cd -P "$test" || error "Cannot setup test environment"\r
++test_failure=0\r
+diff --git a/performance-test/version.sh b/performance-test/version.sh\r
+new file mode 100644\r
+index 0000000..d9270b1\r
+--- /dev/null\r
++++ b/performance-test/version.sh\r
+@@ -0,0 +1,3 @@\r
++# this should be both a valid Makefile fragment and valid POSIX(ish) shell.\r
++\r
++PERFTEST_VERSION=0.2\r
+-- \r
+1.7.10.4\r
+\r
projects / notmuch-archives.git / commitdiff