Re: [PATCH 3/4] perf-test: unpack tags.

diff --git a/8f/e06aa5f8c86235201174924a2e11cc4c85aa36 b/8f/e06aa5f8c86235201174924a2e11cc4c85aa36
new file mode 100644 (file)
index 0000000..03a785f
--- /dev/null
+++ b/8f/e06aa5f8c86235201174924a2e11cc4c85aa36
@@ -0,0 +1,160 @@
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id A8C95431FB6\r
+       for <notmuch@notmuchmail.org>; Tue,  4 Dec 2012 21:23:30 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id AxKSAX0kdBvg for <notmuch@notmuchmail.org>;\r
+       Tue,  4 Dec 2012 21:23:26 -0800 (PST)\r
+Received: from dmz-mailsec-scanner-2.mit.edu (DMZ-MAILSEC-SCANNER-2.MIT.EDU\r
+       [18.9.25.13])\r
+       by olra.theworths.org (Postfix) with ESMTP id 0BCA6431FAE\r
+       for <notmuch@notmuchmail.org>; Tue,  4 Dec 2012 21:23:26 -0800 (PST)\r
+X-AuditID: 1209190d-b7f266d00000092b-08-50beda4d9817\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+       by dmz-mailsec-scanner-2.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id 25.09.02347.D4ADEB05; Wed,  5 Dec 2012 00:23:25 -0500 (EST)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+       by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id qB55NOE7023211; \r
+       Wed, 5 Dec 2012 00:23:24 -0500\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+       (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id qB55NMmw022600\r
+       (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+       Wed, 5 Dec 2012 00:23:23 -0500 (EST)\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+       (envelope-from <amdragon@mit.edu>)\r
+       id 1Tg7S6-0004ne-7R; Wed, 05 Dec 2012 00:23:22 -0500\r
+From: Austin Clements <aclements@csail.mit.edu>\r
+To: david@tethera.net, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH 3/4] perf-test: unpack tags.\r
+In-Reply-To: <1354594712-5179-3-git-send-email-david@tethera.net>\r
+References: <1354583824-10520-3-git-send-email-david@tethera.net>\r
+       <1354594712-5179-1-git-send-email-david@tethera.net>\r
+       <1354594712-5179-3-git-send-email-david@tethera.net>\r
+User-Agent: Notmuch/0.14+100~gcb9b0b0 (http://notmuchmail.org) Emacs/23.4.1\r
+       (i486-pc-linux-gnu)\r
+Date: Wed, 05 Dec 2012 00:23:22 -0500\r
+Message-ID: <87pq2pcbsl.fsf@awakening.csail.mit.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFmpkleLIzCtJLcpLzFFi42IRYrdT1/W9tS/AoPuQgsXGZT+ZLW60djNa\r
+       XL85k9mB2eNX21xmj2erbjF7bDn0njmAOYrLJiU1J7MstUjfLoEr48ifFtaCy8IVq768Zm9g\r
+       3MPfxcjJISFgIrF75x92CFtM4sK99WxdjFwcQgL7GCVuzJjFAuGsZ5S4vqmHCcK5wCSxpG0p\r
+       VNkSRonb6yczgvSzCehLrFg7ibWLkYNDREBPYuFWGZAws4CaxKQ1Z8BWCAsYSKy52MAEUsIp\r
+       4CjRvdMCYsxqRon/PxaCxUUF4iUuL1UAMVkEVCWO/FUD6eQFOvT1smVMELagxMmZT1ggpmtJ\r
+       3Pj3kmkCo+AsJKlZSFILGJlWMcqm5Fbp5iZm5hSnJusWJyfm5aUW6Rrp5WaW6KWmlG5iBIUu\r
+       pyTvDsZ3B5UOMQpwMCrx8Eoa7wsQYk0sK67MPcQoycGkJMqbdRMoxJeUn1KZkVicEV9UmpNa\r
+       fIhRgoNZSYT3nx1QjjclsbIqtSgfJiXNwaIkznsl5aa/kEB6YklqdmpqQWoRTFaGg0NJgncS\r
+       yFDBotT01Iq0zJwShDQTByfIcB6g4fNAaniLCxJzizPTIfKnGHU5ev91PWEUYsnLz0uVEuft\r
+       ACkSACnKKM2DmwNLOa8YxYHeEubdDlLFA0xXcJNeAS1hAlryQmg3yJKSRISUVAOju0bV4tPs\r
+       353lWAUXP600neGl+Ohdq07VhOupyx7NOa65VzCiO+2R5rpYPfmPjHELszSkRZX9Za7m7jbf\r
+       3R1R/6LMLdI4/qvnByuHAy8deB/WSOo/lqrvft5uOt9d9mnYRdk5rgETaj4/frxodsP92kuK\r
+       ZqKv5fkiUibX1k9dsK2S0VG9d6oSS3FGoqEWc1FxIgAJ0floFAMAAA==\r
+Cc: David Bremner <bremner@debian.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 05 Dec 2012 05:23:30 -0000\r
+\r
+On Mon, 03 Dec 2012, david@tethera.net wrote:\r
+> From: David Bremner <bremner@debian.org>\r
+>\r
+> It's a bit annoying to call tar twice, but we cache the results so it\r
+> isn't as bad as it could be.\r
+> ---\r
+>  performance-test/Makefile.local   |    1 +\r
+>  performance-test/perf-test-lib.sh |   25 +++++++++++++++++++------\r
+>  2 files changed, 20 insertions(+), 6 deletions(-)\r
+>\r
+> diff --git a/performance-test/Makefile.local b/performance-test/Makefile.local\r
+> index b136a88..cdd7f19 100644\r
+> --- a/performance-test/Makefile.local\r
+> +++ b/performance-test/Makefile.local\r
+> @@ -30,3 +30,4 @@ download-corpus:\r
+>      wget -O ${TXZFILE} ${DEFAULT_URL}\r
+>  \r
+>  CLEAN := $(CLEAN) $(dir)/tmp.* $(dir)/corpus.mail.* $(dir)/notmuch.cache.*\r
+> +CLEAN := $(CLEAN) $(dir)/corpus.tags\r
+> diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh\r
+> index 08e2ebd..40c88c9 100644\r
+> --- a/performance-test/perf-test-lib.sh\r
+> +++ b/performance-test/perf-test-lib.sh\r
+> @@ -41,6 +41,13 @@ DB_CACHE_DIR=${TEST_DIRECTORY}/notmuch.cache.$corpus_size\r
+>  add_email_corpus ()\r
+>  {\r
+>      rm -rf ${MAIL_DIR}\r
+> +\r
+> +    if command -v pixz > /dev/null; then\r
+> +    XZ=pixz\r
+> +    else\r
+> +    XZ=xz\r
+> +    fi\r
+> +\r
+>      if [ ! -d $CORPUS_DIR ]; then\r
+>      case "$corpus_size" in\r
+>          small)\r
+> @@ -53,12 +60,6 @@ add_email_corpus ()\r
+>              arg=mail\r
+>      esac\r
+>  \r
+> -    if command -v pixz > /dev/null; then\r
+> -        XZ=pixz\r
+> -    else\r
+> -        XZ=xz\r
+> -    fi\r
+> -\r
+>      printf "Unpacking corpus\n"\r
+>      mkdir $CORPUS_DIR\r
+>  \r
+> @@ -72,6 +73,18 @@ add_email_corpus ()\r
+>  \r
+>      fi\r
+>  \r
+> +    if [ ! -d $TEST_DIRECTORY/corpus.tags ]; then\r
+> +\r
+> +    mkdir $TEST_DIRECTORY/corpus.tags\r
+> +\r
+> +    tar --extract --strip-components=2 \\r
+> +        --directory $TEST_DIRECTORY/corpus.tags \\r
+> +        --use-compress-program ${XZ} \\r
+> +        --file ../download/notmuch-email-corpus-${PERFTEST_VERSION}.tar.xz \\r
+> +        notmuch-email-corpus/tags\r
+\r
+Why not --strip-components=1 and unpack both mail/ and tags/ into a\r
+single, shared corpus cache directory in one call to tar?  Since you're\r
+going to cp -lr things anyway, you can structure the corpus cache\r
+however is convenient.\r
+\r
+> +    fi\r
+> +\r
+> +    cp -lr $TEST_DIRECTORY/corpus.tags $TMP_DIRECTORY\r
+>      cp -lr $CORPUS_DIR $MAIL_DIR\r
+>  \r
+>      if [ -d $DB_CACHE_DIR ]; then\r
+> -- \r
+> 1.7.10.4\r
+>\r
+> _______________________________________________\r
+> notmuch mailing list\r
+> notmuch@notmuchmail.org\r
+> http://notmuchmail.org/mailman/listinfo/notmuch\r