Re: [PATCH] perf-test: bump version to 0.4, use manifest files
authorTomi Ollila <tomi.ollila@iki.fi>
Mon, 2 Dec 2013 15:25:31 +0000 (17:25 +0200)
committerW. Trevor King <wking@tremily.us>
Fri, 7 Nov 2014 17:58:35 +0000 (09:58 -0800)
09/d56cb7fedaecafe46585fd44d37ed7db1552ec [new file with mode: 0644]

diff --git a/09/d56cb7fedaecafe46585fd44d37ed7db1552ec b/09/d56cb7fedaecafe46585fd44d37ed7db1552ec
new file mode 100644 (file)
index 0000000..2e79cd0
--- /dev/null
@@ -0,0 +1,218 @@
+Return-Path: <tomi.ollila@iki.fi>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 3113C429E35\r
+       for <notmuch@notmuchmail.org>; Mon,  2 Dec 2013 07:25:46 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id gYSfA737m5vD for <notmuch@notmuchmail.org>;\r
+       Mon,  2 Dec 2013 07:25:41 -0800 (PST)\r
+Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])\r
+       by olra.theworths.org (Postfix) with ESMTP id 937D7429E2F\r
+       for <notmuch@notmuchmail.org>; Mon,  2 Dec 2013 07:25:41 -0800 (PST)\r
+Received: from guru.guru-group.fi (localhost [IPv6:::1])\r
+       by guru.guru-group.fi (Postfix) with ESMTP id D870A100086;\r
+       Mon,  2 Dec 2013 17:25:31 +0200 (EET)\r
+From: Tomi Ollila <tomi.ollila@iki.fi>\r
+To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] perf-test: bump version to 0.4, use manifest files\r
+In-Reply-To: <1385866803-17847-1-git-send-email-david@tethera.net>\r
+References: <1385866803-17847-1-git-send-email-david@tethera.net>\r
+User-Agent: Notmuch/0.17~rc1+17~ga2e1a2f (http://notmuchmail.org) Emacs/24.3.1\r
+       (x86_64-unknown-linux-gnu)\r
+X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL\r
+       $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F\r
+       !)g;OY^,BjTbr)Np:%c_o'jj,Z\r
+Date: Mon, 02 Dec 2013 17:25:31 +0200\r
+Message-ID: <m2ob4zuvf8.fsf@guru.guru-group.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 02 Dec 2013 15:25:46 -0000\r
+\r
+On Sun, Dec 01 2013, David Bremner <david@tethera.net> wrote:\r
+\r
+> The new revision of the performance test includes manifests for each corpus,\r
+> so update the support library to use these manifests at the same time.\r
+> ---\r
+\r
+This is consistent with what is currently in repository so +1\r
+\r
+Tomi\r
+\r
+\r
+>\r
+> Jani's message\r
+>        \r
+>        id:d74499f1e462755676edf9aa6ab689ba47fa2471.1385825425.git.jani@nikula.org\r
+>\r
+> reminded me I had an update of the performance test suite in the\r
+> works.  The new suite is available from\r
+>\r
+>     http://tesseract.cs.unb.ca/notmuch/notmuch-email-corpus-0.4.tar.xz\r
+>     http://tesseract.cs.unb.ca/notmuch/notmuch-email-corpus-0.4.tar.xz.asc\r
+>\r
+> I haven't yet uploaded it to notmuchmail.org because of disk space concerns.\r
+>\r
+>  .../download/notmuch-email-corpus-0.4.tar.xz.asc   | 14 +++++\r
+>  performance-test/perf-test-lib.sh                  | 72 ++++++++++++++--------\r
+>  performance-test/version.sh                        |  2 +-\r
+>  3 files changed, 60 insertions(+), 28 deletions(-)\r
+>  create mode 100644 performance-test/download/notmuch-email-corpus-0.4.tar.xz.asc\r
+>\r
+> diff --git a/performance-test/download/notmuch-email-corpus-0.4.tar.xz.asc b/performance-test/download/notmuch-email-corpus-0.4.tar.xz.asc\r
+> new file mode 100644\r
+> index 0000000..72dedd8\r
+> --- /dev/null\r
+> +++ b/performance-test/download/notmuch-email-corpus-0.4.tar.xz.asc\r
+> @@ -0,0 +1,14 @@\r
+> +-----BEGIN PGP SIGNATURE-----\r
+> +Version: GnuPG v1.4.15 (GNU/Linux)\r
+> +\r
+> +iQGcBAABCAAGBQJSdaDkAAoJEPIClx2kp54sQ54L/ikkvF1fy88hjLitN59v6g2J\r
+> +vw85YNRifNHyp/UXI6nt2eXFzyWJiRHuvHFoBgmEsJVxauOKw61Gs2zd53x9Ear4\r
+> +MGcQWyiM1cnwX/nD7GvxRQNh33f+FEamTjg+QhG47K0A2YdLWcDC7r9GMatGT11x\r
+> +5KE24WQGOqtgQn/9qNtJvkiKIehpRiDTaW/QJ7mTCYeJFjIHJUY8dxyfiTtkJ0z7\r
+> +cJ6omehvWSw4STbEg65XJgqykxMdltNEavfvSbAT73FgmkkyXxul0s5hDZ/esd0n\r
+> +re3dyDxGt085POiAgPti05a4tJI5EQC2wLBUFri0s2JdMtazcD6yVuHNbVzZ4Do3\r
+> +nL/sgwKGUq5wRrPqPWp6HXtZ9zG+/V7hFNrr/l42qGrLqsSh0bqvEnUiwczZLBGy\r
+> +NEs4G8VjmfS2cMKePsWaekBAvFUtb47PSB6JIPwpCNvKXDrcCb28eOQVB2atgj1h\r
+> +9SktOtWYJhWIQp2YW9iae30Z6lhCcdPRRHTFMQq2nQ==\r
+> +=eSMY\r
+> +-----END PGP SIGNATURE-----\r
+> diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh\r
+> index 9ee7661..44708cf 100644\r
+> --- a/performance-test/perf-test-lib.sh\r
+> +++ b/performance-test/perf-test-lib.sh\r
+> @@ -41,52 +41,70 @@ add_email_corpus ()\r
+>  {\r
+>      rm -rf ${MAIL_DIR}\r
+>  \r
+> -    case "$corpus_size" in\r
+> -    small)\r
+> -        mail_subdir="mail/enron/bailey-s"\r
+> -        check_for="${TEST_DIRECTORY}/corpus/$mail_subdir"\r
+> -        ;;\r
+> -    medium)\r
+> -        mail_subdir="mail/notmuch-archive"\r
+> -        check_for="${TEST_DIRECTORY}/corpus/$mail_subdir"\r
+> -        ;;\r
+> -    *)\r
+> -        mail_subdir=mail\r
+> -        check_for="${TEST_DIRECTORY}/corpus/$mail_subdir/enron/wolfe-j"\r
+> -    esac\r
+> +    CORPUS_DIR=${TEST_DIRECTORY}/corpus\r
+> +    mkdir -p "${CORPUS_DIR}"\r
+>  \r
+> -    MAIL_CORPUS="${TEST_DIRECTORY}/corpus/$mail_subdir"\r
+> -    TAG_CORPUS="${TEST_DIRECTORY}/corpus/tags"\r
+> +    MAIL_CORPUS="${CORPUS_DIR}/mail.${corpus_size}"\r
+> +    TAG_CORPUS="${CORPUS_DIR}/tags"\r
+>  \r
+> -    args=()\r
+> -    if [ ! -d "$TAG_CORPUS" ] ; then\r
+> -    args+=("notmuch-email-corpus/tags")\r
+> +    if command -v pixz > /dev/null; then\r
+> +    XZ=pixz\r
+> +    else\r
+> +    XZ=xz\r
+>      fi\r
+>  \r
+> -    if [ ! -d "$check_for" ] ; then\r
+> -    args+=("notmuch-email-corpus/$mail_subdir")\r
+> +    if [ ! -d "${CORPUS_DIR}/manifest" ]; then\r
+> +\r
+> +    printf "Unpacking manifests\n"\r
+> +    tar --extract --use-compress-program ${XZ} --strip-components=1 \\r
+> +        --directory ${TEST_DIRECTORY}/corpus \\r
+> +        --wildcards --file ../download/notmuch-email-corpus-${PERFTEST_VERSION}.tar.xz \\r
+> +        'notmuch-email-corpus/manifest/*'\r
+>      fi\r
+>  \r
+> -    if [[ ${#args[@]} > 0 ]]; then\r
+> -    if command -v pixz > /dev/null; then\r
+> -        XZ=pixz\r
+> +    file_list=$(mktemp file_listXXXXXX)\r
+> +    if [ ! -d "$TAG_CORPUS" ] ; then\r
+> +    echo "notmuch-email-corpus/tags" >> $file_list\r
+> +    fi\r
+> +\r
+> +    if [ ! -d "$MAIL_CORPUS" ] ; then\r
+> +    if [[ "$corpus_size" != "large" ]]; then\r
+> +        sed s,^,notmuch-email-corpus/, < \\r
+> +            ${TEST_DIRECTORY}/corpus/manifest/MANIFEST.${corpus_size} >> $file_list\r
+>      else\r
+> -        XZ=xz\r
+> +        echo "notmuch-email-corpus/mail" >> $file_list\r
+>      fi\r
+> +    fi\r
+>  \r
+> -    printf "Unpacking corpus\n"\r
+> -    mkdir -p "${TEST_DIRECTORY}/corpus"\r
+> +    if [[ -s $file_list ]]; then\r
+>  \r
+> +    printf "Unpacking corpus\n"\r
+>      tar --checkpoint=.5000 --extract --strip-components=1 \\r
+>          --directory ${TEST_DIRECTORY}/corpus \\r
+>          --use-compress-program ${XZ} \\r
+>          --file ../download/notmuch-email-corpus-${PERFTEST_VERSION}.tar.xz \\r
+> -        "${args[@]}"\r
+> +        --anchored --recursion \\r
+> +        --files-from $file_list\r
+>  \r
+>      printf "\n"\r
+>  \r
+> +    if [[ ! -d ${MAIL_CORPUS} ]]; then\r
+> +        printf "creating link farm\n"\r
+> +\r
+> +        if [[ "$corpus_size" = large ]]; then\r
+> +            cp -rl ${TEST_DIRECTORY}/corpus/mail ${MAIL_CORPUS}\r
+> +        else\r
+> +            while read -r file; do\r
+> +                tdir=${MAIL_CORPUS}/$(dirname $file)\r
+> +                mkdir -p $tdir\r
+> +                ln ${TEST_DIRECTORY}/corpus/$file $tdir\r
+> +            done <${TEST_DIRECTORY}/corpus/manifest/MANIFEST.${corpus_size}\r
+> +        fi\r
+> +    fi\r
+> +\r
+>      fi\r
+>  \r
+> +    rm $file_list\r
+>      cp -lr $TAG_CORPUS $TMP_DIRECTORY/corpus.tags\r
+>      cp -lr $MAIL_CORPUS $MAIL_DIR\r
+>  }\r
+> diff --git a/performance-test/version.sh b/performance-test/version.sh\r
+> index afafc73..f02527a 100644\r
+> --- a/performance-test/version.sh\r
+> +++ b/performance-test/version.sh\r
+> @@ -1,3 +1,3 @@\r
+>  # this should be both a valid Makefile fragment and valid POSIX(ish) shell.\r
+>  \r
+> -PERFTEST_VERSION=0.3\r
+> +PERFTEST_VERSION=0.4\r
+> -- \r
+> 1.8.4.2\r
+>\r
+> _______________________________________________\r
+> notmuch mailing list\r
+> notmuch@notmuchmail.org\r
+> http://notmuchmail.org/mailman/listinfo/notmuch\r