Return-Path: X-Original-To: notmuch@notmuchmail.org Delivered-To: notmuch@notmuchmail.org Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 1B2F5431FBD for ; Mon, 24 Dec 2012 06:59:39 -0800 (PST) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: 0 X-Spam-Level: X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] autolearn=disabled Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3lP93YxRBqZj for ; Mon, 24 Dec 2012 06:59:37 -0800 (PST) Received: from tesseract.cs.unb.ca (tesseract.cs.unb.ca [131.202.240.238]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by olra.theworths.org (Postfix) with ESMTPS id 11257431FB6 for ; Mon, 24 Dec 2012 06:59:37 -0800 (PST) Received: from fctnnbsc30w-156034082078.dhcp-dynamic.fibreop.nb.bellaliant.net ([156.34.82.78] helo=zancas.localnet) by tesseract.cs.unb.ca with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1Tn9V9-0000TT-HU; Mon, 24 Dec 2012 10:59:36 -0400 Received: from bremner by zancas.localnet with local (Exim 4.80) (envelope-from ) id 1Tn9V4-0001kZ-0W; Mon, 24 Dec 2012 10:59:30 -0400 From: david@tethera.net To: notmuch@notmuchmail.org Subject: [Patch v2 2/4] perf-test: rename current tests as "time tests" Date: Mon, 24 Dec 2012 10:59:18 -0400 Message-Id: <1356361160-6449-3-git-send-email-david@tethera.net> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1356361160-6449-1-git-send-email-david@tethera.net> References: <1356361160-6449-1-git-send-email-david@tethera.net> X-Spam_bar: - Cc: David Bremner X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Dec 2012 14:59:39 -0000 From: David Bremner This is almost entirely renaming files, except for updating a few references to those file names, and changing the makefile target. A new set of memory tests will be run separately because they take much longer. --- performance-test/00-new | 15 --------------- performance-test/01-dump-restore | 13 ------------- performance-test/02-tag | 14 -------------- performance-test/Makefile.local | 2 +- performance-test/README | 9 +++++---- performance-test/T00-new | 15 +++++++++++++++ performance-test/T01-dump-restore | 13 +++++++++++++ performance-test/T02-tag | 14 ++++++++++++++ performance-test/notmuch-perf-test | 27 --------------------------- performance-test/notmuch-time-test | 27 +++++++++++++++++++++++++++ 10 files changed, 75 insertions(+), 74 deletions(-) delete mode 100755 performance-test/00-new delete mode 100755 performance-test/01-dump-restore delete mode 100755 performance-test/02-tag create mode 100755 performance-test/T00-new create mode 100755 performance-test/T01-dump-restore create mode 100755 performance-test/T02-tag delete mode 100755 performance-test/notmuch-perf-test create mode 100755 performance-test/notmuch-time-test diff --git a/performance-test/00-new b/performance-test/00-new deleted file mode 100755 index 553bb8b..0000000 --- a/performance-test/00-new +++ /dev/null @@ -1,15 +0,0 @@ -#!/bin/bash - -test_description='notmuch new' - -. ./perf-test-lib.sh - -uncache_database - -time_start - -for i in $(seq 2 6); do - time_run "notmuch new #$i" 'notmuch new' -done - -time_done diff --git a/performance-test/01-dump-restore b/performance-test/01-dump-restore deleted file mode 100755 index b2ff940..0000000 --- a/performance-test/01-dump-restore +++ /dev/null @@ -1,13 +0,0 @@ -#!/bin/bash - -test_description='dump and restore' - -. ./perf-test-lib.sh - -time_start - -time_run 'load nmbug tags' 'notmuch restore --accumulate < corpus.tags/nmbug.sup-dump' -time_run 'dump *' 'notmuch dump > tags.out' -time_run 'restore *' 'notmuch restore < tags.out' - -time_done diff --git a/performance-test/02-tag b/performance-test/02-tag deleted file mode 100755 index 78ceccc..0000000 --- a/performance-test/02-tag +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/bash - -test_description='tagging' - -. ./perf-test-lib.sh - -time_start - -time_run 'tag * +new_tag' "notmuch tag +new_tag '*'" -time_run 'tag * +existing_tag' "notmuch tag +new_tag '*'" -time_run 'tag * -existing_tag' "notmuch tag -new_tag '*'" -time_run 'tag * -missing_tag' "notmuch tag -new_tag '*'" - -time_done diff --git a/performance-test/Makefile.local b/performance-test/Makefile.local index 3834e4d..57beb44 100644 --- a/performance-test/Makefile.local +++ b/performance-test/Makefile.local @@ -10,7 +10,7 @@ SIGFILE := ${TXZFILE}.asc TEST_SCRIPT := ${dir}/notmuch-perf-test DEFAULT_URL := http://notmuchmail.org/releases/${CORPUS_NAME} -perf-test: setup-perf-test all +time-test: setup-perf-test all $(TEST_SCRIPT) $(OPTIONS) .PHONY: download-corpus setup-perf-test diff --git a/performance-test/README b/performance-test/README index 1481660..d1fb6de 100644 --- a/performance-test/README +++ b/performance-test/README @@ -36,8 +36,8 @@ for a list of mirrors. Running tests ------------- -The easiest way to run performance tests is to say "make perf-test", (or -simply run the notmuch-perf-test script). Either command will run all +The easiest way to run performance tests is to say "make time-test", (or +simply run the notmuch-time-test script). Either command will run all available performance tests. Alternately, you can run a specific subset of tests by simply invoking @@ -51,7 +51,7 @@ Each test script supports the following arguments Writing tests ------------- -Have a look at "01-dump-restore" for an example. Sourcing +Have a look at "T01-dump-restore" for an example. Sourcing "perf-test-lib.sh" is mandatory. Utility functions include - 'add_email_corpus' unpacks a set of messages and adds them to the database. @@ -65,4 +65,5 @@ Have a look at "01-dump-restore" for an example. Sourcing Scripts are run in the order specified in notmuch-perf-test. In the future this order might be chosen automatically so please follow the -convention of starting the name with two digits to specify the order. +convention of starting the name with 'T' followed by two digits to +specify the order. diff --git a/performance-test/T00-new b/performance-test/T00-new new file mode 100755 index 0000000..553bb8b --- /dev/null +++ b/performance-test/T00-new @@ -0,0 +1,15 @@ +#!/bin/bash + +test_description='notmuch new' + +. ./perf-test-lib.sh + +uncache_database + +time_start + +for i in $(seq 2 6); do + time_run "notmuch new #$i" 'notmuch new' +done + +time_done diff --git a/performance-test/T01-dump-restore b/performance-test/T01-dump-restore new file mode 100755 index 0000000..b2ff940 --- /dev/null +++ b/performance-test/T01-dump-restore @@ -0,0 +1,13 @@ +#!/bin/bash + +test_description='dump and restore' + +. ./perf-test-lib.sh + +time_start + +time_run 'load nmbug tags' 'notmuch restore --accumulate < corpus.tags/nmbug.sup-dump' +time_run 'dump *' 'notmuch dump > tags.out' +time_run 'restore *' 'notmuch restore < tags.out' + +time_done diff --git a/performance-test/T02-tag b/performance-test/T02-tag new file mode 100755 index 0000000..78ceccc --- /dev/null +++ b/performance-test/T02-tag @@ -0,0 +1,14 @@ +#!/bin/bash + +test_description='tagging' + +. ./perf-test-lib.sh + +time_start + +time_run 'tag * +new_tag' "notmuch tag +new_tag '*'" +time_run 'tag * +existing_tag' "notmuch tag +new_tag '*'" +time_run 'tag * -existing_tag' "notmuch tag -new_tag '*'" +time_run 'tag * -missing_tag' "notmuch tag -new_tag '*'" + +time_done diff --git a/performance-test/notmuch-perf-test b/performance-test/notmuch-perf-test deleted file mode 100755 index fc39d8a..0000000 --- a/performance-test/notmuch-perf-test +++ /dev/null @@ -1,27 +0,0 @@ -#!/usr/bin/env bash - -# Run tests -# -# Copyright (c) 2005 Junio C Hamano -# -# Adapted from a Makefile to a shell script by Carl Worth (2010) - -if [ ${BASH_VERSINFO[0]} -lt 4 ]; then - echo "Error: The notmuch test suite requires a bash version >= 4.0" - echo "due to use of associative arrays within the test suite." - echo "Please try again with a newer bash (or help us fix the" - echo "test suite to be more portable). Thanks." - exit 1 -fi - -cd $(dirname "$0") - -TESTS=" - 00-new - 01-dump-restore - 02-tag -" - -for test in $TESTS; do - ./$test "$@" -done diff --git a/performance-test/notmuch-time-test b/performance-test/notmuch-time-test new file mode 100755 index 0000000..54a208f --- /dev/null +++ b/performance-test/notmuch-time-test @@ -0,0 +1,27 @@ +#!/usr/bin/env bash + +# Run tests +# +# Copyright (c) 2005 Junio C Hamano +# +# Adapted from a Makefile to a shell script by Carl Worth (2010) + +if [ ${BASH_VERSINFO[0]} -lt 4 ]; then + echo "Error: The notmuch test suite requires a bash version >= 4.0" + echo "due to use of associative arrays within the test suite." + echo "Please try again with a newer bash (or help us fix the" + echo "test suite to be more portable). Thanks." + exit 1 +fi + +cd $(dirname "$0") + +TESTS=" + T00-new + T01-dump-restore + T02-tag +" + +for test in $TESTS; do + ./$test "$@" +done -- 1.7.10.4