--- /dev/null
+Return-Path: <amthrax@drake.mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 0F4CF431FD0\r
+ for <notmuch@notmuchmail.org>; Sat, 11 Jun 2011 13:05:42 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id lPMwPssIEF4z for <notmuch@notmuchmail.org>;\r
+ Sat, 11 Jun 2011 13:05:40 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-8.mit.edu (DMZ-MAILSEC-SCANNER-8.MIT.EDU\r
+ [18.7.68.37])\r
+ by olra.theworths.org (Postfix) with ESMTP id BD76F429E25\r
+ for <notmuch@notmuchmail.org>; Sat, 11 Jun 2011 13:05:40 -0700 (PDT)\r
+X-AuditID: 12074425-b7b82ae000000a2a-d7-4df3ca7c903d\r
+Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
+ by dmz-mailsec-scanner-8.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id AB.65.02602.C7AC3FD4; Sat, 11 Jun 2011 16:05:16 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+ by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id p5BK5eRg014991; \r
+ Sat, 11 Jun 2011 16:05:40 -0400\r
+Received: from drake.mit.edu\r
+ (209-6-116-242.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com\r
+ [209.6.116.242]) (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id p5BK5cMl005859\r
+ (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+ Sat, 11 Jun 2011 16:05:39 -0400 (EDT)\r
+Received: from amthrax by drake.mit.edu with local (Exim 4.76)\r
+ (envelope-from <amthrax@drake.mit.edu>)\r
+ id 1QVUR8-0000Hq-Sx; Sat, 11 Jun 2011 16:05:38 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 04/17] test: Test atomicity of notmuch new.\r
+Date: Sat, 11 Jun 2011 16:04:30 -0400\r
+Message-Id: <1307822683-848-5-git-send-email-amdragon@mit.edu>\r
+X-Mailer: git-send-email 1.7.5.1\r
+In-Reply-To: <1307822683-848-1-git-send-email-amdragon@mit.edu>\r
+References: <87ei34rnc5.fsf@yoom.home.cworth.org>\r
+ <1307822683-848-1-git-send-email-amdragon@mit.edu>\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFjrFIsWRmVeSWpSXmKPExsUixCmqrFtz6rOvwdlJshbXb85kdmD0eLbq\r
+ FnMAYxSXTUpqTmZZapG+XQJXRv+bs2wF5+0qdm+9zt7AuNaki5GDQ0LAROLuQesuRk4gU0zi\r
+ wr31bF2MXBxCAvsYJRp+LmCGcDYwSmy8soYVwrnPJHF351R2CGc+o8S+6Y/ZQPrZBDQktu1f\r
+ zghiiwhIS+y8O5sVZAWzgJrEny4VkLCwgLXElKWnWUFsFgFViZ9/u8BsXgE7iYZ3W1kgzlCQ\r
+ uHJlHpjNKWAvMe/gVTBbSCBNYsmt3ewTGPkXMDKsYpRNya3SzU3MzClOTdYtTk7My0st0rXQ\r
+ y80s0UtNKd3ECAoadhfVHYwTDikdYhTgYFTi4VVc+9lXiDWxrLgy9xCjJAeTkiivzgmgEF9S\r
+ fkplRmJxRnxRaU5q8SFGCQ5mJRHe9e2ffIV4UxIrq1KL8mFS0hwsSuK88yXVfYUE0hNLUrNT\r
+ UwtSi2CyMhwcShK8m08CDRUsSk1PrUjLzClBSDNxcIIM5wEafhSkhre4IDG3ODMdIn+K0ZJj\r
+ 9tr1hxg5dq3YBCQfbwCSQix5+XmpUuK8qSANAiANGaV5cDNhSeAVozjQi8K8u0CqeIAJBG7q\r
+ K6CFTEALBUrBFpYkIqSkGhj5vYSTSidf9q74FBnMbfgp9kZ0mvFmU9fE0q5XsxdP3PDUJV5i\r
+ wWqZT6Gh9oJ2X1UbzpZpZ3t8VrN6xLJYzl1MaXup94Urny/sOGvZzSt6e2bLLcO1T04+F/Ln\r
+ NJCalcrqVcOepCjvW1y+4fYDm5w4HcFtc2RdXfVnhK4KfvnJubbM/BxPpxJLcUaioRZzUXEi\r
+ ANp4SjndAgAA\r
+Cc: Austin Clements <amdragon@mit.edu>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 11 Jun 2011 20:05:42 -0000\r
+\r
+This tests notmuch new's ability to recover from arbitrary stopping\r
+failures. It interrupts notmuch new after every database commit and,\r
+on every resulting database snapshot, re-runs notmuch new to\r
+completion and checks that the final database state is invariant.\r
+---\r
+ test/atomicity | 103 ++++++++++++++++++++++++++++++++++++++++++++++++++++\r
+ test/atomicity.gdb | 50 +++++++++++++++++++++++++\r
+ test/basic | 1 +\r
+ test/notmuch-test | 1 +\r
+ 4 files changed, 155 insertions(+), 0 deletions(-)\r
+ create mode 100755 test/atomicity\r
+ create mode 100644 test/atomicity.gdb\r
+\r
+diff --git a/test/atomicity b/test/atomicity\r
+new file mode 100755\r
+index 0000000..817819a\r
+--- /dev/null\r
++++ b/test/atomicity\r
+@@ -0,0 +1,103 @@\r
++#!/bin/bash\r
++test_description='atomicity'\r
++. ./test-lib.sh\r
++\r
++# This script tests the effects of killing and restarting "notmuch\r
++# new" at arbitrary points. If notmuch new is properly atomic, the\r
++# final database contents should be the same regardless of when (or\r
++# if) it is killed and restarted.\r
++\r
++if test_expect_success "prereq: GDB is present" "which gdb"; then\r
++ test_set_prereq GDB\r
++fi\r
++\r
++# Create a maildir structure to also stress flag synchronization\r
++mkdir $MAIL_DIR/cur\r
++mkdir $MAIL_DIR/new\r
++mkdir $MAIL_DIR/tmp\r
++mkdir $MAIL_DIR/.remove-dir\r
++\r
++# Prepare the initial database\r
++generate_message [subject]='Duplicate' [filename]='duplicate:2,' [dir]=cur\r
++generate_message [subject]='Remove' [filename]='remove:2,' [dir]=cur\r
++generate_message [subject]='"Remove duplicate"' [filename]='remove-duplicate:2,' [dir]=cur\r
++cp $MAIL_DIR/cur/remove-duplicate:2, $MAIL_DIR/cur/remove-duplicate-copy:2,\r
++generate_message [subject]='Rename' [filename]='rename:2,' [dir]=cur\r
++generate_message [subject]='"Rename duplicate"' [filename]='rename-duplicate:2,' [dir]=cur\r
++generate_message [subject]='"Move 1"' [filename]='move1:2,' [dir]=cur\r
++generate_message [subject]='"Move 2"' [filename]='move2:2,' [dir]=new\r
++generate_message [subject]='Flag' [filename]='flag:2,' [dir]=cur\r
++generate_message [subject]='"Flag duplicate"' [filename]='flag-duplicate:2,' [dir]=cur\r
++cp $MAIL_DIR/cur/flag-duplicate:2, $MAIL_DIR/cur/flag-duplicate-copy:2,F\r
++generate_message [subject]='"Remove directory"' [filename]='remove-directory:2,' [dir]=.remove-dir\r
++generate_message [subject]='"Remove directory duplicate"' [filename]='remove-directory-duplicate:2,' [dir]=.remove-dir\r
++cp $MAIL_DIR/.remove-dir/remove-directory-duplicate:2, $MAIL_DIR/cur/\r
++notmuch new > /dev/null\r
++\r
++# Make all maildir changes, but *don't* update the database\r
++generate_message [subject]='Added' [filename]='added:2,' [dir]=cur\r
++cp $MAIL_DIR/cur/duplicate:2, $MAIL_DIR/cur/duplicate-copy:2,\r
++generate_message [subject]='"Add duplicate"' [filename]='add-duplicate:2,' [dir]=cur\r
++generate_message [subject]='"Add duplicate copy"' [filename]='add-duplicate-copy:2,' [dir]=cur\r
++rm $MAIL_DIR/cur/remove:2,\r
++rm $MAIL_DIR/cur/remove-duplicate-copy:2,\r
++mv $MAIL_DIR/cur/rename:2, $MAIL_DIR/cur/renamed:2,\r
++mv $MAIL_DIR/cur/rename-duplicate:2, $MAIL_DIR/cur/renamed-duplicate:2,\r
++mv $MAIL_DIR/cur/move1:2, $MAIL_DIR/new/move1:2,\r
++mv $MAIL_DIR/new/move2:2, $MAIL_DIR/cur/move2:2,\r
++mv $MAIL_DIR/cur/flag:2, $MAIL_DIR/cur/flag:2,F\r
++rm $MAIL_DIR/cur/flag-duplicate-copy:2,F\r
++rm $MAIL_DIR/.remove-dir/remove-directory:2,\r
++rm $MAIL_DIR/.remove-dir/remove-directory-duplicate:2,\r
++rmdir $MAIL_DIR/.remove-dir\r
++increment_mtime $MAIL_DIR/cur\r
++increment_mtime $MAIL_DIR/new\r
++increment_mtime $MAIL_DIR\r
++\r
++# Prepare a snapshot of the updated maildir. The gdb script will\r
++# update the database in this snapshot as it goes.\r
++cp -ra $MAIL_DIR $MAIL_DIR.snap\r
++cp ${NOTMUCH_CONFIG} ${NOTMUCH_CONFIG}.snap\r
++NOTMUCH_CONFIG=${NOTMUCH_CONFIG}.snap notmuch config set database.path $MAIL_DIR.snap\r
++\r
++\r
++test_begin_subtest '"notmuch new" is idempotent under arbitrary aborts'\r
++\r
++# Execute notmuch new and, at every call to rename, snapshot the\r
++# database, run notmuch new again on the snapshot, and capture the\r
++# results of search.\r
++#\r
++# -tty /dev/null works around a conflict between the 'timeout' wrapper\r
++# and gdb's attempt to control the TTY.\r
++export MAIL_DIR\r
++gdb -tty /dev/null -batch -x $TEST_DIRECTORY/atomicity.gdb notmuch >/dev/null 2>/dev/null\r
++\r
++# Get the final, golden output\r
++notmuch search '*' > expected\r
++\r
++# Check output against golden output\r
++outcount=$(cat outcount)\r
++echo -n > searchall\r
++echo -n > expectall\r
++for ((i = 0; i < $outcount; i++)); do\r
++ if ! cmp -s search.$i expected; then\r
++ # Find the range of interruptions that match this output\r
++ for ((end = $i + 1 ; end < $outcount; end++)); do\r
++ if ! cmp -s search.$i search.$end; then\r
++ break\r
++ fi\r
++ done\r
++ echo "When interrupted after $test/backtrace.$(expr $i - 1) (abort points $i-$(expr $end - 1))" >> searchall\r
++ cat search.$i >> searchall\r
++ cat expected >> expectall\r
++ echo >> searchall\r
++ echo >> expectall\r
++\r
++ i=$(expr $end - 1)\r
++ fi\r
++done\r
++test_expect_equal_failure GDB "$(cat searchall)" "$(cat expectall)"\r
++\r
++test_expect_success GDB "detected $outcount>10 abort points" "test $outcount -gt 10"\r
++\r
++test_done\r
+diff --git a/test/atomicity.gdb b/test/atomicity.gdb\r
+new file mode 100644\r
+index 0000000..fd67525\r
+--- /dev/null\r
++++ b/test/atomicity.gdb\r
+@@ -0,0 +1,50 @@\r
++# This gdb script runs notmuch new and simulates killing and\r
++# restarting notmuch new after every Xapian commit. To simulate this\r
++# more efficiently, this script runs notmuch new and, immediately\r
++# after every Xapian commit, it *pauses* the running notmuch new,\r
++# copies the entire database and maildir to a snapshot directory, and\r
++# executes a full notmuch new on that snapshot, comparing the final\r
++# results with the expected output. It can then resume the paused\r
++# notmuch new, which is still running on the original maildir, and\r
++# repeat this process.\r
++\r
++set args new\r
++\r
++# Make Xapian commit after every operation instead of batching\r
++set environment XAPIAN_FLUSH_THRESHOLD = 1\r
++\r
++# gdb can't keep track of a simple integer. This is me weeping.\r
++shell echo 0 > outcount\r
++\r
++shell touch inodes\r
++\r
++break rename\r
++commands\r
++# As an optimization, only consider snapshots after a Xapian commit.\r
++# Xapian overwrites record.base? as the last step in the commit.\r
++shell echo > gdbcmd\r
++shell stat -c %i $MAIL_DIR/.notmuch/xapian/record.base* > inodes.new\r
++shell if cmp inodes inodes.new; then echo cont > gdbcmd; fi\r
++shell mv inodes.new inodes\r
++source gdbcmd\r
++\r
++# Save a backtrace in case the test does fail\r
++set logging file backtrace\r
++set logging on\r
++backtrace\r
++set logging off\r
++shell mv backtrace backtrace.`cat outcount`\r
++\r
++# Snapshot the database\r
++shell rm -r $MAIL_DIR.snap/.notmuch\r
++shell cp -r $MAIL_DIR/.notmuch $MAIL_DIR.snap/.notmuch\r
++# Restore the mtime of $MAIL_DIR.snap, which we just changed\r
++shell touch -r $MAIL_DIR $MAIL_DIR.snap\r
++# Run notmuch new to completion on the snapshot\r
++shell NOTMUCH_CONFIG=${NOTMUCH_CONFIG}.snap XAPIAN_FLUSH_THRESHOLD=1000 notmuch new > /dev/null\r
++shell NOTMUCH_CONFIG=${NOTMUCH_CONFIG}.snap notmuch search '*' > search.`cat outcount` 2>&1\r
++shell echo $(expr $(cat outcount) + 1) > outcount\r
++cont\r
++end\r
++\r
++run\r
+diff --git a/test/basic b/test/basic\r
+index 808c968..5e0adf0 100755\r
+--- a/test/basic\r
++++ b/test/basic\r
+@@ -60,6 +60,7 @@ available=$(ls -1 ../ | \\r
+ -e "/^(test.expected-output|.*~)/d" \\r
+ -e "/^(gnupg-secret-key.asc)/d" \\r
+ -e "/^(gnupg-secret-key.NOTE)/d" \\r
++ -e "/^(atomicity.gdb)/d" \\r
+ | sort)\r
+ test_expect_equal "$tests_in_suite" "$available"\r
+ \r
+diff --git a/test/notmuch-test b/test/notmuch-test\r
+index 9f58c12..c540228 100755\r
+--- a/test/notmuch-test\r
++++ b/test/notmuch-test\r
+@@ -40,6 +40,7 @@ TESTS="\r
+ emacs-large-search-buffer\r
+ maildir-sync\r
+ crypto\r
++ atomicity\r
+ "\r
+ TESTS=${NOTMUCH_TESTS:=$TESTS}\r
+ \r
+-- \r
+1.7.5.1\r
+\r
projects / notmuch-archives.git / commitdiff