Return-Path: <markwalters1009@gmail.com>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id CED26431FB6
	for <notmuch@notmuchmail.org>; Mon, 23 Jun 2014 13:12:46 -0700 (PDT)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: 0.201
X-Spam-Level: 
X-Spam-Status: No, score=0.201 tagged_above=-999 required=5
	tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,
	FREEMAIL_ENVFROM_END_DIGIT=1, FREEMAIL_FROM=0.001,
	RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id KZhO+mMxiubE for <notmuch@notmuchmail.org>;
	Mon, 23 Jun 2014 13:12:42 -0700 (PDT)
Received: from mail-we0-f176.google.com (mail-we0-f176.google.com
	[74.125.82.176]) (using TLSv1 with cipher RC4-SHA (128/128 bits))
	(No client certificate requested)
	by olra.theworths.org (Postfix) with ESMTPS id 79CFC431FAF
	for <notmuch@notmuchmail.org>; Mon, 23 Jun 2014 13:12:42 -0700 (PDT)
Received: by mail-we0-f176.google.com with SMTP id u56so7781609wes.35
	for <notmuch@notmuchmail.org>; Mon, 23 Jun 2014 13:12:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=from:to:cc:subject:date:message-id:in-reply-to:references;
	bh=x7pbQ1dkhjnOGE2vFTIL3+Rp69xxQzGhYctHkRhegB4=;
	b=dZp6dnlVd0N3vVrRUEfTMk4E1J1dYRB2ZZUL5AaMD3AaA6YP++ax8AVaAtk4gpOaDS
	bOIFcPaCIn+EmrBYT+KVkBviBMd72J9+6XST84I31Etpyo5dgybNJvFDx1y6cb6HcnsY
	hn5FV/8gQe2fMJoy49AE1rSC2p+kP8NVtLc/aBsuUqBlXeHSP8wdMpzjt7Ksgu1x8DCk
	1h2ts4y9W/94Ty7RvmSmNGPFOOKZBjCs+zT6He8EiKbU6xQWG++j1cUgerN7zAwjpIA3
	WXf0TR/+op1oKIjfsIJvzhcPfV7u1RIaxCwe3WF38y4ibHN6j8BJyshXRRHxfWl+Z+F3
	DZRQ==
X-Received: by 10.194.90.106 with SMTP id bv10mr31064118wjb.20.1403554359229; 
	Mon, 23 Jun 2014 13:12:39 -0700 (PDT)
Received: from localhost (5751dfa2.skybroadband.com. [87.81.223.162])
	by mx.google.com with ESMTPSA id
	fb15sm36378270wid.23.2014.06.23.13.12.38 for <multiple recipients>
	(version=TLSv1.2 cipher=RC4-SHA bits=128/128);
	Mon, 23 Jun 2014 13:12:38 -0700 (PDT)
From: Mark Walters <markwalters1009@gmail.com>
To: notmuch@notmuchmail.org
Subject: [PATCH] dump: make dump take Xapian write lock
Date: Mon, 23 Jun 2014 21:12:29 +0100
Message-Id: <1403554349-8888-1-git-send-email-markwalters1009@gmail.com>
X-Mailer: git-send-email 1.7.10.4
In-Reply-To: <87zjhh67e7.fsf@qmul.ac.uk>
References: <87zjhh67e7.fsf@qmul.ac.uk>
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Mon, 23 Jun 2014 20:12:47 -0000

Dump currently only takes the read lock. Xapian can cope with some
changes while maintaining a read snapshot but with more changes it
fails. Currently notmuch just gives a xapian error.

To avoid this we take the write lock when dumping. This prevents other
notmuch processes from modifying the xapian database preventing this
error.

Discussion with Olly on irc indicates that this is currently the best
solution: in xapian trunk there may be better possibilities using
snapshots but they need to make it to a release and propogate out to
users before we can switch approach.

Finally, this breaks one use case: pipelines of the form

notmuch dump | ... | notmuch restore

According to Olly this is already very fragile: it will only work on
small databases. One of the tests relies on this behaviour so fix that
to store the dump rather than use a pipe.
---
I haven't tested this much: all tests pass (with the one fix) and it is simple.

Best wishes

Mark


 notmuch-dump.c       |    2 +-
 test/T150-tagging.sh |    4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 887a208..9c6ad7f 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -212,7 +212,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])
     int ret;
 
     if (notmuch_database_open (notmuch_config_get_database_path (config),
-			       NOTMUCH_DATABASE_MODE_READ_ONLY, &notmuch))
+			       NOTMUCH_DATABASE_MODE_READ_WRITE, &notmuch))
 	return EXIT_FAILURE;
 
     char *output_file_name = NULL;
diff --git a/test/T150-tagging.sh b/test/T150-tagging.sh
index dc118f3..45471ac 100755
--- a/test/T150-tagging.sh
+++ b/test/T150-tagging.sh
@@ -247,8 +247,8 @@ ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} \
 notmuch dump --format=batch-tag | sed 's/^.* -- /+common_tag -- /' | \
     sort > EXPECTED
 
-notmuch dump --format=batch-tag | sed 's/^.* -- /  -- /' | \
-    notmuch restore --format=batch-tag
+notmuch dump --format=batch-tag | sed 's/^.* -- /  -- /' > INTERMEDIATE_STEP
+notmuch restore --format=batch-tag < INTERMEDIATE_STEP
 
 notmuch tag --batch < EXPECTED
 
-- 
1.7.10.4