From: Mark Walters <markwalters1009@gmail.com>
To: David Bremner <david@tethera.net>, Maarten Aertsen <sagi-notmuch@rtsn.nl>,
	notmuch@notmuchmail.org
Subject: Re: notmuch dump: taking write-lock to protect from concurrent
	(cronned) notmuch new?
In-Reply-To: <87mwdhg2jh.fsf@zancas.localnet>
References: <20140606080354.28047.77393@kardo.rtsn.nl>
	<878upafegc.fsf@qmul.ac.uk> <87mwdhg2jh.fsf@zancas.localnet>
User-Agent: Notmuch/0.15.2+615~g78e3a93 (http://notmuchmail.org) Emacs/23.4.1
	(x86_64-pc-linux-gnu)
Date: Fri, 13 Jun 2014 00:21:52 +0100
Message-ID: <87zjhh67e7.fsf@qmul.ac.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list


Hi

On Thu, 12 Jun 2014, David Bremner <david@tethera.net> wrote:
> Mark Walters <markwalters1009@gmail.com> writes:
>
>
>>> mjw1009 suggested to change NOTMUCH_DATABASE_MODE_READ_ONLY on line
>>> 215 of notmuch-dump.c to NOTMUCH_DATABASE_MODE_READ_WRITE
>>>
>>> I'm wondering if this hits enough people to motivate the addition of a
>>> command line switch (or perhaps even a change in default behaviour?)
>>
>> I think this is a clear bug but the fix is a little unclear. The above
>> fix works but it breaks one of the tests: "unicode message-ids" in
>> T150-tagging.sh.
>>
>> I think the problem is that it does 
>> notmuch dump | sed... | notmuch restore
>>
>
> My first reaction was "argh, we should be locking things less, not
> more".  But then I read 
>
>         http://getting-started-with-xapian.readthedocs.org/en/latest/xapian-core-rst/admin_notes.html?highlight=backup#id10
>
> and now I'm not so sure, maybe write lock for dump is the right answer.

On irc Olly said

"I'd suggest locking the db by opening it R/W for the dump at least
until you can use reader locking to keep the read revision valid, but
it'll be a while before that's in a stable release"

and he also said that pipes of the form notmuch dump | ... notmuch
restore will probably fail if they change many tags.

So it is probably the way to go. But it does run the risk of breaking
some peoples (already fragile) scripts.

> It seems hard to do anything sensible with "Database.reopen" in the
> context of a backup.

Yes I agree.

Best wishes

Mark