From: Mark Walters <markwalters1009@gmail.com>
To: Jani Nikula <jani@nikula.org>, notmuch@notmuchmail.org
Subject: Re: automatic database upgrades (was: Re: [PATCH 0/7] cli: notmuch
	new improvements)
In-Reply-To: <87lhy4f6pr.fsf@nikula.org>
References: <cover.1390163335.git.jani@nikula.org> <87ppnga21o.fsf@qmul.ac.uk>
	<87lhy4f6pr.fsf@nikula.org>
User-Agent: Notmuch/0.15.2+484~gfb59956 (http://notmuchmail.org) Emacs/23.4.1
	(x86_64-pc-linux-gnu)
Date: Sat, 25 Jan 2014 17:32:04 +0000
Message-ID: <87fvoc9dd7.fsf@qmul.ac.uk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list


Thanks for posting this. You are quite right about it being orthogonal
to this series so a clear +1 from me for the series.

What about a config option? Something like
database_auto_upgrade=true/false? I wouldn't have a strong preference
which was the default (though I would choose "false" in my own
config). I guess we would need a command line --upgrade to allow people
with database_auto_upgrade=false to do force/allow the upgrade.

Best wishes

Mark








On Sat, 25 Jan 2014, Jani Nikula <jani@nikula.org> wrote:
> On Sat, 25 Jan 2014, Mark Walters <markwalters1009@gmail.com> wrote:
>> This series LGTM.
>
> Hi Mark, thanks for the review!
>
>> I do now recall there was some discussion on irc about the automatic
>> database upgrade: it would be good to have that documented but the
>> consensus was to do it, so +1 from me.
>
> Here's some summary, as promised. Please bear in mind that the
> discussion is pretty much irrelevant to this particular patch
> series. (We might discuss whether a warning about upgrade should be
> printed to stderr also with --quiet, but IMHO that can be a follow-up
> patch.)
>
> A database upgrade is required when the user upgrades to a new version
> of notmuch that has a modified database schema. See
> id:cover.1389304779.git.jani@nikula.org for an example of a proposed
> database change.
>
> A database upgrade is a rare event. Most of the time, it's okay to go
> back and forth with notmuch versions on the same database, but a
> database upgrade is an irreversible process after which the user must
> use the new version of notmuch. To go back requires a full rebuild of
> the database.
>
> We don't have recent experience with the database upgrades. The last
> time it happened was before notmuch 0.1 (yes, 0.1) was released, when
> the whole upgrade mechanism was introduced:
>
> commit 909f52bd8c4bdfa11cd3e75e3d0959e0293689bd
> Author: Carl Worth <cworth@cworth.org>
> Date:   Thu Jan 7 18:26:31 2010 -0800
>
>     lib: Implement versioning in the database and provide upgrade function.
>
> Some of the points in favor of requiring manual intervention (such as
> running 'notmuch new --upgrade' or a new command 'notmuch upgrade')
> before upgrading the database:
>
> * The database upgrade is an irreversible process. The user should have
>   a chance to decide whether to go ahead with that. You can't go back to
>   the old notmuch and database version without rebuilding the database.
>
> * The user should be given the chance to make backups first in case
>   something goes wrong.
>
> * The database upgrade might take a long time for large databases. The
>   user should be able to choose when to go ahead with that.
>
> Some of the points in favor of upgrading automatically:
>
> * cworth: "One potential concern is that [requiring manual intervention]
>   effectively breaks notmuch until the user intervenese and runs this
>   new command. So that can complicate things for any interface that sits
>   on top of notmuch."
>
> * cworth: "In general, I'm often frustrated with programs that say "I
>   cannot continue until you run command <foo>.". If command <foo> needs
>   to be run, and the software knows it, why doesn't it just run it
>   itself? [...] So a message like "Run 'notmuch upgrade'" seems it could
>   corrode the user's trust in notmuch to maintain its own state."
>
> * There are people who run notmuch new non-interactively. There's no
>   easy answer to handling that if manual intervention is required.
>
> * It should always be okay to kill the upgrade, and continue at a later
>   time, in case it takes too long.
>
> Reading the logs again, I'm not so confident about us reaching a
> concensus. Maybe it was just me changing my mind during the course of
> the discussion... so we can try to reach concensus here.
>
>
> BR,
> Jani.