Return-Path: <jani@nikula.org>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 1AAB6431FBD
	for <notmuch@notmuchmail.org>; Sat, 25 Jan 2014 06:59:27 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: -0.7
X-Spam-Level: 
X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5
	tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id AbtrUtCZGzGB for <notmuch@notmuchmail.org>;
	Sat, 25 Jan 2014 06:59:18 -0800 (PST)
Received: from mail-ee0-f43.google.com (mail-ee0-f43.google.com
 [74.125.83.43])	(using TLSv1 with cipher RC4-SHA (128/128 bits))	(No client
 certificate requested)	by olra.theworths.org (Postfix) with ESMTPS id
 404BD431FBC	for <notmuch@notmuchmail.org>; Sat, 25 Jan 2014 06:59:18 -0800
 (PST)
Received: by mail-ee0-f43.google.com with SMTP id c41so1458527eek.2
	for <notmuch@notmuchmail.org>; Sat, 25 Jan 2014 06:59:15 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20130820;
	h=x-gm-message-state:from:to:subject:in-reply-to:references
	:user-agent:date:message-id:mime-version:content-type;
	bh=WzV0sKZ1Z9hOTB5VtkyP0My7aoGQpE+PLXk/C+Rd/vU=;
	b=aBLjonhI2BtBHTbRCkD3V3EbDxvl9np1jLrJpJO6+1XNTRhSs5C8lhJQK5SlmWsuUm
	4rz+kxQEEKPMJ1bDfW2U29gXx0izbnRPpcA+E0O6lyRNiygIwwZT94WOEs9EdHXz53KR
	P3fHQYL+Nc6nqczA5yMJi2J4mnwBpZsQrbd7WSRmEykyBHCPsOLDIH9oiEpubN4g1X1I
	Enx5mchLHBkdEHRsqum3o2WAyarn4uNWNn4hlJtGhepCKrlPt0N6mZE2Q4ze5MOq2imZ
	JbVJQcpJuCSU2Vu2hpfF7ZUUbyjFb/9uzx2qFVWu3MdplDBJ9G5gOo057ZOcxqJ/Te8O
	/C2Q==
X-Gm-Message-State:
 ALoCoQm1eLbJSoTQmCpuF9nW0ZD8mwJKukirxEIDtrjquShccWQJ+uBb5OzC0RbgRm7ooocBcCYk
X-Received: by 10.15.43.10 with SMTP id w10mr17785277eev.13.1390661955569;
	Sat, 25 Jan 2014 06:59:15 -0800 (PST)
Received: from localhost (dsl-hkibrasgw2-58c36f-91.dhcp.inet.fi.
	[88.195.111.91])
	by mx.google.com with ESMTPSA id w4sm16827820eef.20.2014.01.25.06.59.13
	for <multiple recipients>
	(version=TLSv1.2 cipher=RC4-SHA bits=128/128);
	Sat, 25 Jan 2014 06:59:14 -0800 (PST)
From: Jani Nikula <jani@nikula.org>
To: Mark Walters <markwalters1009@gmail.com>, notmuch@notmuchmail.org
Subject: automatic database upgrades (was: Re: [PATCH 0/7] cli: notmuch new
	improvements)
In-Reply-To: <87ppnga21o.fsf@qmul.ac.uk>
References: <cover.1390163335.git.jani@nikula.org> <87ppnga21o.fsf@qmul.ac.uk>
User-Agent: Notmuch/0.17+44~ge3b4cd9 (http://notmuchmail.org) Emacs/24.3.1
	(x86_64-pc-linux-gnu)
Date: Sat, 25 Jan 2014 16:59:12 +0200
Message-ID: <87lhy4f6pr.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: text/plain
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Sat, 25 Jan 2014 14:59:27 -0000

On Sat, 25 Jan 2014, Mark Walters <markwalters1009@gmail.com> wrote:
> This series LGTM.

Hi Mark, thanks for the review!

> I do now recall there was some discussion on irc about the automatic
> database upgrade: it would be good to have that documented but the
> consensus was to do it, so +1 from me.

Here's some summary, as promised. Please bear in mind that the
discussion is pretty much irrelevant to this particular patch
series. (We might discuss whether a warning about upgrade should be
printed to stderr also with --quiet, but IMHO that can be a follow-up
patch.)

A database upgrade is required when the user upgrades to a new version
of notmuch that has a modified database schema. See
id:cover.1389304779.git.jani@nikula.org for an example of a proposed
database change.

A database upgrade is a rare event. Most of the time, it's okay to go
back and forth with notmuch versions on the same database, but a
database upgrade is an irreversible process after which the user must
use the new version of notmuch. To go back requires a full rebuild of
the database.

We don't have recent experience with the database upgrades. The last
time it happened was before notmuch 0.1 (yes, 0.1) was released, when
the whole upgrade mechanism was introduced:

commit 909f52bd8c4bdfa11cd3e75e3d0959e0293689bd
Author: Carl Worth <cworth@cworth.org>
Date:   Thu Jan 7 18:26:31 2010 -0800

    lib: Implement versioning in the database and provide upgrade function.

Some of the points in favor of requiring manual intervention (such as
running 'notmuch new --upgrade' or a new command 'notmuch upgrade')
before upgrading the database:

* The database upgrade is an irreversible process. The user should have
  a chance to decide whether to go ahead with that. You can't go back to
  the old notmuch and database version without rebuilding the database.

* The user should be given the chance to make backups first in case
  something goes wrong.

* The database upgrade might take a long time for large databases. The
  user should be able to choose when to go ahead with that.

Some of the points in favor of upgrading automatically:

* cworth: "One potential concern is that [requiring manual intervention]
  effectively breaks notmuch until the user intervenese and runs this
  new command. So that can complicate things for any interface that sits
  on top of notmuch."

* cworth: "In general, I'm often frustrated with programs that say "I
  cannot continue until you run command <foo>.". If command <foo> needs
  to be run, and the software knows it, why doesn't it just run it
  itself? [...] So a message like "Run 'notmuch upgrade'" seems it could
  corrode the user's trust in notmuch to maintain its own state."

* There are people who run notmuch new non-interactively. There's no
  easy answer to handling that if manual intervention is required.

* It should always be okay to kill the upgrade, and continue at a later
  time, in case it takes too long.

Reading the logs again, I'm not so confident about us reaching a
concensus. Maybe it was just me changing my mind during the course of
the discussion... so we can try to reach concensus here.


BR,
Jani.