From: Carl Worth <cworth@cworth.org>
To: notmuch@notmuchmail.org
Date: Sun, 10 Jan 2010 09:12:22 -0800
Message-ID: <87r5pxlw95.fsf@yoom.home.cworth.org>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha1; protocol="application/pgp-signature"
Subject: [notmuch] Rename support and database upgrade
Precedence: list

--=-=-=
Content-Transfer-Encoding: quoted-printable

Hi notmuch users,

I know that things have been fairly silent from me on both the mailing
list and the git repository lately. Hopefully that will change now.

Just yesterday I merged in my "rename-support" branch to master (it's
about 60 commits and took me two or three weeks for what I originally
thought would be two or three hours of work).

The big feature here is that notmuch now deals sanely with files being
deleted or renamed within the mail store. This is a feature we've talked
about for some time that will make it feasible to use notmuch while also
using various other email software on the same mail store.

To support this feature, I did change some details about how we store
data in the Xapian database. I wanted to avoid a flag day where all
users would have to dump, rebuild, and restore their database, (with
risk of making a mistake and losing data). So instead, I added
versioning to the database and implemented an automatic upgrade of the
database content. I've tested this upgrade support enough to be have
quite a bit of confidence in it.

So after you upgrade your notmuch, the next run of "notmuch new" will
detect that the database needs to be upgraded and will do that
automatically. The upgrade process does take quite a bit of time, but is
about 4 times faster than building the same database from scratch (even
ignoring the time it would take to restore all tags.

Oh, and that "4 times faster" result is with an upgraded Xapian as well,
(with the recent fix for defect #250). I'll say more about Xapian in a
separate post. But even if you have an older Xapian, the upgrade should
be no slower than a rebuild, (and still avoids the step to restore all
tags which is *really* slow with old Xapian).

So I definitely recommend everyone try the upgrade path. Of course, it's
always possible that something will go very wrong with the process. So
it's still a good idea to do a "notmuch dump" to save all your tags
before beginning. (It's really a good idea to do a "notmuch tag" on a
regular basis anyway---such as just before nightly backups or similar.)

Anyway, I just wanted to let people know what to expect so they wouldn't
be surprised by the "please wait while upgrading" message from "notmuch
new". And hopefully I'll now be able to start working through the
backlog of mail and patches I've got. I've skimmed things and it looks
like there's some really interesting stuff in there. So thanks!

=2DCarl
=20

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iD8DBQFLSgp26JDdNq8qSWgRAotmAJ4yTZ/UN5/Okr1hfT0zO4TYMdrHNwCeP4mR
wMOaOeMvWSwD42lVCP1c1Dw=
=QhhE
-----END PGP SIGNATURE-----
--=-=-=--