From: Carl Worth <cworth@cworth.org>
To: david@tethera.net, notmuch@notmuchmail.org
In-Reply-To: <87my1z97un.wl%bremner@pivot.cs.unb.ca>
References: <87zl60xjl7.wl%bremner@pivot.cs.unb.ca>
	<87my1z97un.wl%bremner@pivot.cs.unb.ca>
Date: Fri, 05 Feb 2010 17:41:48 -0800
Message-ID: <87ock3cf8j.fsf@yoom.home.cworth.org>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha1; protocol="application/pgp-signature"
Subject: Re: [notmuch] hack to retag a directory
Precedence: list

--=-=-=
Content-Transfer-Encoding: quoted-printable

On Thu, 03 Dec 2009 15:26:56 -0400, david@tethera.net wrote:
> > I think this will be obsolete pretty soon when the equivalent is
> > built-in to notmuch, but in the mean time, here is a script that
> > somebody might find useful: retag a whole directory (recursively). I
> > don't claim it is nice in any way, but it seems usable for me, taking
> > about 5 seconds to retag a directory containing 1000 messages.
>=20
> Sigh. And of course the version I posted was broken. I put a fixed versio=
n at=20
>=20
>       http://pivot.cs.unb.ca/git/?p=3Dnotmuch-scripts.git;a=3Dblob_plain;=
f=3Dtagdir;hb=3DHEAD

Thanks for sharing that, David.

Obviously, we now have outstanding patches to provide search
capabilities based on the folder containing messages. So once that gets
merged, you shouldn't need this script anymore.

> You might, or might not also be interested in=20
>=20
>     http://pivot.cs.unb.ca/git/?p=3Dnotmuch-scripts.git;a=3Dblob_plain;f=
=3Dgitmuch;hb=3DHEAD
>=20
> which is the beginnings of how to keep tags in git (for syncing
> between machines).

But this one looks quite interesting. Obviously, it's not a complex
script, but it looks pretty handy to me. I might start using this to
have a little history of my tag changes, (rather than just including the
dump output in occasional backups like I have been doing).

And it's interesting that this script might be just good enough for the
synchronization needs of some people. It's not integrated, and might
require manual fixup of any resulting git conflicts, but it might be
handy for some.

The biggest problem I see is that if I were to read some messages
locally, and then run "gitmuch restore" then this would wipe out the
local changes I had made. So we'll definitely want a more integrated
solution to eliminate the chance of problems like this.

>                     Right now the notmuch restore step is the
> bottleneck, but Carl apparently knows how to speed 'notmuch restore'
> up.

One easy answer is to just make "notmuch restore" do nothing for
messages where the existing tags are the same as the tags mentioned in
the input file. I just pushed a change to implement this, (along with
new tests for "notmuch dump" and "notmuch restore" of course).

For me, this takes a "notmuch restore" right after a "notmuch dump" from
about 10 minutes down to 1 minute, (and it was about 2 hours before the
Xapian Defect #250 fix).

The other idea that I didn't do yet is to change "notmuch restore" to do
a single search for all messages rather than N searches each resulting
in 1 message. But the 1-minute time I'm getting for "notmuch restore"
now is basically the same time required for a "notmuch dump", (which is
already doing a single global search). So perhaps Xapian is just plain
fast enough that a change like that won't help at all.

=2DCarl

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iD8DBQFLbMjc6JDdNq8qSWgRAv/JAJ0Ww9bYrhClItuBwMjYpGVYyUn3+gCghVes
iYbcAXtggeCYCuKgRl6ibzM=
=sKdH
-----END PGP SIGNATURE-----
--=-=-=--