From: Sebastien Binet <binet@cern.ch>
To: Austin Clements <amdragon@MIT.EDU>
Subject: Re: query on a subset of messages ?
In-Reply-To: <87obno3luq.fsf@cern.ch>
References: <871ukl5oj7.fsf@cern.ch> <20120709163000.GG18195@mit.edu>
	<87obno3luq.fsf@cern.ch>
User-Agent: Notmuch/0.13.2+77~g39beeb2 (http://notmuchmail.org) Emacs/24.1.1
	(x86_64-unknown-linux-gnu)
Date: Thu, 19 Jul 2012 10:13:25 +0200
Message-ID: <87sjcoi2xm.fsf@cern.ch>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1;
	protocol="application/pgp-signature"
Keywords: CERN SpamKiller Note: -50
Cc: Notmuch developer list <notmuch@notmuchmail.org>
Precedence: list

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Sebastien Binet <binet@cern.ch> writes:

> Austin Clements <amdragon@MIT.EDU> writes:
>
>> Quoth Sebastien Binet on Jul 09 at 10:25 am:
>>>=20
>>> hi there,
>>>=20
>>> I was trying to reduce the I/O stress during my usual email
>>> fetching+tagging by writing a little program using the go bindings to
>>> notmuch.
>>>=20
>>> ie:
>>> db, status :=3D notmuch.OpenDatabase(db_path,
>>>     		notmuch.DATABASE_MODE_READ_WRITE)
>>> query :=3D db.CreateQuery("(tag:new AND tag:inbox)")
>>> msgs :=3D query.SearchMessages()
>>> for _,msg :=3D range msgs {
>>>   tag_msg(msg, tagqueries)
>>> }
>>>=20
>>>=20
>>> where tagqueries is a subquery of the form:
>>> [
>>>     {
>>>         "Cmd": "+to-me",
>>>         "Query": "(to:sebastien.binet@cern.ch and not tag:to-me)"
>>>     },
>>>     {
>>>         "Cmd": "+sci-notmuch",
>>>         "Query": "from:notmuch@notmuchmail.org or to:notmuch@notmuchmai=
l.org or subject:notmuch"
>>>     }
>>> ]
>>>=20
>>>=20
>>> the idea being that I only need to crawl through the db only once and
>>> then iteratively apply tags on those messages (instead of repeatedly
>>> running "notmuch tag ..." for each and every of those many
>>> 'tag-queries')
>>>=20
>>> I couldn't find any C-API to do such a thing using the notmuch library.
>>> did I overlook something ?
>>>=20
>>> Is it something useful to add ?
>>>=20
>>> -s
>>
>> Have you tried a more direct translation of the multiple notmuch tag
>> commands into Go, where you don't worry about subsetting the queries?
>> Unless you're tagging a huge number of messages, the cost of notmuch
>> tag is almost certainly the fsync that it does when it closes the
>> database (which every call to notmuch tag must do).  However, in Go,
>> you can keep the database open across all of the tagging operations
>> and then close and fsync it just once.
>
> nope, I haven't tried that, but will do.
>
>>
>> Note that there is an important optimization in notmuch tag that you
>> might have to replicate.  It manipulates the original query to exclude
>> messages that already have the desired tags, so that they get skipped
>> very efficiently at the earliest stage possible.
> I already have this in my original shell script.
> (wouldn't be too hard to automatically do, though.)

FYI, I've put this into a new notmuch-mtag go-based binary over here:
https://github.com/sbinet/notmuch/blob/dev/go-bindings/bindings/go/src/notm=
uch-mtag/main.go


=2Ds

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlAHwawACgkQPj4AVbsZCCaJrwCglm3LMNwaYJuCCCWGmGHt9WGv
KB4An3LxEhxb//8thxBUNoOHQq5kluqn
=1b+2
-----END PGP SIGNATURE-----
--=-=-=--