From: Sebastien Binet Date: Thu, 19 Jul 2012 08:13:25 +0000 (+0200) Subject: Re: query on a subset of messages ? X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=8a17eec745b2942693e3b1dd748d0fa1a2d1f81a;p=notmuch-archives.git Re: query on a subset of messages ? --- diff --git a/b8/481f7df754f1ad9fa83594d48043c1214117d1 b/b8/481f7df754f1ad9fa83594d48043c1214117d1 new file mode 100644 index 000000000..ecf8aa9c1 --- /dev/null +++ b/b8/481f7df754f1ad9fa83594d48043c1214117d1 @@ -0,0 +1,145 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 44EBD431FC0 + for ; Thu, 19 Jul 2012 01:13:52 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -5 +X-Spam-Level: +X-Spam-Status: No, score=-5 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_HI=-5] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id BeLaeLKVqzwI for ; + Thu, 19 Jul 2012 01:13:51 -0700 (PDT) +Received: from CERNMX32.cern.ch (cernmx32.cern.ch [137.138.144.178]) + (using TLSv1 with cipher AES128-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 3F728431FAE + for ; Thu, 19 Jul 2012 01:13:51 -0700 (PDT) +Received: from CERNFE22.cern.ch (137.138.144.151) by cernmxgwlb2.cern.ch + (137.138.144.178) with Microsoft SMTP Server (TLS) id 14.2.283.3; + Thu, 19 Jul 2012 10:13:46 +0200 +Received: from localhost (134.158.88.68) by smtp.cern.ch (137.138.144.172) + with Microsoft SMTP Server (TLS) id 14.2.298.4; + Thu, 19 Jul 2012 10:13:46 +0200 +From: Sebastien Binet +To: Austin Clements +Subject: Re: query on a subset of messages ? +In-Reply-To: <87obno3luq.fsf@cern.ch> +References: <871ukl5oj7.fsf@cern.ch> <20120709163000.GG18195@mit.edu> + <87obno3luq.fsf@cern.ch> +User-Agent: Notmuch/0.13.2+77~g39beeb2 (http://notmuchmail.org) Emacs/24.1.1 + (x86_64-unknown-linux-gnu) +Date: Thu, 19 Jul 2012 10:13:25 +0200 +Message-ID: <87sjcoi2xm.fsf@cern.ch> +MIME-Version: 1.0 +Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha1; + protocol="application/pgp-signature" +X-Originating-IP: [134.158.88.68] +Keywords: CERN SpamKiller Note: -50 +Cc: Notmuch developer list +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 19 Jul 2012 08:13:52 -0000 + +--=-=-= +Content-Type: text/plain +Content-Transfer-Encoding: quoted-printable + +Sebastien Binet writes: + +> Austin Clements writes: +> +>> Quoth Sebastien Binet on Jul 09 at 10:25 am: +>>>=20 +>>> hi there, +>>>=20 +>>> I was trying to reduce the I/O stress during my usual email +>>> fetching+tagging by writing a little program using the go bindings to +>>> notmuch. +>>>=20 +>>> ie: +>>> db, status :=3D notmuch.OpenDatabase(db_path, +>>> notmuch.DATABASE_MODE_READ_WRITE) +>>> query :=3D db.CreateQuery("(tag:new AND tag:inbox)") +>>> msgs :=3D query.SearchMessages() +>>> for _,msg :=3D range msgs { +>>> tag_msg(msg, tagqueries) +>>> } +>>>=20 +>>>=20 +>>> where tagqueries is a subquery of the form: +>>> [ +>>> { +>>> "Cmd": "+to-me", +>>> "Query": "(to:sebastien.binet@cern.ch and not tag:to-me)" +>>> }, +>>> { +>>> "Cmd": "+sci-notmuch", +>>> "Query": "from:notmuch@notmuchmail.org or to:notmuch@notmuchmai= +l.org or subject:notmuch" +>>> } +>>> ] +>>>=20 +>>>=20 +>>> the idea being that I only need to crawl through the db only once and +>>> then iteratively apply tags on those messages (instead of repeatedly +>>> running "notmuch tag ..." for each and every of those many +>>> 'tag-queries') +>>>=20 +>>> I couldn't find any C-API to do such a thing using the notmuch library. +>>> did I overlook something ? +>>>=20 +>>> Is it something useful to add ? +>>>=20 +>>> -s +>> +>> Have you tried a more direct translation of the multiple notmuch tag +>> commands into Go, where you don't worry about subsetting the queries? +>> Unless you're tagging a huge number of messages, the cost of notmuch +>> tag is almost certainly the fsync that it does when it closes the +>> database (which every call to notmuch tag must do). However, in Go, +>> you can keep the database open across all of the tagging operations +>> and then close and fsync it just once. +> +> nope, I haven't tried that, but will do. +> +>> +>> Note that there is an important optimization in notmuch tag that you +>> might have to replicate. It manipulates the original query to exclude +>> messages that already have the desired tags, so that they get skipped +>> very efficiently at the earliest stage possible. +> I already have this in my original shell script. +> (wouldn't be too hard to automatically do, though.) + +FYI, I've put this into a new notmuch-mtag go-based binary over here: +https://github.com/sbinet/notmuch/blob/dev/go-bindings/bindings/go/src/notm= +uch-mtag/main.go + + +=2Ds + +--=-=-= +Content-Type: application/pgp-signature + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v2.0.19 (GNU/Linux) + +iEYEARECAAYFAlAHwawACgkQPj4AVbsZCCaJrwCglm3LMNwaYJuCCCWGmGHt9WGv +KB4An3LxEhxb//8thxBUNoOHQq5kluqn +=1b+2 +-----END PGP SIGNATURE----- +--=-=-=--