Re: query on a subset of messages ?
authorAustin Clements <amdragon@MIT.EDU>
Mon, 9 Jul 2012 16:30:00 +0000 (12:30 +2000)
committerW. Trevor King <wking@tremily.us>
Fri, 7 Nov 2014 17:48:04 +0000 (09:48 -0800)
cc/1d8c3a5355e4a33022ff94d48c86b3487049f9 [new file with mode: 0644]

diff --git a/cc/1d8c3a5355e4a33022ff94d48c86b3487049f9 b/cc/1d8c3a5355e4a33022ff94d48c86b3487049f9
new file mode 100644 (file)
index 0000000..e1e2de7
--- /dev/null
@@ -0,0 +1,131 @@
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 4B71E431FBD\r
+       for <notmuch@notmuchmail.org>; Mon,  9 Jul 2012 09:30:08 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id PzMxaJqX0H-e for <notmuch@notmuchmail.org>;\r
+       Mon,  9 Jul 2012 09:30:07 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU\r
+       [18.9.25.15])\r
+       by olra.theworths.org (Postfix) with ESMTP id 8DB73431FAE\r
+       for <notmuch@notmuchmail.org>; Mon,  9 Jul 2012 09:30:07 -0700 (PDT)\r
+X-AuditID: 1209190f-b7f306d0000008b4-82-4ffb070ee00b\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+       by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id F4.9D.02228.E070BFF4; Mon,  9 Jul 2012 12:30:06 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+       by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id q69GU5d9023574; \r
+       Mon, 9 Jul 2012 12:30:05 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+       (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q69GU0KR019822\r
+       (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+       Mon, 9 Jul 2012 12:30:01 -0400 (EDT)\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77)\r
+       (envelope-from <amdragon@mit.edu>)\r
+       id 1SoGqW-0001Zy-II; Mon, 09 Jul 2012 12:30:00 -0400\r
+Date: Mon, 9 Jul 2012 12:30:00 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Sebastien Binet <binet@cern.ch>\r
+Subject: Re: query on a subset of messages ?\r
+Message-ID: <20120709163000.GG18195@mit.edu>\r
+References: <871ukl5oj7.fsf@cern.ch>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <871ukl5oj7.fsf@cern.ch>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFmpkleLIzCtJLcpLzFFi42IRYrdT1+Vj/+1v0HuPx2Li81CL6zdnMjsw\r
+       eTw6N5XR49mqW8wBTFFcNimpOZllqUX6dglcGXM3H2Ir+MNX0fnnDXsD41HuLkZODgkBE4n9\r
+       S7+zQ9hiEhfurWfrYuTiEBLYxyix/cc3RghnPaPEw40LWSCcE0wSjd/mAjkcQM4SRonHNSDd\r
+       LAIqEm//L2MEsdkENCS27V8OZosIKEscP7WGDcRmBtq29P0VFhBbWEBH4u/Ni2BxXiB736/P\r
+       YFcIAc3pbznLDBEXlDg58wkLRK+WxI1/L5lA1jILSEss/8cBEuYUUJW49usbWKsoUOuUk9vY\r
+       JjAKzULSPQtJ9yyE7gWMzKsYZVNyq3RzEzNzilOTdYuTE/PyUot0TfRyM0v0UlNKNzGCQppT\r
+       kn8H47eDSocYBTgYlXh4N7H89hdiTSwrrsw9xCjJwaQkynuRESjEl5SfUpmRWJwRX1Sak1p8\r
+       iFGCg1lJhHfR11/+QrwpiZVVqUX5MClpDhYlcd6rKTf9hQTSE0tSs1NTC1KLYLIyHBxKErx9\r
+       bEBDBYtS01Mr0jJzShDSTBycIMN5gIZHgNTwFhck5hZnpkPkTzEqSonzOoAkBEASGaV5cL2w\r
+       lPOKURzoFWHeHJAqHmC6gut+BTSYCWhwew/I1cUliQgpqQZGls0XBQ/dM7plm37P85xXn+Fv\r
+       sZ0v3jK/Wnal9FDB9med3VeWxVod3L/DPK9//vUGj9xdO+VuGf3rWMDV5vOn3D37bvrsv58F\r
+       H/zgyem+rMQdv6/P8+nXS49nBFY/ffFm96yHTPfVdDVPpAaIfL6++JiM1PsfV63Ynud8YOG/\r
+       0PBpzdfOPV253kosxRmJhlrMRcWJAKzczs0UAwAA\r
+Cc: Notmuch developer list <notmuch@notmuchmail.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 09 Jul 2012 16:30:08 -0000\r
+\r
+Quoth Sebastien Binet on Jul 09 at 10:25 am:\r
+> \r
+> hi there,\r
+> \r
+> I was trying to reduce the I/O stress during my usual email\r
+> fetching+tagging by writing a little program using the go bindings to\r
+> notmuch.\r
+> \r
+> ie:\r
+> db, status := notmuch.OpenDatabase(db_path,\r
+>              notmuch.DATABASE_MODE_READ_WRITE)\r
+> query := db.CreateQuery("(tag:new AND tag:inbox)")\r
+> msgs := query.SearchMessages()\r
+> for _,msg := range msgs {\r
+>   tag_msg(msg, tagqueries)\r
+> }\r
+> \r
+> \r
+> where tagqueries is a subquery of the form:\r
+> [\r
+>     {\r
+>         "Cmd": "+to-me",\r
+>         "Query": "(to:sebastien.binet@cern.ch and not tag:to-me)"\r
+>     },\r
+>     {\r
+>         "Cmd": "+sci-notmuch",\r
+>         "Query": "from:notmuch@notmuchmail.org or to:notmuch@notmuchmail.org or subject:notmuch"\r
+>     }\r
+> ]\r
+> \r
+> \r
+> the idea being that I only need to crawl through the db only once and\r
+> then iteratively apply tags on those messages (instead of repeatedly\r
+> running "notmuch tag ..." for each and every of those many\r
+> 'tag-queries')\r
+> \r
+> I couldn't find any C-API to do such a thing using the notmuch library.\r
+> did I overlook something ?\r
+> \r
+> Is it something useful to add ?\r
+> \r
+> -s\r
+\r
+Have you tried a more direct translation of the multiple notmuch tag\r
+commands into Go, where you don't worry about subsetting the queries?\r
+Unless you're tagging a huge number of messages, the cost of notmuch\r
+tag is almost certainly the fsync that it does when it closes the\r
+database (which every call to notmuch tag must do).  However, in Go,\r
+you can keep the database open across all of the tagging operations\r
+and then close and fsync it just once.\r
+\r
+Note that there is an important optimization in notmuch tag that you\r
+might have to replicate.  It manipulates the original query to exclude\r
+messages that already have the desired tags, so that they get skipped\r
+very efficiently at the earliest stage possible.\r