Re: [notmuch] Asynchronous tagging

diff --git a/91/da56e7532e953916a4511de566affce3f10f06 b/91/da56e7532e953916a4511de566affce3f10f06
new file mode 100644 (file)
index 0000000..5d6f298
--- /dev/null
+++ b/91/da56e7532e953916a4511de566affce3f10f06
@@ -0,0 +1,116 @@
+Return-Path: <cworth@cworth.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 2C8F4431FC2;\r
+       Sat, 21 Nov 2009 12:01:33 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id Q23isl6Uuqdi; Sat, 21 Nov 2009 12:01:32 -0800 (PST)\r
+Received: from cworth.org (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id E1BAF431FBF;\r
+       Sat, 21 Nov 2009 12:01:31 -0800 (PST)\r
+From: Carl Worth <cworth@cworth.org>\r
+To: Jed Brown <jed@59A2.org>, notmuch@notmuchmail.org\r
+In-Reply-To: <87fx87u3no.fsf@59A2.org>\r
+References: <87fx87u3no.fsf@59A2.org>\r
+Date: Sat, 21 Nov 2009 21:01:20 +0100\r
+Message-ID: <874oon4pgv.fsf@yoom.home.cworth.org>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Subject: Re: [notmuch] Asynchronous tagging\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.12\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 21 Nov 2009 20:01:33 -0000\r
+\r
+On Sat, 21 Nov 2009 19:35:39 +0100, Jed Brown <jed@59A2.org> wrote:\r
+> I'm really enjoying notmuch, thanks!  I have a minor issue and a couple\r
+> observations worth noting.\r
+\r
+Thanks, Jed! And welcome to notmuch.\r
+\r
+> 1. Changing tags (like removing inbox/unread) has really high latency.\r
+\r
+Yes, this is a known bug in Xapian (it rewrites all of the indexed terms\r
+for the email message even though you're just trying to add/remove one\r
+term). The Xapian ticket for this is here:\r
+\r
+       replace_document should make minimal changes to database file\r
+       http://trac.xapian.org/ticket/250\r
+\r
+I've looked at the code, and it looks like it's going to be easy to\r
+fix. If anyone wants to try, here's the file to change:\r
+\r
+       xapian-core/backends/flint/flint_database.cc\r
+\r
+And look for:\r
+\r
+       // FIXME - in the case where there is overlap between the new\r
+       // termlist and the old termlist, it would be better to compare the\r
+       // two lists, and make the minimum set of modifications required.\r
+       // This would lead to smaller changesets for replication, and\r
+       // probably be faster overall\r
+\r
+So I think this might be as easy as just walking over two sorted lists\r
+looking for differences.\r
+\r
+Note that this is in the currently default "flint" backend, but the\r
+Xapian folks are probably more interested in fixing the in-development\r
+"chert" backend. So the patch to get upstreamed there will probably also\r
+fix:\r
+\r
+       xapian-core/backends/chert/chert_database.cc\r
+\r
+(I'm hoping the fix will be the same---an identical comment exists\r
+there.)\r
+\r
+Also, if you want to experiment with the chert backend, compile current\r
+Xapian source and run notmuch with XAPIAN_PREFER_CHERT=1. I haven't\r
+tried that yet, but there are claims that a chert database can be 40%\r
+smaller than an equivalent flint database.\r
+\r
+> 2. I have 'notmuch new' in an offlineimap postsync hook, but\r
+> notmuch-search-refresh-view occasionally complains that another process\r
+> has the lock (since I might press '=' when 'notmuch new' is running).\r
+> Waiting a moment and trying again works fine, but it would be nice to\r
+> clean this up eventually.\r
+\r
+Chris Wilson just contributed a patch to enable read-only usage of\r
+notmuch while another notmuch process holds the write lock. This should\r
+be very nice, (and means that new users will be able to start playing\r
+with notmuch even while the initial index creation is happening).\r
+\r
+> 3. I had initially put 'notmuch new' in a cron job (instead of\r
+> offlineimap postsync hook) and new/search would sometimes complain about\r
+> missing files in the maildir.  The first time this happened, it did not\r
+> correct itself and I ended up reimporting the database (I had moved some\r
+> things around so I could have been at fault).  Since then I have seen\r
+> these errors on a couple occasions, but they always go away upon\r
+> rerunning 'notmuch new'.  My guess is that it has to do with offlineimap\r
+> changing flags so I moved 'notmuch new' to the postsync hook and have\r
+> not seen the errors since.  If it is important that notmuch never runs\r
+> concurrently with an offlineimap sync, it should eventually go in the\r
+> docs.\r
+\r
+Thanks for the pointer.\r
+\r
+Does offlineimap use tmp while it's delivering message and then move\r
+things to new? If so, then maybe all we need to do to fix notmuch to not\r
+look into tmp directories?\r
+\r
+-Carl\r
+\r
+\r
+\r
+\r