[PATCH 0/3] Speed up notmuch new for unchanged directories

diff --git a/e5/c247ca37b58fcc6f90804ad3d2ffa303ba17e7 b/e5/c247ca37b58fcc6f90804ad3d2ffa303ba17e7
new file mode 100644 (file)
index 0000000..a46f7ab
--- /dev/null
+++ b/e5/c247ca37b58fcc6f90804ad3d2ffa303ba17e7
@@ -0,0 +1,75 @@
+Return-Path: <sascha-ml-email-notmuch-notmuch@silbe.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 978E4431FAF\r
+       for <notmuch@notmuchmail.org>; Sun, 24 Jun 2012 09:29:43 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id T0rOlydDZeIF for <notmuch@notmuchmail.org>;\r
+       Sun, 24 Jun 2012 09:29:43 -0700 (PDT)\r
+Received: from smtp.chost.de (setoy.chost.de [217.160.209.225])\r
+       (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id A79AA431FAE\r
+       for <notmuch@notmuchmail.org>; Sun, 24 Jun 2012 09:29:42 -0700 (PDT)\r
+Received: (qmail 14457 invoked by uid 5015); 24 Jun 2012 16:29:40 -0000\r
+Received: (nullmailer pid 3401 invoked by uid 123);\r
+       Sun, 24 Jun 2012 16:29:39 -0000\r
+Received: from twin.sascha.silbe.org (twin.sascha.silbe.org [192.168.1.2])\r
+       by flatty.sascha.silbe.org ([192.168.1.252])\r
+       with SMTP via TCP; 24 Jun 2012 16:29:39 -0000\r
+Received: (nullmailer pid 25943 invoked by uid 8193);\r
+       Sun, 24 Jun 2012 16:29:39 -0000\r
+From: Sascha Silbe <sascha-pgp@silbe.org>\r
+To: notmuch <notmuch@notmuchmail.org>\r
+Subject: [PATCH 0/3] Speed up notmuch new for unchanged directories\r
+Date: Sun, 24 Jun 2012 18:29:23 +0200\r
+Message-Id: <1340555366-25891-1-git-send-email-sascha-pgp@silbe.org>\r
+X-Mailer: git-send-email 1.7.10\r
+Mail-Followup-To: notmuch <notmuch@notmuchmail.org>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+Reply-To: Sascha Silbe <sascha-ml-reply-to-2012-3@silbe.org>\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 24 Jun 2012 16:29:43 -0000\r
+\r
+All the time I thought what makes "notmuch new" so abysmally slow is the\r
+stat() for each maildir. But as it continued to be slow even after I\r
+moved most mails out of 'new' (into 'new-20120624'), I strace'd notmuch\r
+and noticed it listed even unchanged directories, thereby listing and\r
+iterating over each and every single of the 900k mails in my mail store.\r
+\r
+There's still quite some room for further improvements as it continues\r
+to take several minutes to scan < 100 new mails in changed directories\r
+containing < 1000 mails in total. Even the rsync run that fetches the\r
+new mails is faster.\r
+\r
+Sascha Silbe (3):\r
+  lib: fix NULL checks for filenames iterators\r
+  lib: add support for rewinding a filenames iterator\r
+  new: don't read unchanged directories from disk\r
+\r
+ lib/filenames.c |   15 ++++++-\r
+ lib/notmuch.h   |    8 ++++\r
+ notmuch-new.c   |  130 +++++++++++++++++++++++++++++++++++++++++--------------\r
+ 3 files changed, 119 insertions(+), 34 deletions(-)\r
+\r
+-- \r
+1.7.10\r
+\r