notmuch killed by out of memory - how to move forward

diff --git a/c0/87790e53f63c7b7e57cd8f80d2baea07836a1f b/c0/87790e53f63c7b7e57cd8f80d2baea07836a1f
new file mode 100644 (file)
index 0000000..8caf65d
--- /dev/null
+++ b/c0/87790e53f63c7b7e57cd8f80d2baea07836a1f
@@ -0,0 +1,88 @@
+Return-Path: <dmishd@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id C93AA431FAF\r
+       for <notmuch@notmuchmail.org>; Mon,  3 Feb 2014 12:14:24 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id PRvah66rEiN2 for <notmuch@notmuchmail.org>;\r
+       Mon,  3 Feb 2014 12:14:20 -0800 (PST)\r
+Received: from mail-qc0-f170.google.com (mail-qc0-f170.google.com\r
+       [209.85.216.170]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 86D16431E64\r
+       for <notmuch@notmuchmail.org>; Mon,  3 Feb 2014 12:14:14 -0800 (PST)\r
+Received: by mail-qc0-f170.google.com with SMTP id e9so12319606qcy.29\r
+       for <notmuch@notmuchmail.org>; Mon, 03 Feb 2014 12:14:13 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+       h=mime-version:from:date:message-id:subject:to:content-type;\r
+       bh=DBJeWxgwVGzPGI813wDGXZhMJ2r6I48Va+o9AKwFc5w=;\r
+       b=ueMnxjrvpzAKlzKUFb2l5d4zkO6wIKU27pxb1I4KJ8ueMO3OS3oH4gWJaOs6xyKML9\r
+       gAESqU6f+/luNe2MdpNYUeigXSwSp2H/5qOs10hyZ/lCkp67Fd6pkJsVS7iZ60r5Sndj\r
+       cMFp+Beyt+yc/sI1dbPJKNVbmtxK0O3mismfjsp56tccagcy45O2HC7zrqH8aGyPWiND\r
+       f9blUxwz382NyJZPvFplumEFIriVfLO3mSEpFYcheHayccdyQ1WVn7tZroeLw3uFZt1N\r
+       Hck5FylDhShuqotGaZV416jk3sqz6mq35aw9Baj0eNBxhwUKTa62X9AUwqQSbdXIblhP\r
+       S5LA==\r
+X-Received: by 10.224.147.75 with SMTP id k11mr59116803qav.22.1391458452825;\r
+       Mon, 03 Feb 2014 12:14:12 -0800 (PST)\r
+MIME-Version: 1.0\r
+Received: by 10.140.49.83 with HTTP; Mon, 3 Feb 2014 12:13:32 -0800 (PST)\r
+From: Hamish D <dmishd@gmail.com>\r
+Date: Mon, 3 Feb 2014 20:13:32 +0000\r
+Message-ID:\r
+ <CAOxvSbcOb_0_KtrkdXT9ASXU8-BsPF2jeKAdOpUc92XBHBwxAQ@mail.gmail.com>\r
+Subject: notmuch killed by out of memory - how to move forward\r
+To: notmuch@notmuchmail.org\r
+Content-Type: text/plain; charset=UTF-8\r
+X-Mailman-Approved-At: Tue, 04 Feb 2014 00:28:10 -0800\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 03 Feb 2014 20:14:25 -0000\r
+\r
+Hello\r
+\r
+I recently deleted almost 30000 old messages from my maildirs, and\r
+since I did that, notmuch new has not managed to complete.  I have it\r
+running on a server with 1 GB of RAM, and the output is telling me\r
+\r
+    Cleaned up 25515 of 29803 messages (1m 10s remaining).\r
+\r
+at the point it is killed.  I have followed it up to this point with\r
+top and free and I can see memory usage growing to something like\r
+700MB (RES) at which point the process is killed.\r
+\r
+I am using notmuch 0.17 (from the debian jessie package) on debian\r
+wheezy 64 bit.\r
+\r
+Any pointers as to what to do to get past this point would be greatly\r
+appreciated.  If I have just deleted that many messages should I just\r
+expect notmuch to use that much memory?  I guess I could enable swap\r
+and hope that allows enough memory to get past this operation. (Just checked\r
+and swap is not an option with this host).\r
+\r
+I did also try running it with NOTMUCH_TALLOC_REPORT, but that didn't\r
+produce a report (though I was unsure if I needed to rebuild notmuch\r
+before that would work).\r
+\r
+Happy to build notmuch from source if this might be an interesting bug\r
+worth tracking down.\r
+\r
+Hamish\r