notmuch new crashing due to out of memory - how to deal with it

diff --git a/39/d6de71d7eecde7d39c48bfd318def98af33e62 b/39/d6de71d7eecde7d39c48bfd318def98af33e62
new file mode 100644 (file)
index 0000000..ac100e4
--- /dev/null
+++ b/39/d6de71d7eecde7d39c48bfd318def98af33e62
@@ -0,0 +1,87 @@
+Return-Path: <dmishd@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 1EF16431FBD\r
+       for <notmuch@notmuchmail.org>; Mon,  3 Feb 2014 10:23:01 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id YS6izLEtHfI9 for <notmuch@notmuchmail.org>;\r
+       Mon,  3 Feb 2014 10:22:57 -0800 (PST)\r
+Received: from mail-qa0-f44.google.com (mail-qa0-f44.google.com\r
+       [209.85.216.44]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 8630B431FBC\r
+       for <notmuch@notmuchmail.org>; Mon,  3 Feb 2014 10:22:57 -0800 (PST)\r
+Received: by mail-qa0-f44.google.com with SMTP id w5so10740302qac.31\r
+       for <notmuch@notmuchmail.org>; Mon, 03 Feb 2014 10:22:54 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+       h=mime-version:from:date:message-id:subject:to:content-type;\r
+       bh=j+c50hMoqJ7TI7SwpNzglz5Y8CQNoJdIA4MXYCJ6oKY=;\r
+       b=T01sV3g/ZrFfGQ9MpiFPkwg4oLXYSA/Z6q3IsL7n58V4zfqxdjSyrpHYzMW1UpuqPZ\r
+       LUiYMDOPbl2y7UXMxRwT+ag/HOrwtZnAdHYlwOdxnyb9lGFC0QW8o10uF4aOzTE1FZs0\r
+       jn2LhRmtxGqctUpe43hmCLrnWmoNcdgnImDHYfy0bp2aYnutqV5UCuzXRhTj5X6n/OsP\r
+       2Z2obrWFZHiGdbMRXsKpuYzxVroCtVny1ptXvUif/Q4//PR1u0Itah2qIil/cIr2SIEM\r
+       z8OOgZOlCKSJe9NLaOCcxdgnIpR16D/uPMz1uhHuiSEc8+1GrBn0c+NgLBYXejIUxZi0\r
+       MaBg==\r
+X-Received: by 10.140.43.230 with SMTP id e93mr55902686qga.35.1391451770881;\r
+       Mon, 03 Feb 2014 10:22:50 -0800 (PST)\r
+MIME-Version: 1.0\r
+Received: by 10.140.49.83 with HTTP; Mon, 3 Feb 2014 10:22:10 -0800 (PST)\r
+From: Hamish D <dmishd@gmail.com>\r
+Date: Mon, 3 Feb 2014 18:22:10 +0000\r
+Message-ID:\r
+ <CAOxvSbdUUhRGWRjhtbmiGL==___Vptby-tz_eAeuqZtESdABWA@mail.gmail.com>\r
+Subject: notmuch new crashing due to out of memory - how to deal with it\r
+To: notmuch@notmuchmail.org\r
+Content-Type: text/plain; charset=UTF-8\r
+X-Mailman-Approved-At: Tue, 04 Feb 2014 00:28:10 -0800\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 03 Feb 2014 18:23:01 -0000\r
+\r
+Hello\r
+\r
+I recently deleted almost 30000 old messages from my maildirs, and\r
+since I did that, notmuch new has not managed to complete.  I have it\r
+running on a server with 1 GB of RAM, and the output is telling me\r
+\r
+    Cleaned up 25515 of 29803 messages (1m 10s remaining).\r
+\r
+at the point it is killed.  I have followed it up to this point with\r
+top and free and I can see memory usage growing to something like\r
+700MB (RES) at which point the process is killed.\r
+\r
+I am using notmuch 0.17 (from the debian jessie package) on debian\r
+wheezy 64 bit.\r
+\r
+Any pointers as to what to do to get past this point would be greatly\r
+appreciated.  If I have just deleted that many messages should I just\r
+expect notmuch to use that much memory?  I guess I could enable swap\r
+and hope that allows enough memory to get past this operation.\r
+\r
+I did also try running it with NOTMUCH_TALLOC_REPORT, but that didn't\r
+produce a report (though I was unsure if I needed to rebuild notmuch\r
+before that would work).\r
+\r
+Happy to build notmuch from source if this might be an interesting bug\r
+worth tracking down.\r
+\r
+Hamish\r