From dbb7434e79ec3851380cf9f30eb0141e2b337921 Mon Sep 17 00:00:00 2001 From: Jani Nikula Date: Tue, 4 Feb 2014 09:25:17 +0100 Subject: [PATCH] Re: notmuch killed due to out of memory - how to move forward --- 06/aba6f513d21523fc0fbd2151e15acbe1c97ea9 | 110 ++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 06/aba6f513d21523fc0fbd2151e15acbe1c97ea9 diff --git a/06/aba6f513d21523fc0fbd2151e15acbe1c97ea9 b/06/aba6f513d21523fc0fbd2151e15acbe1c97ea9 new file mode 100644 index 000000000..2395a7cbe --- /dev/null +++ b/06/aba6f513d21523fc0fbd2151e15acbe1c97ea9 @@ -0,0 +1,110 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 1B506431FBC + for ; Tue, 4 Feb 2014 00:25:37 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id dvf8Y2BBuakD for ; + Tue, 4 Feb 2014 00:25:29 -0800 (PST) +Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com + [209.85.216.47]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id C37D8431FAF + for ; Tue, 4 Feb 2014 00:25:29 -0800 (PST) +Received: by mail-qa0-f47.google.com with SMTP id j5so11545887qaq.6 + for ; Tue, 04 Feb 2014 00:25:28 -0800 (PST) +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:from:to:subject:in-reply-to:references + :user-agent:date:message-id:mime-version:content-type; + bh=a01g9Gmh+8b36jvtepoObCJuCt0oG8P/EvAbxZt5vGM=; + b=Q4eAFKSbIKoqCMv+8bQz5EW12rDYskTdY+4aT/iqryXqqPLpMJZfozsBVXwbWYNTBy + XxuuRz4I8iGa1x36CSl02rttCV6ZYg3fYQH8QId3vYRkAgkOxu1KV5anhobzMywWxxGg + XpnEurnNPko3d8IDjY2R9bGAu0FZu4FHZb4uCoFwRFKIKqi8jGl+QChEKYaLQ2rr9/PU + OHml6xH3aCmyXuxPsYw/CSzDWXVSzeiTG3yHNCdg9fdFkLEr04Sf1ovAsT4f6gu//pDw + 0kd4ClUxu+WhvhD5QBb9xHg/cO2SBYSM7YNwS8uFGzP+u6/XMDQzlPLFdmTLSzJ0b+xb + PuPg== +X-Gm-Message-State: + ALoCoQn7Xs1IuzaYF4174TOpa9o+oE6D3yuFmzKjnTW4wC4uDA4a2DxXpof3IqRO3ZHtlRn9rGtO +X-Received: by 10.140.107.53 with SMTP id g50mr60371021qgf.63.1391502328087; + Tue, 04 Feb 2014 00:25:28 -0800 (PST) +Received: from localhost ([2001:4b98:dc0:43:216:3eff:fe1b:25f3]) + by mx.google.com with ESMTPSA id o68sm31252698qge.8.2014.02.04.00.25.25 + for + (version=TLSv1.1 cipher=RC4-SHA bits=128/128); + Tue, 04 Feb 2014 00:25:27 -0800 (PST) +From: Jani Nikula +To: Hamish Downer , notmuch@notmuchmail.org +Subject: Re: notmuch killed due to out of memory - how to move forward +In-Reply-To: + +References: + +User-Agent: Notmuch/0.17+64~g6eaa780 (http://notmuchmail.org) Emacs/23.2.1 + (x86_64-pc-linux-gnu) +Date: Tue, 04 Feb 2014 09:25:17 +0100 +Message-ID: <87r47jcmj6.fsf@nikula.org> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 04 Feb 2014 08:25:37 -0000 + +On Mon, 03 Feb 2014, Hamish Downer wrote: +> Hello +> +> I recently deleted almost 30000 old messages from my maildirs, and +> since I did that, notmuch new has not managed to complete. I have it +> running on a server with 1 GB of RAM, and the output is telling me +> +> Cleaned up 25515 of 29803 messages (1m 10s remaining). +> +> at the point it is killed. I have followed it up to this point with +> top and free and I can see memory usage growing to something like +> 700MB (RES) at which point the process is killed. +> +> I am using notmuch 0.17 (from the debian jessie package 0.17-3) on +> debian wheezy 64 bit. +> +> Any pointers as to what to do to get past this point would be greatly +> appreciated. If I have just deleted that many messages should I just +> expect notmuch to use that much memory? (Unfortunately the company +> supplying the server does not allow swap to be enabled so I can't fudge +> it that way) +> +> I did also try running it with NOTMUCH_TALLOC_REPORT, but that didn't +> produce a report (though I was unsure if I needed to rebuild notmuch +> before that would work). +> +> Happy to build notmuch from source if this might be an interesting bug +> worth tracking down. + +Please try SIGINT, or ^C, on notmuch new before it gets killed. That +should be handled gracefully, making progress, and letting you chop up +and eventually finish the operation. Please let us know if this helps. + +Needless to say, we shouldn't use that much memory just to delete files +from the index. + + +BR, +Jani. + -- 2.26.2