Return-Path: <jani@nikula.org>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 1B506431FBC
	for <notmuch@notmuchmail.org>; Tue,  4 Feb 2014 00:25:37 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: -0.7
X-Spam-Level: 
X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5
	tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id dvf8Y2BBuakD for <notmuch@notmuchmail.org>;
	Tue,  4 Feb 2014 00:25:29 -0800 (PST)
Received: from mail-qa0-f47.google.com (mail-qa0-f47.google.com
	[209.85.216.47]) (using TLSv1 with cipher RC4-SHA (128/128 bits))
	(No client certificate requested)
	by olra.theworths.org (Postfix) with ESMTPS id C37D8431FAF
	for <notmuch@notmuchmail.org>; Tue,  4 Feb 2014 00:25:29 -0800 (PST)
Received: by mail-qa0-f47.google.com with SMTP id j5so11545887qaq.6
	for <notmuch@notmuchmail.org>; Tue, 04 Feb 2014 00:25:28 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20130820;
	h=x-gm-message-state:from:to:subject:in-reply-to:references
	:user-agent:date:message-id:mime-version:content-type;
	bh=a01g9Gmh+8b36jvtepoObCJuCt0oG8P/EvAbxZt5vGM=;
	b=Q4eAFKSbIKoqCMv+8bQz5EW12rDYskTdY+4aT/iqryXqqPLpMJZfozsBVXwbWYNTBy
	XxuuRz4I8iGa1x36CSl02rttCV6ZYg3fYQH8QId3vYRkAgkOxu1KV5anhobzMywWxxGg
	XpnEurnNPko3d8IDjY2R9bGAu0FZu4FHZb4uCoFwRFKIKqi8jGl+QChEKYaLQ2rr9/PU
	OHml6xH3aCmyXuxPsYw/CSzDWXVSzeiTG3yHNCdg9fdFkLEr04Sf1ovAsT4f6gu//pDw
	0kd4ClUxu+WhvhD5QBb9xHg/cO2SBYSM7YNwS8uFGzP+u6/XMDQzlPLFdmTLSzJ0b+xb
	PuPg==
X-Gm-Message-State:
 ALoCoQn7Xs1IuzaYF4174TOpa9o+oE6D3yuFmzKjnTW4wC4uDA4a2DxXpof3IqRO3ZHtlRn9rGtO
X-Received: by 10.140.107.53 with SMTP id g50mr60371021qgf.63.1391502328087;
	Tue, 04 Feb 2014 00:25:28 -0800 (PST)
Received: from localhost ([2001:4b98:dc0:43:216:3eff:fe1b:25f3])
	by mx.google.com with ESMTPSA id o68sm31252698qge.8.2014.02.04.00.25.25
	for <multiple recipients>
	(version=TLSv1.1 cipher=RC4-SHA bits=128/128);
	Tue, 04 Feb 2014 00:25:27 -0800 (PST)
From: Jani Nikula <jani@nikula.org>
To: Hamish Downer <hamish@foobacca.co.uk>, notmuch@notmuchmail.org
Subject: Re: notmuch killed due to out of memory - how to move forward
In-Reply-To:
 <CAOxvSbfWu3M=M1PyFrZcUhszjW0rpb0wYOiia+3sSSd9AGqD4w@mail.gmail.com>
References:
 <CAOxvSbfWu3M=M1PyFrZcUhszjW0rpb0wYOiia+3sSSd9AGqD4w@mail.gmail.com>
User-Agent: Notmuch/0.17+64~g6eaa780 (http://notmuchmail.org) Emacs/23.2.1
	(x86_64-pc-linux-gnu)
Date: Tue, 04 Feb 2014 09:25:17 +0100
Message-ID: <87r47jcmj6.fsf@nikula.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Tue, 04 Feb 2014 08:25:37 -0000

On Mon, 03 Feb 2014, Hamish Downer <hamish@foobacca.co.uk> wrote:
> Hello
>
> I recently deleted almost 30000 old messages from my maildirs, and
> since I did that, notmuch new has not managed to complete.  I have it
> running on a server with 1 GB of RAM, and the output is telling me
>
>     Cleaned up 25515 of 29803 messages (1m 10s remaining).
>
> at the point it is killed.  I have followed it up to this point with
> top and free and I can see memory usage growing to something like
> 700MB (RES) at which point the process is killed.
>
> I am using notmuch 0.17 (from the debian jessie package 0.17-3) on
> debian wheezy 64 bit.
>
> Any pointers as to what to do to get past this point would be greatly
> appreciated.  If I have just deleted that many messages should I just
> expect notmuch to use that much memory?  (Unfortunately the company
> supplying the server does not allow swap to be enabled so I can't fudge
> it that way)
>
> I did also try running it with NOTMUCH_TALLOC_REPORT, but that didn't
> produce a report (though I was unsure if I needed to rebuild notmuch
> before that would work).
>
> Happy to build notmuch from source if this might be an interesting bug
> worth tracking down.

Please try SIGINT, or ^C, on notmuch new before it gets killed. That
should be handled gracefully, making progress, and letting you chop up
and eventually finish the operation. Please let us know if this helps.

Needless to say, we shouldn't use that much memory just to delete files
from the index.


BR,
Jani.