From: John Fremlin <john@fremlin.org>
To: notmuch@notmuchmail.org
Subject: Re: Notmuch success: Xapian database corrupt
References: <87r5mchmji.fsf-genuine-vii@john.fremlin.org>
	<874oj4wdbn.fsf@yoom.home.cworth.org>
Date: Thu, 22 Apr 2010 07:19:58 +0000
In-Reply-To: <874oj4wdbn.fsf@yoom.home.cworth.org> (Carl Worth's message of
	"Wed, 21 Apr 2010 17:17:16 -0700")
Message-ID: <87sk6oc5sx.fsf-genuine-vii@john.fremlin.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

After the encouraging message from Sebastian. I deleted the
.notmuch/xapian dir and started again.

It went off a good rate (300+ files/sec) and here was the final score

Processed 494764 total files in 2h 54m 41s (47 files/sec.).                 
Added 226817 new messages to the database.

This is much faster than before. As I haven't changed the storage or the
filesystem (ext4,data=ordered over encrypted aes-xts-plain), I just
don't know what made the difference. My kernel is now 2.6.32-21-generic
#32-Ubuntu and I had an older one the first try a month or so ago.

Carl Worth <cworth@cworth.org> writes:
[...]
> Welcome to notmuch, and I'm so sorry to hear that your initial attempt
> to use it was so frustrating.

Thanks for the welcome! I was initially impressed by it but rather
worried about relying on it after the database corruption.

> I'm not aware of any bugs in notmuch that can result in a corrupt Xapian
> database. In fact, this can't be a bug in notmuch alone (since Xapian is
> detecting the corruption). There must at least be a bug in Xapian or
> else some lower-level failure is occurring (disk full?) that Xapian
> can't deal with.

Disk full is quite likely. I'll try to avoid that in future.

[...]
> So there's some performance problem that you're having in addition to
> the database corruption. Hopefully we can figure that out. What kernel
> and filesystem are you using? Are you using an encrypted partition?

Happy to say (though frustrating for you), this time it's much
faster. Maybe because I had more disk free this time round so the Xapian
database became less fragmented? (Speculation, no evidence.)