From: Eric J <eric@deptj.eu>
Subject: Re: Lost updates to Notmuch database
To: notmuch@notmuchmail.org
In-Reply-To: <m27fi2nl9x.fsf@guru.guru-group.fi>
References: <337393ce5e2484d64f837a44b2c4f2ea32f91295-NM@bruno.deptj.eu>
 <871t8avnhq.fsf@zancas.localnet> <m27fi2nl9x.fsf@guru.guru-group.fi>
Message-ID: <d64b4eafab192f6a44fb615730ff3350bae8876b-NM@bruno.deptj.eu>
Date: Thu, 18 Feb 2016 22:26:56 +0100 (CET)
Precedence: list

On Thu, 18 Feb 2016 16:30:34 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> On Thu, Feb 18 2016, David Bremner <david@tethera.net> wrote:
> 
> > Eric J <eric@deptj.eu> writes:
> >
> >> However, if I do it twice, in different processes, at the same time, one
> >> file is added and tagged properly, the other is not (totally unfindable
> >> by notmuch search). Neither process reports any error, and they both log
> >> their actions normally. Actually a third simultaneous process also fails
> >> to leave any result in the database.
> >
> > It should be impossible for more than one process to open a Xapian
> > database for writing at the same time. So if the processes are really
> > running in parallel, you should be getting error codes from the later
> > calls to notmuch_database_open{_verbose}. You claim that's not
> > happening, which is puzzling. Maybe you can try to duplicate your
> > problem with a tiny C program.
> 
> In addition to that (or even before), you could
> 
> 1) be able to reproduce the problem

As I said in my response to David, I can do just the open, and see that
there is no lock, and do the same open in another session without error.

> 2) try to reproduce it prefixing the command with ltrace -tt

Well, as you say below, no Xapian references. Even worse, ltracing the
Tcl version sees no Notmuch references either.

> 3) examine carefully the ltrace logs to figure out where the proble lies

So all I can see is that the notmuch_database_open from notmuch new
looks just like I think mine would look - if I could see it.

(Adding -S to ltrace was not particularly informative either.)

> Tomi
> 
> Hmm, Interestingly when I run
> 
> LD_LIBRARY_PATH=~/vc/ext/notmuch/lib ltrace -f -tt ~/vc/ext/notmuch/notmuch-shared new
> 
> I did not see any Xapian references, but when I did
> 
> ltrace -f -tt ~/vc/ext/notmuch/notmuch new
> 
> I did. Interestingly when using libnotmuch.so.4 the xapian interface
> is hidden (is it baked inside ~/vc/ext/notmuch/lib/libnotmuch.so.4.3.0 :O)

My latest piece of evidence suggests that this may not be a Notmuch
problem, because opening a Xapian database via the Xapian-Tcl bindings
results in:

COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
cat     2803 eric    5w   REG    8,9        0 526601 /home/eric/omega/data/newsgroups/flintlock

No lock here either (and this is not even a Notmuch database). 

Investigations will continue.

Eric
-- 
ms fnd in a lbry