From: Guyzmo <guyzmo+notmuch@m0g.net>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Xapian lockup when writing to the notmuch database
Message-Id: <67EEA3E1-918F-47AE-8AD7-EF0A5923D800@m0g.net>
Date: Wed, 6 Jan 2016 15:49:28 +0000
To: notmuch@notmuchmail.org
Mime-Version: 1.0
Precedence: list

Hello,

I have migrated my mail configuration from an old machine to a new one, =
which is a xen=20
server with several VM instances. I have one VM dedicated to do mail =
stuff, and I want
to manage my Maildir using notmuch in another one. So I set up an NFSv4 =
share (with
sync option set) from the mail server, to have my Maildir accessible =
from the other VM.=20
Of course, I made sure my user is able to have full privileges over the =
mounted share,
and there=E2=80=99s only one instance of notmuch running over that =
xapian db at all times.

All read only operations work just perfectly fine, I can even run =
notmuch-kz and have
it list my search mailboxes.

But when I do `notmuch new` it hangs. So I did ran it through gdb and =
here is the result:

% gdb notmuch
(gdb) run new
Starting program: /usr/local/bin/notmuch new
[Thread debugging using libthread_db enabled]
Using host libthread_db library =
"/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1".
^C
Program received signal SIGINT, Interrupt.
0xb7fe1428 in __kernel_vsyscall ()
(gdb) bt
#0  0xb7fe1428 in __kernel_vsyscall ()
#1  0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2  0xb7adf538 in read (__nbytes=3D1, __buf=3D0xbfffedfc, =
__fd=3D<optimized out>) at /usr/include/i386-linux-gnu/bits/unistd.h:45
#3  FlintLock::lock (this=3D0x8079180, exclusive=3Dtrue, =
explanation=3D...) at ../backends/flint_lock.cc:222
#4  0xb7b30a86 in ChertDatabase::get_database_write_lock =
(this=3Dthis@entry=3D0x80789a0, creating=3Dcreating@entry=3Dfalse)
  at ../backends/chert/chert_database.cc:505
#5  0xb7b34fd2 in ChertDatabase::ChertDatabase =
(this=3Dthis@entry=3D0x80789a0, chert_dir=3D..., action=3Daction@entry=3D1=
, block_size=3Dblock_size@entry=3D8192)
  at ../backends/chert/chert_database.cc:154
#6  0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase =
(this=3D0x80789a0, dir=3D..., action=3D1, block_size=3D8192)
  at ../backends/chert/chert_database.cc:1036
#7  0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase =
(this=3D0x80780c8, path=3D..., action=3D1) at =
../backends/dbfactory.cc:490
#8  0xb7fb8c68 in notmuch_database_open_verbose (path=3D0x80780c8 =
"\340\327=C5=B7", mode=3DNOTMUCH_DATABASE_MODE_READ_WRITE, =
database=3D0xbffff230,=20
  status_string=3D0xbffff248) at lib/database.cc:933
#9  0x08054e93 in notmuch_new_command (config=3D0x80745b8, argc=3D1, =
argv=3D0xbffff678) at notmuch-new.c:1008
#10 0x0804df87 in main (argc=3D2, argv=3D0xbffff674) at notmuch.c:421
(gdb)=20

then I tried doing another write operation, doing a `notmuch tag` =
command

% gdb notmuch
(gdb) run tag +tag id:<a_unique_mail_id>
(gdb) bt
#0  0xb7fe1428 in __kernel_vsyscall ()
#1  0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6
#2  0xb7adf538 in read (__nbytes=3D1, __buf=3D0xbfffefac, =
__fd=3D<optimized out>) at /usr/include/i386-linux-gnu/bits/unistd.h:45
#3  FlintLock::lock (this=3D0x8079360, exclusive=3Dtrue, =
explanation=3D...) at ../backends/flint_lock.cc:222
#4  0xb7b30a86 in ChertDatabase::get_database_write_lock =
(this=3Dthis@entry=3D0x8078b80, creating=3Dcreating@entry=3Dfalse)
  at ../backends/chert/chert_database.cc:505
#5  0xb7b34fd2 in ChertDatabase::ChertDatabase =
(this=3Dthis@entry=3D0x8078b80, chert_dir=3D..., action=3Daction@entry=3D1=
, block_size=3Dblock_size@entry=3D8192)
  at ../backends/chert/chert_database.cc:154
#6  0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase =
(this=3D0x8078b80, dir=3D..., action=3D1, block_size=3D8192)
  at ../backends/chert/chert_database.cc:1036
#7  0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase =
(this=3D0x80780c8, path=3D..., action=3D1) at =
../backends/dbfactory.cc:490
#8  0xb7fb8c68 in notmuch_database_open_verbose (path=3D0x80780c8 =
"\340\327=C5=B7", mode=3DNOTMUCH_DATABASE_MODE_READ_WRITE, =
database=3D0xbffff3e4,=20
  status_string=3D0xbffff39c) at lib/database.cc:933
#9  0xb7fb975c in notmuch_database_open (path=3D0x8078068 =
"/home/guyzmo/Maildir", =
mode=3Dmode@entry=3DNOTMUCH_DATABASE_MODE_READ_WRITE,=20
  database=3Ddatabase@entry=3D0xbffff3e4) at lib/database.cc:848
#10 0x0805cbe4 in notmuch_tag_command (config=3D0x80745b8, argc=3D3, =
argv=3D0xbffff638) at notmuch-tag.c:262
#11 0x0804df87 in main (argc=3D4, argv=3D0xbffff634) at notmuch.c:421
(gdb)=20

I=E2=80=99m running notmuch on a Debian Wheezy, on which I compiled =
latest notmuch
from git, using the repository=E2=80=99s libxapian which is v1.2.12-2.

Now, I could use a newer version of libxapian, either V1.2.16 from the =
debian=20
wheezy backports, or get the latest compiling it from sources, as it =
looks=20
like the flintlock.cc compilation unit has been rewritten since v1.2.12
=E2=80=94  though it looks like the part where it hangs is looking quite =
alike, but=20
it=E2=80=99s hard to tell from the changes whether that would have an =
inpact.

So my question is whether this is a known issue, and is actually related =
to
the fact that I=E2=80=99m running it over NFS. I looked for NFS related =
comments on
xapian website and so far I found nothing looking alike my issue.

I=E2=80=99ll try testing with other versions of xapian and see it solves =
it. If the
issue is still on with HEAD of xapian=E2=80=99s sources, then I=E2=80=99ll=
 do a bug report.
Until then, I=E2=80=99m just checking on this list for ideas and =
comments, in hope
for a working solution.

Cheers,

--=20
Guyzmo=