From: Guyzmo Date: Wed, 6 Jan 2016 15:35:43 +0000 (+0000) Subject: Xapian lockup when writing to the notmuch database X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=9301633369ae3239c979418c46f04e4f80f91936;p=notmuch-archives.git Xapian lockup when writing to the notmuch database --- diff --git a/33/f1c2f1c340fc85c18bc77de20d350f8c8001ef b/33/f1c2f1c340fc85c18bc77de20d350f8c8001ef new file mode 100644 index 000000000..d09618d5d --- /dev/null +++ b/33/f1c2f1c340fc85c18bc77de20d350f8c8001ef @@ -0,0 +1,191 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id D30846DE10F8 + for ; Wed, 6 Jan 2016 07:41:30 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.652 +X-Spam-Level: +X-Spam-Status: No, score=-0.652 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + RP_MATCHES_RCVD=-0.55, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] + autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id irhWR65Y7RhG for ; + Wed, 6 Jan 2016 07:41:28 -0800 (PST) +X-Greylist: delayed 340 seconds by postgrey-1.35 at arlo; + Wed, 06 Jan 2016 07:41:27 PST +Received: from elendil.m0g.net (elendil.m0g.net [212.83.155.195]) + by arlo.cworth.org (Postfix) with ESMTPS id 97BBC6DE103A + for ; Wed, 6 Jan 2016 07:41:27 -0800 (PST) +Received: from authenticated-user (elendil.m0g.net [212.83.155.195]) + (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + by elendil.m0g.net (Postfix) with ESMTPSA id 0C6D82A096B + for ; Wed, 6 Jan 2016 16:35:43 +0100 (CET) +DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=m0g.net; s=mail; + t=1452094544; bh=hQjmQxPSz+u23SSXrZFWUFClEE7MoEYqvPJEs1DlCFE=; + h=From:Subject:Date:To:From; + b=iv0Wk9DttI+aAoPMukck+xeTNYONKy77azibelYW5VBt8fAD0oRMfDlWwCXpsbgfU + pc8SyFJ50DbquNmreiXgK3IwuRW+Wc79V7uPf/B3rP/qoTNAwV3nIfE533NlL+4Bo5 + 7ldetkRr9W88SHwPnVO+GzJPHtcxhO9TIqKvAx8/J6M/5FTgqAzS8nYlW4cVKsLjdH + LU/6L0PMIQ0KK7hsUimrYLHpOprZ4s2zPwpucl9ZTiOnhUetfyedoY+WwD3JNhCMff + rj10pVfr7MNGIRJS1xcvFOQtJmpZOh6IokG3H0aLKghfRnk41x3hpqtbF5eUKY6d2V + J5O8/Sly3MVRg== +From: Guyzmo +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable +Subject: Xapian lockup when writing to the notmuch database +Message-Id: +Date: Wed, 6 Jan 2016 15:35:43 +0000 +To: notmuch@notmuchmail.org +Mime-Version: 1.0 +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 06 Jan 2016 15:41:31 -0000 + +Hello, + +I have migrated my mail configuration from an old machine to a new one, = +which is a xen=20 +server with several VM instances. I have one VM dedicated to do mail = +stuff, and I want +to manage my Maildir using notmuch in another one. So I set up an NFSv4 = +share (with +sync option set) from the mail server, to have my Maildir accessible = +from the other VM.=20 +Of course, I made sure my user is able to have full privileges over the = +mounted share, +and there=E2=80=99s only one instance of notmuch running over that = +xapian db at all times. + +All read only operations work just perfectly fine, I can even run = +notmuch-kz and have +it list my search mailboxes. + +But when I do `notmuch new` it hangs. So I did ran it through gdb and = +here is the result: + +% gdb notmuch +(gdb) run new +Starting program: /usr/local/bin/notmuch new +[Thread debugging using libthread_db enabled] +Using host libthread_db library = +"/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1". +^C +Program received signal SIGINT, Interrupt. +0xb7fe1428 in __kernel_vsyscall () +(gdb) bt +#0 0xb7fe1428 in __kernel_vsyscall () +#1 0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 +#2 0xb7adf538 in read (__nbytes=3D1, __buf=3D0xbfffedfc, = +__fd=3D) at /usr/include/i386-linux-gnu/bits/unistd.h:45 +#3 FlintLock::lock (this=3D0x8079180, exclusive=3Dtrue, = +explanation=3D...) at ../backends/flint_lock.cc:222 +#4 0xb7b30a86 in ChertDatabase::get_database_write_lock = +(this=3Dthis@entry=3D0x80789a0, creating=3Dcreating@entry=3Dfalse) + at ../backends/chert/chert_database.cc:505 +#5 0xb7b34fd2 in ChertDatabase::ChertDatabase = +(this=3Dthis@entry=3D0x80789a0, chert_dir=3D..., action=3Daction@entry=3D1= +, block_size=3Dblock_size@entry=3D8192) + at ../backends/chert/chert_database.cc:154 +#6 0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase = +(this=3D0x80789a0, dir=3D..., action=3D1, block_size=3D8192) + at ../backends/chert/chert_database.cc:1036 +#7 0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase = +(this=3D0x80780c8, path=3D..., action=3D1) at = +../backends/dbfactory.cc:490 +#8 0xb7fb8c68 in notmuch_database_open_verbose (path=3D0x80780c8 = +"\340\327=C5=B7", mode=3DNOTMUCH_DATABASE_MODE_READ_WRITE, = +database=3D0xbffff230,=20 + status_string=3D0xbffff248) at lib/database.cc:933 +#9 0x08054e93 in notmuch_new_command (config=3D0x80745b8, argc=3D1, = +argv=3D0xbffff678) at notmuch-new.c:1008 +#10 0x0804df87 in main (argc=3D2, argv=3D0xbffff674) at notmuch.c:421 +(gdb)=20 + +then I tried doing another write operation, doing a `notmuch tag` = +command + +% gdb notmuch +(gdb) run tag +tag id: +(gdb) bt +#0 0xb7fe1428 in __kernel_vsyscall () +#1 0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 +#2 0xb7adf538 in read (__nbytes=3D1, __buf=3D0xbfffefac, = +__fd=3D) at /usr/include/i386-linux-gnu/bits/unistd.h:45 +#3 FlintLock::lock (this=3D0x8079360, exclusive=3Dtrue, = +explanation=3D...) at ../backends/flint_lock.cc:222 +#4 0xb7b30a86 in ChertDatabase::get_database_write_lock = +(this=3Dthis@entry=3D0x8078b80, creating=3Dcreating@entry=3Dfalse) + at ../backends/chert/chert_database.cc:505 +#5 0xb7b34fd2 in ChertDatabase::ChertDatabase = +(this=3Dthis@entry=3D0x8078b80, chert_dir=3D..., action=3Daction@entry=3D1= +, block_size=3Dblock_size@entry=3D8192) + at ../backends/chert/chert_database.cc:154 +#6 0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase = +(this=3D0x8078b80, dir=3D..., action=3D1, block_size=3D8192) + at ../backends/chert/chert_database.cc:1036 +#7 0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase = +(this=3D0x80780c8, path=3D..., action=3D1) at = +../backends/dbfactory.cc:490 +#8 0xb7fb8c68 in notmuch_database_open_verbose (path=3D0x80780c8 = +"\340\327=C5=B7", mode=3DNOTMUCH_DATABASE_MODE_READ_WRITE, = +database=3D0xbffff3e4,=20 + status_string=3D0xbffff39c) at lib/database.cc:933 +#9 0xb7fb975c in notmuch_database_open (path=3D0x8078068 = +"/home/guyzmo/Maildir", = +mode=3Dmode@entry=3DNOTMUCH_DATABASE_MODE_READ_WRITE,=20 + database=3Ddatabase@entry=3D0xbffff3e4) at lib/database.cc:848 +#10 0x0805cbe4 in notmuch_tag_command (config=3D0x80745b8, argc=3D3, = +argv=3D0xbffff638) at notmuch-tag.c:262 +#11 0x0804df87 in main (argc=3D4, argv=3D0xbffff634) at notmuch.c:421 +(gdb)=20 + +I=E2=80=99m running notmuch on a Debian Jessie (v7), on which I compiled = +latest notmuch +from git, using the repository=E2=80=99s libxapian which is v1.2.12-2. + +Now, I could use a newer version of libxapian, either V1.2.19 from the = +debian=20 +backports (https://packages.debian.org/jessie/libxapian22), or get the = +latest +compiling it from sources, as it looks like the flintlock.cc compilation = +unit +has been rewritten since v1.2.12 =E2=80=94 though it looks like the part = +where it hangs +is looking quite alike, but it=E2=80=99s hard to tell from the changes = +whether that=20 +would have an inpact. + +So my question is whether this is a known issue, and is actually related = +to +the fact that I=E2=80=99m running it over NFS. I looked for NFS related = +comments on +xapian website and so far I found nothing looking alike my issue. + +I=E2=80=99ll try testing with other versions of xapian and see it solves = +it. If the +issue is still on with HEAD of xapian=E2=80=99s sources, then I=E2=80=99ll= + do a bug report. +Until then, I=E2=80=99m just checking on this list for ideas and = +comments, in hope +for a working solution. + +Cheers, + +--=20 +Guyzmo=