From 1df960eae7451f68b98d63afa5cdca4f976b2e56 Mon Sep 17 00:00:00 2001 From: Guyzmo Date: Wed, 6 Jan 2016 15:49:28 +0000 Subject: [PATCH] Xapian lockup when writing to the notmuch database --- 10/c04e841f8a9b0291dbd00847981ccba3873cfe | 186 ++++++++++++++++++++++ 1 file changed, 186 insertions(+) create mode 100644 10/c04e841f8a9b0291dbd00847981ccba3873cfe diff --git a/10/c04e841f8a9b0291dbd00847981ccba3873cfe b/10/c04e841f8a9b0291dbd00847981ccba3873cfe new file mode 100644 index 000000000..1566bbba2 --- /dev/null +++ b/10/c04e841f8a9b0291dbd00847981ccba3873cfe @@ -0,0 +1,186 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id 5C6E46DE10F8 + for ; Wed, 6 Jan 2016 07:49:49 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.315 +X-Spam-Level: +X-Spam-Status: No, score=-0.315 tagged_above=-999 required=5 tests=[AWL=0.337, + DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.55, + SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id t2DqjJfSpLEN for ; + Wed, 6 Jan 2016 07:49:47 -0800 (PST) +Received: from elendil.m0g.net (elendil.m0g.net [212.83.155.195]) + by arlo.cworth.org (Postfix) with ESMTPS id 642BF6DE103A + for ; Wed, 6 Jan 2016 07:49:47 -0800 (PST) +Received: from authenticated-user (elendil.m0g.net [212.83.155.195]) + (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + by elendil.m0g.net (Postfix) with ESMTPSA id EA1552A0C5A + for ; Wed, 6 Jan 2016 16:49:45 +0100 (CET) +DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=m0g.net; s=mail; + t=1452095386; bh=luCc/k/ih3GmMNLCHbAHU0RqsP2PvvE2Z1GXzh685Ek=; + h=From:Subject:Date:To:From; + b=d52dhNmMm5PmWm6JxMV0b842v4/Ie9IeFdcKHkgvQ87iGoCqfmwe0SA+4iATpw0Yv + IqxpsDAiTW0NzOHwRdoQ/QyQ7V8QuS6WfLdzN+QTs5K5g0YZKiACpD+QX2zXAFkrf6 + cf0zotXh2eallC9w0Sb9x6y8ip4Qt5YlMMFkq2SWdycnZb5SJIcgQdW0uj30mDuvHG + MqH4A6ud63LdI6IfASzkBcXuy6rrPTB3428JnS9FZDdGI+1F6v0PN0HXJRrFbakHak + dEzV64ePd/NK3wG9beKSciQG5ZVPQxhJa81BYVZZqQpD3cgP5kEgpwWHIDATxvbzX0 + ZokF56CUa8maQ== +From: Guyzmo +Content-Type: text/plain; charset=utf-8 +Content-Transfer-Encoding: quoted-printable +Subject: Xapian lockup when writing to the notmuch database +Message-Id: <67EEA3E1-918F-47AE-8AD7-EF0A5923D800@m0g.net> +Date: Wed, 6 Jan 2016 15:49:28 +0000 +To: notmuch@notmuchmail.org +Mime-Version: 1.0 +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 06 Jan 2016 15:49:49 -0000 + +Hello, + +I have migrated my mail configuration from an old machine to a new one, = +which is a xen=20 +server with several VM instances. I have one VM dedicated to do mail = +stuff, and I want +to manage my Maildir using notmuch in another one. So I set up an NFSv4 = +share (with +sync option set) from the mail server, to have my Maildir accessible = +from the other VM.=20 +Of course, I made sure my user is able to have full privileges over the = +mounted share, +and there=E2=80=99s only one instance of notmuch running over that = +xapian db at all times. + +All read only operations work just perfectly fine, I can even run = +notmuch-kz and have +it list my search mailboxes. + +But when I do `notmuch new` it hangs. So I did ran it through gdb and = +here is the result: + +% gdb notmuch +(gdb) run new +Starting program: /usr/local/bin/notmuch new +[Thread debugging using libthread_db enabled] +Using host libthread_db library = +"/lib/i386-linux-gnu/i686/cmov/libthread_db.so.1". +^C +Program received signal SIGINT, Interrupt. +0xb7fe1428 in __kernel_vsyscall () +(gdb) bt +#0 0xb7fe1428 in __kernel_vsyscall () +#1 0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 +#2 0xb7adf538 in read (__nbytes=3D1, __buf=3D0xbfffedfc, = +__fd=3D) at /usr/include/i386-linux-gnu/bits/unistd.h:45 +#3 FlintLock::lock (this=3D0x8079180, exclusive=3Dtrue, = +explanation=3D...) at ../backends/flint_lock.cc:222 +#4 0xb7b30a86 in ChertDatabase::get_database_write_lock = +(this=3Dthis@entry=3D0x80789a0, creating=3Dcreating@entry=3Dfalse) + at ../backends/chert/chert_database.cc:505 +#5 0xb7b34fd2 in ChertDatabase::ChertDatabase = +(this=3Dthis@entry=3D0x80789a0, chert_dir=3D..., action=3Daction@entry=3D1= +, block_size=3Dblock_size@entry=3D8192) + at ../backends/chert/chert_database.cc:154 +#6 0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase = +(this=3D0x80789a0, dir=3D..., action=3D1, block_size=3D8192) + at ../backends/chert/chert_database.cc:1036 +#7 0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase = +(this=3D0x80780c8, path=3D..., action=3D1) at = +../backends/dbfactory.cc:490 +#8 0xb7fb8c68 in notmuch_database_open_verbose (path=3D0x80780c8 = +"\340\327=C5=B7", mode=3DNOTMUCH_DATABASE_MODE_READ_WRITE, = +database=3D0xbffff230,=20 + status_string=3D0xbffff248) at lib/database.cc:933 +#9 0x08054e93 in notmuch_new_command (config=3D0x80745b8, argc=3D1, = +argv=3D0xbffff678) at notmuch-new.c:1008 +#10 0x0804df87 in main (argc=3D2, argv=3D0xbffff674) at notmuch.c:421 +(gdb)=20 + +then I tried doing another write operation, doing a `notmuch tag` = +command + +% gdb notmuch +(gdb) run tag +tag id: +(gdb) bt +#0 0xb7fe1428 in __kernel_vsyscall () +#1 0xb7d2a953 in read () from /lib/i386-linux-gnu/i686/cmov/libc.so.6 +#2 0xb7adf538 in read (__nbytes=3D1, __buf=3D0xbfffefac, = +__fd=3D) at /usr/include/i386-linux-gnu/bits/unistd.h:45 +#3 FlintLock::lock (this=3D0x8079360, exclusive=3Dtrue, = +explanation=3D...) at ../backends/flint_lock.cc:222 +#4 0xb7b30a86 in ChertDatabase::get_database_write_lock = +(this=3Dthis@entry=3D0x8078b80, creating=3Dcreating@entry=3Dfalse) + at ../backends/chert/chert_database.cc:505 +#5 0xb7b34fd2 in ChertDatabase::ChertDatabase = +(this=3Dthis@entry=3D0x8078b80, chert_dir=3D..., action=3Daction@entry=3D1= +, block_size=3Dblock_size@entry=3D8192) + at ../backends/chert/chert_database.cc:154 +#6 0xb7b35554 in ChertWritableDatabase::ChertWritableDatabase = +(this=3D0x8078b80, dir=3D..., action=3D1, block_size=3D8192) + at ../backends/chert/chert_database.cc:1036 +#7 0xb7adcee4 in Xapian::WritableDatabase::WritableDatabase = +(this=3D0x80780c8, path=3D..., action=3D1) at = +../backends/dbfactory.cc:490 +#8 0xb7fb8c68 in notmuch_database_open_verbose (path=3D0x80780c8 = +"\340\327=C5=B7", mode=3DNOTMUCH_DATABASE_MODE_READ_WRITE, = +database=3D0xbffff3e4,=20 + status_string=3D0xbffff39c) at lib/database.cc:933 +#9 0xb7fb975c in notmuch_database_open (path=3D0x8078068 = +"/home/guyzmo/Maildir", = +mode=3Dmode@entry=3DNOTMUCH_DATABASE_MODE_READ_WRITE,=20 + database=3Ddatabase@entry=3D0xbffff3e4) at lib/database.cc:848 +#10 0x0805cbe4 in notmuch_tag_command (config=3D0x80745b8, argc=3D3, = +argv=3D0xbffff638) at notmuch-tag.c:262 +#11 0x0804df87 in main (argc=3D4, argv=3D0xbffff634) at notmuch.c:421 +(gdb)=20 + +I=E2=80=99m running notmuch on a Debian Wheezy, on which I compiled = +latest notmuch +from git, using the repository=E2=80=99s libxapian which is v1.2.12-2. + +Now, I could use a newer version of libxapian, either V1.2.16 from the = +debian=20 +wheezy backports, or get the latest compiling it from sources, as it = +looks=20 +like the flintlock.cc compilation unit has been rewritten since v1.2.12 +=E2=80=94 though it looks like the part where it hangs is looking quite = +alike, but=20 +it=E2=80=99s hard to tell from the changes whether that would have an = +inpact. + +So my question is whether this is a known issue, and is actually related = +to +the fact that I=E2=80=99m running it over NFS. I looked for NFS related = +comments on +xapian website and so far I found nothing looking alike my issue. + +I=E2=80=99ll try testing with other versions of xapian and see it solves = +it. If the +issue is still on with HEAD of xapian=E2=80=99s sources, then I=E2=80=99ll= + do a bug report. +Until then, I=E2=80=99m just checking on this list for ideas and = +comments, in hope +for a working solution. + +Cheers, + +--=20 +Guyzmo= -- 2.26.2