Return-Path: <teythoon@jade-hamburg.de>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 0E369431FBC
	for <notmuch@notmuchmail.org>; Fri, 24 Jan 2014 08:40:29 -0800 (PST)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: 0
X-Spam-Level: 
X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
	autolearn=disabled
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id Ez7wGiwbOeS3 for <notmuch@notmuchmail.org>;
	Fri, 24 Jan 2014 08:40:24 -0800 (PST)
X-Greylist: delayed 385 seconds by postgrey-1.32 at olra;
	Fri, 24 Jan 2014 08:40:24 PST
Received: from mail.cryptobitch.de (cryptobitch.de [88.198.7.68])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by olra.theworths.org (Postfix) with ESMTPS id 73AA7431FB6
	for <notmuch@notmuchmail.org>; Fri, 24 Jan 2014 08:40:24 -0800 (PST)
Received: from mail.jade-hamburg.de (mail.jade-hamburg.de [85.183.11.228])
	(using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
	(No client certificate requested)
	by mail.cryptobitch.de (Postfix) with ESMTPSA id 72AD070DD0D
	for <notmuch@notmuchmail.org>; Fri, 24 Jan 2014 17:33:54 +0100 (CET)
Received: by mail.jade-hamburg.de (Postfix, from userid 401)
	id C86DEDF2A2; Fri, 24 Jan 2014 17:33:07 +0100 (CET)
Received: from thinkbox.jade-hamburg.de (cryptobitch.de [88.198.7.68])
	(using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits))
	(No client certificate requested) (Authenticated sender: teythoon)
	by mail.jade-hamburg.de (Postfix) with ESMTPSA id 04425DF28B;
	Fri, 24 Jan 2014 17:33:03 +0100 (CET)
Received: from teythoon by thinkbox.jade-hamburg.de with local (Exim 4.80)
	(envelope-from <teythoon@thinkbox.jade-hamburg.de>)
	id 1W6jgk-00019F-KY; Fri, 24 Jan 2014 17:33:02 +0100
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
From: Justus Winter <4winter@informatik.uni-hamburg.de>
User-Agent: alot/0.3.4
To: David Bremner <david@tethera.net>,  notmuch@notmuchmail.org
References:
 <1318198374-926-1-git-send-email-4winter@informatik.uni-hamburg.de>
	<1318198374-926-2-git-send-email-4winter@informatik.uni-hamburg.de>
	<4eddf2b1.4288980a.0b74.5557@mx.google.com> <87fwgx7vmm.fsf@SSpaeth.de>
	<E1RYMoW-0004Sc-R1@thinkbox.jade-hamburg.de>	<87r47xpli0.fsf@zancas.localnet>
In-Reply-To: <87r47xpli0.fsf@zancas.localnet>
Message-ID: <20140124163302.2392.83892@thinkbox.jade-hamburg.de>
Subject: Re: [PATCH 2/2] python: annotate all calls into libnotmuch with types
Date: Fri, 24 Jan 2014 17:33:02 +0100
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Fri, 24 Jan 2014 16:40:29 -0000

Hi :)

Quoting David Bremner (2014-01-24 14:17:27)
> Justus Winter <4winter@informatik.uni-hamburg.de> writes:
> =

> > Quoting Justus Winter (2011-12-07 19:49:31)
> =

> > And another one:
> [stack trace snipped]
> > #69 0x00000000004e10be in PyRun_SimpleFileExFlags (fp=3D0xb99c10, filen=
ame=3D<optimized out>, closeit=3D1, flags=3D0x7fffc13a1f60) at ../Python/py=
thonrun.c:936
> > #70 0x00000000004f10fd in Py_Main (argc=3D<optimized out>, argv=3D<opti=
mized out>) at ../Modules/main.c:599
> > #71 0x00007f72f2cbaead in __libc_start_main (main=3D<optimized out>, ar=
gc=3D<optimized out>, ubp_av=3D<optimized out>, init=3D<optimized out>, fin=
i=3D<optimized out>, =

> >     rtld_fini=3D<optimized out>, stack_end=3D0x7fffc13a2078) at libc-st=
art.c:228
> > #72 0x000000000041f199 in _start ()
> > (gdb) q
> >
> > with stderr saying:
> >
> > terminate called after throwing an instance of 'Xapian::DatabaseModifie=
dError'
> > Aborted (core dumped)
> >
> > This is an uncought c++ exception, right? If so I think it has to be
> > cought somewhere in libnotmuch and turned into the appropriate error
> > code (hm, there is only the generic XAPIAN_EXCEPTION, I thought there
> > was a way to indicate that the db has been modified?).
> =

> Can you still reproduce this bug?  If so, a small test case (python is
> fine, but ideally not requiring afew) would be helpful.

I'm sorry, I've no idea how to reproduce this. I have, however, some
more test cases that crash libnotmuch, like this one:

~~~ snap ~~~
import os
import notmuch

db_path =3D os.path.expanduser('~/Maildir')

db =3D notmuch.Database(db_path, mode=3Dnotmuch.Database.MODE.READ_WRITE)
directory =3D db.get_directory(os.path.join(db_path, 'cur'))
db._destroy(db._db)

try:
    directory.get_child_files()
except notmuch.errors.NotInitializedError as e:
    print('yeah')
else:
    assert False, "Hey, where's my exception?"
~~~ snap ~~~

% python destroy_minimal.py
[2]    3907 abort (core dumped)  python destroy_minimal.py

Program terminated with signal 6, Aborted.
#0  0x00007f996ad021d5 in __GI_raise (sig=3Dsig@entry=3D6) at ../nptl/sysde=
ps/unix/sysv/linux/raise.c:56
56      ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007f996ad021d5 in __GI_raise (sig=3Dsig@entry=3D6) at ../nptl/sysde=
ps/unix/sysv/linux/raise.c:56
#1  0x00007f996ad05388 in __GI_abort () at abort.c:90
#2  0x00007f99699d4486 in talloc_abort (reason=3D0x7f99699db648 "Bad talloc=
 magic value - access after free") at ../talloc.c:317
#3  0x00007f99699d8097 in talloc_abort_access_after_free () at ../talloc.c:=
336
#4  talloc_chunk_from_ptr (ptr=3D0x23e6f20) at ../talloc.c:357
#5  talloc_chunk_from_ptr (ptr=3D0x23e6f20) at ../talloc.c:2064
#6  __talloc (size=3D17, context=3D0x23e6f20) at ../talloc.c:555
#7  talloc_vasprintf (t=3D<optimized out>, fmt=3D0x7f996a3a56ef "%s%u:", ap=
=3D0x7fffb612bff8) at ../talloc.c:2079
#8  0x00007f99699d8167 in talloc_asprintf (t=3D<optimized out>, fmt=3D<opti=
mized out>) at ../talloc.c:2101
#9  0x00007f996a39c02e in notmuch_directory_get_child_files () from /home/t=
eythoon/.local/lib/libnotmuch.so.3
#10 0x00007f996a5b1cfc in ffi_call_unix64 () from /usr/lib/x86_64-linux-gnu=
/libffi.so.6
#11 0x00007f996a5b162c in ffi_call () from /usr/lib/x86_64-linux-gnu/libffi=
.so.6
#12 0x00007f996a7c86d0 in _ctypes_callproc () from /usr/lib/python2.7/lib-d=
ynload/_ctypes.x86_64-linux-gnu.so
#13 0x00007f996a7ca08e in PyCFuncPtr_call.3149.2622 () from /usr/lib/python=
2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so
#14 0x000000000055f47a in PyEval_EvalFrameEx () at ../Objects/abstract.c:25=
29
#15 0x000000000055f7ba in PyEval_EvalFrameEx () at ../Python/ceval.c:4107
#16 0x0000000000566bfb in PyEval_EvalCode () at ../Python/ceval.c:3253
#17 0x0000000000469499 in run_mod.42569 () at ../Python/pythonrun.c:1370
#18 0x0000000000469819 in PyRun_FileExFlags () at ../Python/pythonrun.c:1356
#19 0x0000000000469d52 in PyRun_SimpleFileExFlags () at ../Python/pythonrun=
.c:948
#20 0x000000000046b65f in Py_Main () at ../Modules/main.c:640
#21 0x00007f996acee995 in __libc_start_main (main=3D0x46b71d <main>, argc=
=3D2, ubp_av=3D0x7fffb612ca58, init=3D<optimized out>, fini=3D<optimized ou=
t>, rtld_fini=3D<optimized out>, =

    stack_end=3D0x7fffb612ca48) at libc-start.c:276
#22 0x0000000000573f2e in _start ()

This is a bit contrived b/c I'm destroying the db object by
hand. Never the less libnotmuch calls abort, and there is no way to
contain something like this in the python bindings.

I've seen your recent proposal to improve the error reporting for
libnotmuch functions. I think it's awesome, albeit a little late.

I've no idea how other frontends deal with libnotmuch, but alot has
moved libnotmuch to it's own process to contain any crashes (or
libnotmuch calling exit(3)), and to prevent libnotmuch from writing to
stderr destroying alots curses gui. I've always felt like libnotmuch
users were second-class citizens, the primary target of libnotmuch
being the notmuch binary.

Justus