Date: Sun, 17 Jul 2011 20:35:38 +0100
From: Patrick Totzke <patricktotzke@googlemail.com>
To: notmuch@notmuchmail.org
Subject: Re: xapian exceptions not caught in python bindings?
Message-ID: <20110717193538.GA16615@brick.lan>
References: <20110626202733.GA26837@brick>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="r5Pyd7+fXNt84Ff3"
Content-Disposition: inline
In-Reply-To: <20110626202733.GA26837@brick>
User-Agent: Mutt/1.5.21 (2010-09-15)
Precedence: list


--r5Pyd7+fXNt84Ff3
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi all,

I know this issue is not easy to fix and that it's partly a problem
with xapians bad error handling. I just want to state another problem we run
into with xapians exceptions in combination with the python bindings.
If you run the following snippet, you notice that not only do we get
xapian-garbage on stderr but we don't really get any exceptions at the
position where it would make sense:

--------------------------------
=66rom notmuch import Database
bad_querystring =3D "test AND"

# This should raise a NotmuchError and query should be NoneType
query =3D Database().create_query(bad_querystring)

# this prints to stderr but returns a Threads object.
threads =3D query.search_threads()

# this finally raises a NotmuchError(STATUS.NOT_INITIALIZED)
for t in threads: print t
--------------------------------

Although the querystring is invalid syntactically, the query object gets=20
instantiated and can of course not behave as it should later on.
=46rom a user point of view it is really quite strange that only later
(if one iterates over threads) an exception is raised.

One solution that comes to mind here is simply bending sys.sdterr to someth=
ing internal,
interpreting its output and raising the correct exceptions accordingly.
The downside is, this would have to happen every time the library gets call=
ed.
Also, a fix like this would be python specific, so everybody else would sti=
ll get
garbage printed to stderr occasionally.

Thoughts anyone?
/p



On Sun, Jun 26, 2011 at 09:27:33PM +0100, Patrick Totzke wrote:
> Hi all!
> It's me again, with some strange python behaviour :/
>=20
> When I iterate over threads or messages only partially,
> then make changes to the index and continue the iteration,
> then I don't necessarily get NotmuchError's but the underlying
> libnotmuch seems to call terminate. I get the following two lines on
> stderr:
>   terminate called after throwing an instance of 'Xapian::DatabaseModifie=
dError'
>   Aborted
> then python aborts without an exception.
> This happens on current master with current python bindings. To reproduce
> use this to iterate over messages:
>=20
> ------------------%<-------------------
> #it_read.py
> from notmuch import Query,Database
> msgs =3D Database().create_query('*').search_messages()
>=20
> i=3D0
> for m in msgs:
>     i=3Di+1
>     if i%50=3D=3D0:
>         raw_input()
>     readit =3D str(m)
> ------------------>%-------------------
>=20
> start it, then use this to change the index:
>=20
> ------------------%<-------------------
> #it_write.py=20
> from notmuch import Query,Database
> db =3D Database(mode =3D Database.MODE.READ_WRITE)
> msgs =3D Query(db,'*').search_messages()
>=20
> i=3D0
> for m in msgs:
>     i=3Di+1
>     if i%20=3D=3D0:
>         m.add_tag('REMOVEME')
> ------------------>%-------------------
>=20
> you have to alternate a bit between hitting enter in the
> read code and rerunning the write code.
> Read will fail at some point.
>=20
> The strange thing is that its not consistent:
> I stumbled upon this while reading a list of threads from notmuch.threads=
 on
> demand. If reading from the iterator catches a NotmuchError, i create a n=
ew one
> and update my list threads read so far. This does work, but: on every
> NotmuchError, I get this on stderr:
>  A Xapian exception occurred performing query: The revision being read ha=
s been discarded - you should call Xapian::Database::reopen() and retry the=
 operation
>  Query string was: thread:0000000000001876
>=20
> After a couple of resets this does not happen again but the next time i r=
ead
> from an outdated index, the whole UI terminates with the "Aborted" message
> stated above.
>=20
> To sum up, I think there are two bad things happening here:
> 1) should libnotmuch really print to stderr?
> 2) the python bindings are either not relaying all exceptions or libnotmu=
ch is not relaying=20
> all xapian exceptions in order for the bindings to intercept them
>=20
> cheers,
> /p



--r5Pyd7+fXNt84Ff3
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk4jOYoACgkQlDQDZ9fWxaqAqACg0YgDBWosSX5bPFp0uRBJ17nY
EBYAnjfDYSXsGqpDpTi5xghHi4REcyKz
=0rOH
-----END PGP SIGNATURE-----

--r5Pyd7+fXNt84Ff3--