MIME-Version: 1.0
Date: Mon, 11 Aug 2014 14:17:54 +0200
Message-ID:
 <CABKe4MvEdcsq8BZ-vq6R0Vnw87zEgBvqW_2F-Wysf5GNchqweg@mail.gmail.com>
Subject: DatabaseModifiedErrors causing troubles
From: Gaute Hope <eg@gaute.vetsj.com>
To: notmuch <notmuch@notmuchmail.org>
Content-Type: multipart/mixed; boundary=001a11c2cc3074c37c0500598b50
Precedence: list

--001a11c2cc3074c37c0500598b50
Content-Type: text/plain; charset=UTF-8

Hi,

I've been working on an application that keeps a read-only handle on
the notmuch database open for a long time. In some cases when a new
message is added along with some renames of other messages using
'notmuch new' while the application is running I get an Xapian
exception: DatabaseModifiedError:

  A Xapian exception occurred performing query: The revision being
read has been discarded - you
  should call Xapian::Database::reopen() and retry the operation.

Which seems to be printed from: notmuch_query_search_threads ->
notmuch_query_search_messages:294.

I have not been able to make a smaller test case at the moment (this
happens with offlineimap updating an maildir and notmuch new run
afterwards + some tagging).

I can work around this by checking for a NULL pointer returned from
notmuch_query_search_threads () and re-open the database
(notmuch_database_close () -> notmuch_database_open ()). But I have no
way of knowing programatically if this really is the error that has
happened. There should be some way of propagating the error
information or (even better for my case; for notmuch to reopen the
database), one option is the Gmime way of passing an pointer to an
error structure that is filled up by the notmuch interface function.

I made some attempts at exposing the ::reopen() function as suggested
by Xapian (http://xapian.org/docs/apidoc/html/classXapian_1_1Database.html#af140b1f8d948d13cf7be4a11a7c699a4),
but I end up with other errors afterwards. Possibly from leftover
structures created with the original database handle:

after notmuch_database_reopen (see attached patch for your reference):
  A Xapian exception occurred when reading header: Expected block
24615 to be level 1, not 0
  A Xapian exception occurred when reading header: Error reading block
419480589: got end of file
  A Xapian exception occurred when reading date: Error reading block
419480589: got end of file
  A Xapian exception occurred when reading header: Error reading block
419480589: got end of file


as mentioned, doing a manual _close and _open works. Again, the best
would be a consistent way to really know that this (or something else)
is the error that really happened.

Cheers, Gaute

--001a11c2cc3074c37c0500598b50
Content-Type: text/x-patch; charset=US-ASCII; 
	name="0001-lib-expose-XapianDatabae-reopen-through-notmuch_data.patch"
Content-Disposition: attachment; 
	filename="0001-lib-expose-XapianDatabae-reopen-through-notmuch_data.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: f_hyprlohc0

RnJvbSA5MDM0MGZlNTlkNjc3Yzk4OTM1MmYwOGU4MmY5MDgwMTZjMjVmYWZhIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBHYXV0ZSBIb3BlIDxlZ0BnYXV0ZS52ZXRzai5jb20+CkRhdGU6
IE1vbiwgMTEgQXVnIDIwMTQgMTQ6MTY6MTIgKzAyMDAKU3ViamVjdDogW1BBVENIXSBsaWI6IGV4
cG9zZSBYYXBpYW5EYXRhYmFlOjpyZW9wZW4oKSB0aHJvdWdoCiBub3RtdWNoX2RhdGFiYXNlX3Jl
b3BlbigpCgotLS0KIGxpYi9kYXRhYmFzZS5jYyB8IDE4ICsrKysrKysrKysrKysrKysrKwogbGli
L25vdG11Y2guaCAgIHwgMjIgKysrKysrKysrKysrKysrKysrKysrKwogMiBmaWxlcyBjaGFuZ2Vk
LCA0MCBpbnNlcnRpb25zKCspCgpkaWZmIC0tZ2l0IGEvbGliL2RhdGFiYXNlLmNjIGIvbGliL2Rh
dGFiYXNlLmNjCmluZGV4IGM3NjAyOTAuLjgwYWY0MTAgMTAwNjQ0Ci0tLSBhL2xpYi9kYXRhYmFz
ZS5jYworKysgYi9saWIvZGF0YWJhc2UuY2MKQEAgLTc3OCw2ICs3NzgsMjQgQEAgbm90bXVjaF9k
YXRhYmFzZV9vcGVuIChjb25zdCBjaGFyICpwYXRoLAogfQogCiBub3RtdWNoX3N0YXR1c190Citu
b3RtdWNoX2RhdGFiYXNlX3Jlb3BlbiAobm90bXVjaF9kYXRhYmFzZV90ICpub3RtdWNoKQorewor
ICB0cnkgeworICAgIG5vdG11Y2gtPnhhcGlhbl9kYi0+cmVvcGVuICgpOworCisgICAgcmV0dXJu
IE5PVE1VQ0hfU1RBVFVTX1NVQ0NFU1M7CisKKyAgfSBjYXRjaCAoY29uc3QgWGFwaWFuOjpFcnJv
ciAmZXJyb3IpIHsKKwkgIGlmICghIG5vdG11Y2gtPmV4Y2VwdGlvbl9yZXBvcnRlZCkgeworCSAg
ICBmcHJpbnRmIChzdGRlcnIsICJFcnJvcjogQSBYYXBpYW4gZXhjZXB0aW9uIG9jY3VycmVkIHdo
aWxlIHJlb3BlbmluZyBkYXRhYmFzZTogJXNcbiIsCisJCSAgICAgZXJyb3IuZ2V0X21zZygpLmNf
c3RyKCkpOworCSAgfQorCisgICAgcmV0dXJuIE5PVE1VQ0hfU1RBVFVTX1hBUElBTl9FWENFUFRJ
T047CisgIH0KK30KKworbm90bXVjaF9zdGF0dXNfdAogbm90bXVjaF9kYXRhYmFzZV9jbG9zZSAo
bm90bXVjaF9kYXRhYmFzZV90ICpub3RtdWNoKQogewogICAgIG5vdG11Y2hfc3RhdHVzX3Qgc3Rh
dHVzID0gTk9UTVVDSF9TVEFUVVNfU1VDQ0VTUzsKZGlmZiAtLWdpdCBhL2xpYi9ub3RtdWNoLmgg
Yi9saWIvbm90bXVjaC5oCmluZGV4IDNjNWVjOTguLjAwOTUwYWYgMTAwNjQ0Ci0tLSBhL2xpYi9u
b3RtdWNoLmgKKysrIGIvbGliL25vdG11Y2guaApAQCAtMjc3LDYgKzI3NywyOCBAQCBub3RtdWNo
X2RhdGFiYXNlX29wZW4gKGNvbnN0IGNoYXIgKnBhdGgsCiAJCSAgICAgICBub3RtdWNoX2RhdGFi
YXNlX3QgKipkYXRhYmFzZSk7CiAKIC8qKgorICogUmVvcGVuIHRoZSBnaXZlbiBub3RtdWNoIGRh
dGFiYXNlLgorICoKKyAqIFRoZSB1bmRlcmx5aW5nIFhhcGlhbiBkYXRhYmFzZSB3aWxsIGJlIHJl
LW9wZW5lZCB0byB0aGUgbGF0ZXN0CisgKiBhdmFpbGFibGUgdmVyc2lvbi4gSXQgY2FuIGJlIHVz
ZWQgdG8gbWFrZSBzdXJlIHRoZSBsYXRlc3QgcmVzdWx0cworICogYXJlIHJldHVybmVkIG9yIHRv
IHJlY292ZXIgZnJvbSBhbiBYYXBhaWFuOjpEYXRhYmFzZU1vZGlmaWVkRXJyb3IKKyAqIHdoaWNo
IGNhbiBvY2N1ciBhZnRlciBleHRlcm5hbCBkYXRhYmFzZSBtb2RpZmljYXRpb24uCisgKgorICog
Q2FsbGluZyBub3RtdWNoX2RhdGFiYXNlX3Jlb3BlbiBvbiBhIGRhdGFiYXNlIHRoYXQgaGFzIGJl
ZW4gY2xvc2VkCisgKiB3aWxsIHJlc3VsdCBpbiBhIE5PVE1VQ0hfU1RBVFVTX1hBUEFJQU5fRVhD
RVBUSU9OLgorICoKKyAqIFJldHVybiB2YWx1ZToKKyAqCisgKiBOT1RNVUNIX1NUQVRVU19TVUND
RVNTOiBTdWNjZXNzZnVsbHkgcmVvcGVuZWQgZGF0YWJhc2UuCisgKgorICogTk9UTVVDSF9TVEFU
VVNfWEFQSUFOX0VYQ0VQVElPTjogQSBYYXBpYW4gZXhjZXB0aW9uIG9jY3VycmVkLgorICoKKyAq
Lworbm90bXVjaF9zdGF0dXNfdAorbm90bXVjaF9kYXRhYmFzZV9yZW9wZW4gKG5vdG11Y2hfZGF0
YWJhc2VfdCAqZGF0YWJhc2UpOworCisKKy8qKgogICogQ2xvc2UgdGhlIGdpdmVuIG5vdG11Y2gg
ZGF0YWJhc2UuCiAgKgogICogQWZ0ZXIgbm90bXVjaF9kYXRhYmFzZV9jbG9zZSBoYXMgYmVlbiBj
YWxsZWQsIGNhbGxzIHRvIG90aGVyCi0tIAoyLjAuMwoK
--001a11c2cc3074c37c0500598b50--