From a41f7c3b83d890939fd44ba9c0bdc3aaa133d036 Mon Sep 17 00:00:00 2001 From: Gaute Hope Date: Mon, 11 Aug 2014 14:17:54 +0200 Subject: [PATCH] DatabaseModifiedErrors causing troubles --- a5/c2815329b480916727a304a3af1824095b3705 | 161 ++++++++++++++++++++++ 1 file changed, 161 insertions(+) create mode 100644 a5/c2815329b480916727a304a3af1824095b3705 diff --git a/a5/c2815329b480916727a304a3af1824095b3705 b/a5/c2815329b480916727a304a3af1824095b3705 new file mode 100644 index 000000000..2eabdef0c --- /dev/null +++ b/a5/c2815329b480916727a304a3af1824095b3705 @@ -0,0 +1,161 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id AB025431FAF + for ; Mon, 11 Aug 2014 05:18:08 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 74MwyFmpTj8R for ; + Mon, 11 Aug 2014 05:17:58 -0700 (PDT) +Received: from mail-qg0-f48.google.com (mail-qg0-f48.google.com + [209.85.192.48]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 120CA431FAE + for ; Mon, 11 Aug 2014 05:17:58 -0700 (PDT) +Received: by mail-qg0-f48.google.com with SMTP id i50so8336100qgf.35 + for ; Mon, 11 Aug 2014 05:17:56 -0700 (PDT) +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:mime-version:date:message-id:subject:from:to + :content-type; + bh=zNSOgcDHBtBfm3lylf7EplrsfpI2eV7SX6iIlF1Edyw=; + b=iJGTIxnT6TTZvB4t035d2YqzXuipL/xpGjKKS2j5qkRRLr8y3VtD1mKdlR4Fnh808x + +zO4X0DZVdJDxt/yc/rg++Dv3YsNeGjhswW0u1FB7LkZJ5g1UfLw39955YNysjhCDu9J + dys2bONIS/02aJKdO75wX3W8dnq/cQ7BjhB6FbVff5OCGS02+3M7uKfv14UwxyXr1XCu + 4FZelACrz+r65HIDNPHD4XroZX3lqnK3IW2/Wvjd3iIIarv7B0XKd0PvAlWCmwRwqXEq + nafGEiN2rLvxKRCxzYzigNTL8eY8Ss3+3Pot4iaj0JyamjNBKr1TRfGW8+KV/gpXE+IV + Ugdg== +X-Gm-Message-State: + ALoCoQnNRkazZ2Ia8zvIDNVivGrJr7+FDOg+ha5SzL2v7T8U88SuHDsLk1uxoWIiWinreLZosh9V +MIME-Version: 1.0 +X-Received: by 10.224.137.65 with SMTP id v1mr62602608qat.53.1407759475308; + Mon, 11 Aug 2014 05:17:55 -0700 (PDT) +Received: by 10.140.86.148 with HTTP; Mon, 11 Aug 2014 05:17:54 -0700 (PDT) +Date: Mon, 11 Aug 2014 14:17:54 +0200 +Message-ID: + +Subject: DatabaseModifiedErrors causing troubles +From: Gaute Hope +To: notmuch +Content-Type: multipart/mixed; boundary=001a11c2cc3074c37c0500598b50 +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Mon, 11 Aug 2014 12:18:08 -0000 + +--001a11c2cc3074c37c0500598b50 +Content-Type: text/plain; charset=UTF-8 + +Hi, + +I've been working on an application that keeps a read-only handle on +the notmuch database open for a long time. In some cases when a new +message is added along with some renames of other messages using +'notmuch new' while the application is running I get an Xapian +exception: DatabaseModifiedError: + + A Xapian exception occurred performing query: The revision being +read has been discarded - you + should call Xapian::Database::reopen() and retry the operation. + +Which seems to be printed from: notmuch_query_search_threads -> +notmuch_query_search_messages:294. + +I have not been able to make a smaller test case at the moment (this +happens with offlineimap updating an maildir and notmuch new run +afterwards + some tagging). + +I can work around this by checking for a NULL pointer returned from +notmuch_query_search_threads () and re-open the database +(notmuch_database_close () -> notmuch_database_open ()). But I have no +way of knowing programatically if this really is the error that has +happened. There should be some way of propagating the error +information or (even better for my case; for notmuch to reopen the +database), one option is the Gmime way of passing an pointer to an +error structure that is filled up by the notmuch interface function. + +I made some attempts at exposing the ::reopen() function as suggested +by Xapian (http://xapian.org/docs/apidoc/html/classXapian_1_1Database.html#af140b1f8d948d13cf7be4a11a7c699a4), +but I end up with other errors afterwards. Possibly from leftover +structures created with the original database handle: + +after notmuch_database_reopen (see attached patch for your reference): + A Xapian exception occurred when reading header: Expected block +24615 to be level 1, not 0 + A Xapian exception occurred when reading header: Error reading block +419480589: got end of file + A Xapian exception occurred when reading date: Error reading block +419480589: got end of file + A Xapian exception occurred when reading header: Error reading block +419480589: got end of file + + +as mentioned, doing a manual _close and _open works. Again, the best +would be a consistent way to really know that this (or something else) +is the error that really happened. + +Cheers, Gaute + +--001a11c2cc3074c37c0500598b50 +Content-Type: text/x-patch; charset=US-ASCII; + name="0001-lib-expose-XapianDatabae-reopen-through-notmuch_data.patch" +Content-Disposition: attachment; + filename="0001-lib-expose-XapianDatabae-reopen-through-notmuch_data.patch" +Content-Transfer-Encoding: base64 +X-Attachment-Id: f_hyprlohc0 + +RnJvbSA5MDM0MGZlNTlkNjc3Yzk4OTM1MmYwOGU4MmY5MDgwMTZjMjVmYWZhIE1vbiBTZXAgMTcg +MDA6MDA6MDAgMjAwMQpGcm9tOiBHYXV0ZSBIb3BlIDxlZ0BnYXV0ZS52ZXRzai5jb20+CkRhdGU6 +IE1vbiwgMTEgQXVnIDIwMTQgMTQ6MTY6MTIgKzAyMDAKU3ViamVjdDogW1BBVENIXSBsaWI6IGV4 +cG9zZSBYYXBpYW5EYXRhYmFlOjpyZW9wZW4oKSB0aHJvdWdoCiBub3RtdWNoX2RhdGFiYXNlX3Jl +b3BlbigpCgotLS0KIGxpYi9kYXRhYmFzZS5jYyB8IDE4ICsrKysrKysrKysrKysrKysrKwogbGli +L25vdG11Y2guaCAgIHwgMjIgKysrKysrKysrKysrKysrKysrKysrKwogMiBmaWxlcyBjaGFuZ2Vk +LCA0MCBpbnNlcnRpb25zKCspCgpkaWZmIC0tZ2l0IGEvbGliL2RhdGFiYXNlLmNjIGIvbGliL2Rh +dGFiYXNlLmNjCmluZGV4IGM3NjAyOTAuLjgwYWY0MTAgMTAwNjQ0Ci0tLSBhL2xpYi9kYXRhYmFz +ZS5jYworKysgYi9saWIvZGF0YWJhc2UuY2MKQEAgLTc3OCw2ICs3NzgsMjQgQEAgbm90bXVjaF9k +YXRhYmFzZV9vcGVuIChjb25zdCBjaGFyICpwYXRoLAogfQogCiBub3RtdWNoX3N0YXR1c190Citu +b3RtdWNoX2RhdGFiYXNlX3Jlb3BlbiAobm90bXVjaF9kYXRhYmFzZV90ICpub3RtdWNoKQorewor +ICB0cnkgeworICAgIG5vdG11Y2gtPnhhcGlhbl9kYi0+cmVvcGVuICgpOworCisgICAgcmV0dXJu +IE5PVE1VQ0hfU1RBVFVTX1NVQ0NFU1M7CisKKyAgfSBjYXRjaCAoY29uc3QgWGFwaWFuOjpFcnJv +ciAmZXJyb3IpIHsKKwkgIGlmICghIG5vdG11Y2gtPmV4Y2VwdGlvbl9yZXBvcnRlZCkgeworCSAg +ICBmcHJpbnRmIChzdGRlcnIsICJFcnJvcjogQSBYYXBpYW4gZXhjZXB0aW9uIG9jY3VycmVkIHdo +aWxlIHJlb3BlbmluZyBkYXRhYmFzZTogJXNcbiIsCisJCSAgICAgZXJyb3IuZ2V0X21zZygpLmNf +c3RyKCkpOworCSAgfQorCisgICAgcmV0dXJuIE5PVE1VQ0hfU1RBVFVTX1hBUElBTl9FWENFUFRJ +T047CisgIH0KK30KKworbm90bXVjaF9zdGF0dXNfdAogbm90bXVjaF9kYXRhYmFzZV9jbG9zZSAo +bm90bXVjaF9kYXRhYmFzZV90ICpub3RtdWNoKQogewogICAgIG5vdG11Y2hfc3RhdHVzX3Qgc3Rh +dHVzID0gTk9UTVVDSF9TVEFUVVNfU1VDQ0VTUzsKZGlmZiAtLWdpdCBhL2xpYi9ub3RtdWNoLmgg +Yi9saWIvbm90bXVjaC5oCmluZGV4IDNjNWVjOTguLjAwOTUwYWYgMTAwNjQ0Ci0tLSBhL2xpYi9u +b3RtdWNoLmgKKysrIGIvbGliL25vdG11Y2guaApAQCAtMjc3LDYgKzI3NywyOCBAQCBub3RtdWNo +X2RhdGFiYXNlX29wZW4gKGNvbnN0IGNoYXIgKnBhdGgsCiAJCSAgICAgICBub3RtdWNoX2RhdGFi +YXNlX3QgKipkYXRhYmFzZSk7CiAKIC8qKgorICogUmVvcGVuIHRoZSBnaXZlbiBub3RtdWNoIGRh +dGFiYXNlLgorICoKKyAqIFRoZSB1bmRlcmx5aW5nIFhhcGlhbiBkYXRhYmFzZSB3aWxsIGJlIHJl +LW9wZW5lZCB0byB0aGUgbGF0ZXN0CisgKiBhdmFpbGFibGUgdmVyc2lvbi4gSXQgY2FuIGJlIHVz +ZWQgdG8gbWFrZSBzdXJlIHRoZSBsYXRlc3QgcmVzdWx0cworICogYXJlIHJldHVybmVkIG9yIHRv +IHJlY292ZXIgZnJvbSBhbiBYYXBhaWFuOjpEYXRhYmFzZU1vZGlmaWVkRXJyb3IKKyAqIHdoaWNo +IGNhbiBvY2N1ciBhZnRlciBleHRlcm5hbCBkYXRhYmFzZSBtb2RpZmljYXRpb24uCisgKgorICog +Q2FsbGluZyBub3RtdWNoX2RhdGFiYXNlX3Jlb3BlbiBvbiBhIGRhdGFiYXNlIHRoYXQgaGFzIGJl +ZW4gY2xvc2VkCisgKiB3aWxsIHJlc3VsdCBpbiBhIE5PVE1VQ0hfU1RBVFVTX1hBUEFJQU5fRVhD +RVBUSU9OLgorICoKKyAqIFJldHVybiB2YWx1ZToKKyAqCisgKiBOT1RNVUNIX1NUQVRVU19TVUND +RVNTOiBTdWNjZXNzZnVsbHkgcmVvcGVuZWQgZGF0YWJhc2UuCisgKgorICogTk9UTVVDSF9TVEFU +VVNfWEFQSUFOX0VYQ0VQVElPTjogQSBYYXBpYW4gZXhjZXB0aW9uIG9jY3VycmVkLgorICoKKyAq +Lworbm90bXVjaF9zdGF0dXNfdAorbm90bXVjaF9kYXRhYmFzZV9yZW9wZW4gKG5vdG11Y2hfZGF0 +YWJhc2VfdCAqZGF0YWJhc2UpOworCisKKy8qKgogICogQ2xvc2UgdGhlIGdpdmVuIG5vdG11Y2gg +ZGF0YWJhc2UuCiAgKgogICogQWZ0ZXIgbm90bXVjaF9kYXRhYmFzZV9jbG9zZSBoYXMgYmVlbiBj +YWxsZWQsIGNhbGxzIHRvIG90aGVyCi0tIAoyLjAuMwoK +--001a11c2cc3074c37c0500598b50-- -- 2.26.2