A systematic way of handling Xapian lock errors?

diff --git a/3c/630faf4f61dfd4c29e9a042e49e3d4febdd739 b/3c/630faf4f61dfd4c29e9a042e49e3d4febdd739
new file mode 100644 (file)
index 0000000..8da82c6
--- /dev/null
+++ b/3c/630faf4f61dfd4c29e9a042e49e3d4febdd739
@@ -0,0 +1,105 @@
+Return-Path: <marmstrong@google.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 614A16DE00BD\r
+ for <notmuch@notmuchmail.org>; Tue,  2 Aug 2016 08:26:54 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.924\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.924 tagged_above=-999 required=5 tests=[AWL=0.098,\r
+  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.211, SPF_PASS=-0.001,\r
+ T_RP_MATCHES_RCVD=-0.01] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id kCwxpYkKRXiE for <notmuch@notmuchmail.org>;\r
+ Tue,  2 Aug 2016 08:26:46 -0700 (PDT)\r
+Received: from mail-pf0-f172.google.com (mail-pf0-f172.google.com\r
+ [209.85.192.172])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id A65856DE00B8\r
+ for <notmuch@notmuchmail.org>; Tue,  2 Aug 2016 08:26:46 -0700 (PDT)\r
+Received: by mail-pf0-f172.google.com with SMTP id h186so67298282pfg.3\r
+ for <notmuch@notmuchmail.org>; Tue, 02 Aug 2016 08:26:46 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\r
+ s=20120113; h=from:to:subject:user-agent:date:message-id:mime-version;\r
+ bh=ffRJGbD5vRd7RuYrRx7W82o+QEU4NMc1xAvYrFV93qQ=;\r
+ b=PSR8Ok7Kkb68Pq8N232rXqEyY04XaZEmaumHaHDiXRbvhPeksrPF2QYtpIiibShYR4\r
+ LzzsTmmy5hc7DYpwgQk/ya3bYe1oQSwbt1qjs9u06Sa5S1WwFQZiNe0z45jTfsoPkPno\r
+ H1gHzY6H9twsCeWKpbkoKUU2EPMierUPRaXtIjSQ9R+eqHVc6PlWxo+2+xpQN3obiEog\r
+ r+cqD9ah6ZnBdEJDMHBpC5zTabCpU8lKLGoFUT6Ua6IV8SsIuNXTQdvKcm0+a7fQHT+M\r
+ LK5PiLlFidrufFzzn4p828IUU+J2o2FnCa4C28rigHx+py8pC+IzjTS6TKp60Cp78gXJ gIfA==\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:from:to:subject:user-agent:date:message-id\r
+ :mime-version;\r
+ bh=ffRJGbD5vRd7RuYrRx7W82o+QEU4NMc1xAvYrFV93qQ=;\r
+ b=AYpTdK9MFyxOjFf51AHRmLF+Kikrmlnu3TvQjAsMvgLB65VPvELQ99GtSsyvFCrX4l\r
+ zkO5YGtWD+adoElyC29+AjNRja+HbsdI0T6lssi7JCA7pXAZ3WlbSDB52cwNwcTca1LW\r
+ FkLECq90ZYXfZLJctMT8xtOQ32DcccaWnExmYxXTiwUBi+Yas2KzyVUYtIiNmfSUtXBQ\r
+ vJXXK+ucJ1KIAGBt4Ed1GGvT/S7EgPke8QcVrm1oo8R36uFveeUtlGh8QbVfJ6qHqJov\r
+ r8UI8nqg+7bmjFJWM9ngLMMZaftZMCPFwtDKdQ1FKBxyh6Rv6FjILujAVCE8CcCFm+l+\r
+ GGYw==\r
+X-Gm-Message-State:\r
+ AEkooutuhwv4MOUdqvrR2wMWBNbjx7lYXJP5GhWemalsXVtD9VhD22M+uuQE1GWeZGWnF3At\r
+X-Received: by 10.98.89.23 with SMTP id n23mr106972651pfb.34.1470151605729;\r
+ Tue, 02 Aug 2016 08:26:45 -0700 (PDT)\r
+Received: from marmstrong-linux.kir.corp.google.com\r
+ ([2620:0:1008:1101:419b:e62d:4938:a6b6])\r
+ by smtp.gmail.com with ESMTPSA id x66sm5700831pfb.86.2016.08.02.08.26.44\r
+ for <notmuch@notmuchmail.org>\r
+ (version=TLS1_2 cipher=AES128-SHA bits=128/128);\r
+ Tue, 02 Aug 2016 08:26:44 -0700 (PDT)\r
+From: Matt Armstrong <marmstrong@google.com>\r
+To: notmuch@notmuchmail.org\r
+Subject: A systematic way of handling Xapian lock errors?\r
+User-Agent: Notmuch/0.21 (https://notmuchmail.org) Emacs/24.3.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Tue, 02 Aug 2016 08:26:43 -0700\r
+Message-ID: <qf5d1lrkx8s.fsf@marmstrong-linux.kir.corp.google.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 02 Aug 2016 15:26:54 -0000\r
+\r
+Simple notmuch commands like "notmuch tag" can fail to grab the Xapian\r
+lock.  When this occurs they bail with:\r
+\r
+A Xapian exception occurred opening database: Unable to get write lock on /example/.notmuch/xapian: already locked\r
+\r
+I've noticed a few issues with this:\r
+\r
+1) The notmuch command line doesn't define semantics for exit codes.  I\r
+notice that notmuch exits with 1 for "xapian write lock error".  I\r
+suspect this code is not reserved for write lock errors?  That would be\r
+needed for any sensible downgrade of the write lock error into a soft\r
+failure with a retry loop in things like shell scripts.\r
+\r
+2) The notmuch Emacs client just bails whatever it was trying when this\r
+occurs.  I run background mail fetching, which I expect is typical, and\r
+this makes for an unpleasant experience as errors are thrown at me\r
+randomly.  Since my post-injection filter commands run quickly, I'd\r
+rather Emacs simply spin a short while against the lock trying to\r
+perform the command.  Even a one second spin attempt is likely to\r
+succeed most of the time.  But I would happily configure an infinite\r
+retry loop -- Emacs can show me a spinning ball, and I'd rather not be\r
+subject to odd effects due to failures.\r
+\r
+To address both, has something like this ever been considered:\r
+\r
+  notmuch --lock_timeout <seconds> COMMAND ARG...\r
+\r
+Then frontends like Emacs could lean on retry logic written in C.  If\r
+this seems workable, it is something I can try implementing.\r