From: Justus Winter <4winter@informatik.uni-hamburg.de> Date: Thu, 12 Apr 2012 09:05:33 +0000 (+0200) Subject: Re: [RFC] Split notmuch_database_close into two functions X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=08bed37b3587e5487360bf929cb7a908857415e7;p=notmuch-archives.git Re: [RFC] Split notmuch_database_close into two functions --- diff --git a/c8/b95192f68906ec30c48fdf813aa983bc2a2ca2 b/c8/b95192f68906ec30c48fdf813aa983bc2a2ca2 new file mode 100644 index 000000000..2fed99052 --- /dev/null +++ b/c8/b95192f68906ec30c48fdf813aa983bc2a2ca2 @@ -0,0 +1,151 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id E5605431FAF + for ; Thu, 12 Apr 2012 02:05:44 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 0 +X-Spam-Level: +X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none] + autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id yZ1fjiln2LuU for ; + Thu, 12 Apr 2012 02:05:42 -0700 (PDT) +Received: from mail.cryptobitch.de (cryptobitch.de [88.198.7.68]) + (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 47BF7431FAE + for ; Thu, 12 Apr 2012 02:05:42 -0700 (PDT) +Received: from mail.jade-hamburg.de (unknown [85.183.11.228]) + (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) + (No client certificate requested) + by mail.cryptobitch.de (Postfix) with ESMTPSA id 08B8656A613 + for ; Thu, 12 Apr 2012 11:05:40 +0200 (CEST) +Received: by mail.jade-hamburg.de (Postfix, from userid 401) + id 57F4BDF2A3; Thu, 12 Apr 2012 11:05:39 +0200 (CEST) +Received: from thinkbox.jade-hamburg.de (unknown [193.174.12.196]) + (using TLSv1 with cipher AES256-SHA (256/256 bits)) + (No client certificate requested) (Authenticated sender: teythoon) + by mail.jade-hamburg.de (Postfix) with ESMTPSA id 18186DF2A0; + Thu, 12 Apr 2012 11:05:35 +0200 (CEST) +Received: from teythoon by thinkbox.jade-hamburg.de with local (Exim 4.77) + (envelope-from ) + id 1SIFy9-0005cO-7n; Thu, 12 Apr 2012 11:05:33 +0200 +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: quoted-printable +To: Austin Clements +From: Justus Winter <4winter@informatik.uni-hamburg.de> +In-Reply-To: <20120401032323.GH5949@mit.edu> +References: + <1332291311-28954-1-git-send-email-4winter@informatik.uni-hamburg.de> + <20120401032323.GH5949@mit.edu> +Message-ID: <20120412090533.2074.78211@thinkbox.jade-hamburg.de> +User-Agent: alot/0.21+ +Subject: Re: [RFC] Split notmuch_database_close into two functions +Date: Thu, 12 Apr 2012 11:05:33 +0200 +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 12 Apr 2012 09:05:45 -0000 + +Quoting Austin Clements (2012-04-01 05:23:23) +>Quoth Justus Winter on Mar 21 at 1:55 am: +>> I propose to split the function notmuch_database_close into +>> notmuch_database_close and notmuch_database_destroy so that long +>> running processes like alot can close the database while still using +>> data obtained from queries to that database. +> +>Is this actually safe? My understanding of Xapian::Database::close is +>that, once you've closed the database, basically anything can throw a +>Xapian exception. A lot of data is retrieved lazily, both by notmuch +>and by Xapian, so simply having, say, a notmuch_message_t object isn't +>enough to guarantee that you'll be able to get data out of it after +>closing the database. Hence, I don't see how this interface could be +>used correctly. + +I do not know how, but both alot and afew (and occasionally the +notmuch binary) are somehow safely using this interface on my box for +the last three weeks. + +>Maybe you could describe your use case in more detail? + +Uh, okay. There are two long running processes (alot and afew in my +case) competing for a resource ("writable access to the notmuch +database"). This access is serialized by a lock maintained by +libxapian. This is a rather classic cs problem. + +In order to avoid starvation of one process, both processes must +release the lock after a finite amount of time. Now for all practical +purposes the requirement is usually that both processes should +minimize the time spent while they aquired the lock. + +The only way to ensure that the lock is released is to ask notmuch to +release the lock. When Patrick asked me for a way to release the lock +I exposed the function notmuch_database_close [0]. + +I made a mistake. The mistake was to assume that a function named +notmuch_database_close would actually do as the name implies. But that +wasn't the case. I should have been more suspicious, but being +somewhat naive I patched notmuch_database_close to actually close the +database [1]. + +I'm basically trying to correct that mistake. One way to fix that is +this patchset, the other one is to rename notmuch_database_close to +notmuch_database_destroy and to remove the ability to call this +function from the python bindings. + +There is a branch of alot [2] that needs this functionality and hence +this patchset to avoid crashes (the ticket contains more information +why this leads to crashes). Incidentally this branch also fixes +another bug in alot [3]. Patrick spoke up in this thread stating that +this patchset is useful for alot. There is a unpublished branch of +afew that requires this functionality. + +I think it very much boils down to the question whether or not +libnotmuch and its users are second class citizens. This issue is not +a problem for the notmuch binary since it is short lived in nature. + +In fact I feel reminded of [4] when I pointed out problems in the code +that are problematic for libnotmuch users. You - Austin - suggested to +whip up patches instead of raising concerns. That's a valid response, +after all, talk is cheap ;) + +But now that I have a patchset that is rather small (yes, it changes +the API but any program can be adjusted automatically using sed...) +and I kind of thought that it would be accepted more easily. And +although the changes are trivial it took me quite some time to track +it down. + +The thread [4] dried out without someone like you or David stating +that he cares about libnotmuch and its users as much as the users of +the notmuch binary. These kind of problems require invasive code and +API changes, I asked in that very same thread how to do this properly +but noone answered. + +I think I'm just not willing to devote my time to fix these problems +if these issues aren't even perceived as problems by the majority of +notmuch users and developers b/c they are using the emacs interface +which just calls the notmuch binary that has no such problems. + +Cheers, +Justus + +0: b2734519db78fdec76eeafc5fe8f5631a6436cf6 +1: cfc5f1059aa16753cba610c41601cacc97260e08 +2: https://github.com/pazz/alot/issues/413 +3: https://github.com/pazz/alot/issues/414 +4: 20120221002921.8534.57091@thinkbox.jade-hamburg.de