--- /dev/null
+Return-Path: <teythoon@jade-hamburg.de>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 57402431FB6\r
+ for <notmuch@notmuchmail.org>; Thu, 12 Apr 2012 10:20:49 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+ autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id DHZmjrBs27HI for <notmuch@notmuchmail.org>;\r
+ Thu, 12 Apr 2012 10:20:48 -0700 (PDT)\r
+Received: from mail.cryptobitch.de (cryptobitch.de [88.198.7.68])\r
+ (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 66748431FAF\r
+ for <notmuch@notmuchmail.org>; Thu, 12 Apr 2012 10:20:48 -0700 (PDT)\r
+Received: from mail.jade-hamburg.de (unknown [85.183.11.228])\r
+ (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by mail.cryptobitch.de (Postfix) with ESMTPSA id 9DCF356A61B\r
+ for <notmuch@notmuchmail.org>; Thu, 12 Apr 2012 19:20:46 +0200 (CEST)\r
+Received: by mail.jade-hamburg.de (Postfix, from userid 401)\r
+ id 07F4FDF2A3; Thu, 12 Apr 2012 19:20:46 +0200 (CEST)\r
+Received: from thinkbox.jade-hamburg.de (unknown [10.1.1.153])\r
+ (using TLSv1 with cipher AES256-SHA (256/256 bits))\r
+ (No client certificate requested) (Authenticated sender: teythoon)\r
+ by mail.jade-hamburg.de (Postfix) with ESMTPSA id 34EF2DF2A0;\r
+ Thu, 12 Apr 2012 19:20:43 +0200 (CEST)\r
+Received: from teythoon by thinkbox.jade-hamburg.de with local (Exim 4.77)\r
+ (envelope-from <teythoon@thinkbox.jade-hamburg.de>)\r
+ id 1SINgJ-0005ft-4i; Thu, 12 Apr 2012 19:19:39 +0200\r
+Content-Type: text/plain; charset="utf-8"\r
+MIME-Version: 1.0\r
+Content-Transfer-Encoding: quoted-printable\r
+To: Austin Clements <amdragon@MIT.EDU>, \r
+From: Justus Winter <4winter@informatik.uni-hamburg.de>\r
+In-Reply-To: <20120412165744.GF13549@mit.edu>\r
+References:\r
+ <1332291311-28954-1-git-send-email-4winter@informatik.uni-hamburg.de>\r
+ <20120401032323.GH5949@mit.edu>\r
+ <20120412090533.2074.78211@thinkbox.jade-hamburg.de>\r
+ <20120412165744.GF13549@mit.edu>\r
+Message-ID: <20120412171939.5852.22809@thinkbox.jade-hamburg.de>\r
+User-Agent: alot/0.21+\r
+Subject: Re: [RFC] Split notmuch_database_close into two functions\r
+Date: Thu, 12 Apr 2012 19:19:39 +0200\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 12 Apr 2012 17:20:49 -0000\r
+\r
+Quoting Austin Clements (2012-04-12 18:57:44)\r
+>Quoth Justus Winter on Apr 12 at 11:05 am:\r
+>> Quoting Austin Clements (2012-04-01 05:23:23)\r
+>> >Quoth Justus Winter on Mar 21 at 1:55 am:\r
+>> >> I propose to split the function notmuch_database_close into\r
+>> >> notmuch_database_close and notmuch_database_destroy so that long\r
+>> >> running processes like alot can close the database while still using\r
+>> >> data obtained from queries to that database.\r
+>> >\r
+>> >Is this actually safe? My understanding of Xapian::Database::close is\r
+>> >that, once you've closed the database, basically anything can throw a\r
+>> >Xapian exception. A lot of data is retrieved lazily, both by notmuch\r
+>> >and by Xapian, so simply having, say, a notmuch_message_t object isn't\r
+>> >enough to guarantee that you'll be able to get data out of it after\r
+>> >closing the database. Hence, I don't see how this interface could be\r
+>> >used correctly.\r
+>> =\r
+\r
+>> I do not know how, but both alot and afew (and occasionally the\r
+>> notmuch binary) are somehow safely using this interface on my box for\r
+>> the last three weeks.\r
+>\r
+>I see. TL;DR: This isn't safe, but that's okay if we document it.\r
+>\r
+>The bug report [0] you pointed to was quite informative. At its core,\r
+>this is really a memory management issue. To sum up for the record\r
+>(and to check my own thinking): It sounds like alot is careful not to\r
+>use any notmuch objects after closing the database. The problem is\r
+>that, currently, closing the database also talloc_free's it, which\r
+>recursively free's everything derived from it. Python later GCs the\r
+>wrapper objects, which *also* try to free their underlying objects,\r
+>resulting in a double free.\r
+>\r
+>Before the change to expose notmuch_database_close, the Python\r
+>bindings would only talloc_free from destructors. Furthermore, they\r
+>prevented the library from recursively freeing things at other times\r
+>by internally maintaining a reverse reference for every library talloc\r
+>reference (e.g., message is a sub-allocation of query, so the bindings\r
+>keep a reference from each message to its query to ensure the query\r
+>doesn't get freed). The ability to explicitly talloc_free the\r
+>database subverts this mechanism.\r
+\r
+Exactly.\r
+\r
+>So, I've come around to thinking that splitting notmuch_database_close\r
+>and _destroy is okay. It certainly parallels the rest of the API\r
+>better. However, notmuch_database_close needs a big warning similar\r
+>to Xapian::Database::close's warning that retrieving information from\r
+>objects derived from this database may not work after calling close.\r
+\r
+Yes, but then again one should always expect function calls to fail\r
+and most APIs have mechanisms to communicate failures.\r
+\r
+OTOH this might be an indication that the notmuch API should be\r
+redesigned. Both alot and afew have their own wrappers around the\r
+notmuch API to work around some limitations (e.g. changes to messages\r
+are enqueued and executed at some point, with some kind of mechanism\r
+to cope with the notmuch database temporarily not being available,\r
+message objects have to be re-fetched if they got outdated (IIRC,\r
+whatever that means)).\r
+\r
+Justus\r
projects / notmuch-archives.git / commitdiff