--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id B170A431FAF\r
+ for <notmuch@notmuchmail.org>; Thu, 12 Apr 2012 09:57:50 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id Ks05ixbL1F5s for <notmuch@notmuchmail.org>;\r
+ Thu, 12 Apr 2012 09:57:50 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (DMZ-MAILSEC-SCANNER-4.MIT.EDU\r
+ [18.9.25.15])\r
+ by olra.theworths.org (Postfix) with ESMTP id E21AC431FAE\r
+ for <notmuch@notmuchmail.org>; Thu, 12 Apr 2012 09:57:49 -0700 (PDT)\r
+X-AuditID: 1209190f-b7f8a6d000000914-bb-4f87098b16b6\r
+Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
+ by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 12.0B.02324.B89078F4; Thu, 12 Apr 2012 12:57:47 -0400 (EDT)\r
+Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103])\r
+ by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id q3CGvkWV021095; \r
+ Thu, 12 Apr 2012 12:57:47 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id q3CGviOd024327\r
+ (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT);\r
+ Thu, 12 Apr 2012 12:57:45 -0400 (EDT)\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.77)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1SINL6-0004IU-2r; Thu, 12 Apr 2012 12:57:44 -0400\r
+Date: Thu, 12 Apr 2012 12:57:44 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Justus Winter <4winter@informatik.uni-hamburg.de>\r
+Subject: Re: [RFC] Split notmuch_database_close into two functions\r
+Message-ID: <20120412165744.GF13549@mit.edu>\r
+References:\r
+ <1332291311-28954-1-git-send-email-4winter@informatik.uni-hamburg.de>\r
+ <20120401032323.GH5949@mit.edu>\r
+ <20120412090533.2074.78211@thinkbox.jade-hamburg.de>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <20120412090533.2074.78211@thinkbox.jade-hamburg.de>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFmphleLIzCtJLcpLzFFi42IR4hRV1u3mbPc3uDFZ32J26w8mi+s3ZzI7\r
+ MHlMPH+azePZqlvMAUxRXDYpqTmZZalF+nYJXBl/Ln1jLNgpUfH2u1cDY4NIFyMnh4SAicTc\r
+ tl5mCFtM4sK99WxdjFwcQgL7GCVu3XkI5WxglDhyfSY7hHOSSeLezqVMEM4SRonmnd3sIP0s\r
+ AqoSl59tYQWx2QQ0JLbtX84IYosImEpsePAArIZZQFri2+9mJhBbWMBJYs2FSWBxXgEdiWln\r
+ vkINXcUoMW3Tb6iEoMTJmU9YIJq1JG78ewlUxAE2aPk/DpAwp4CjxOOek2B7RQVUJKac3MY2\r
+ gVFoFpLuWUi6ZyF0L2BkXsUom5JbpZubmJlTnJqsW5ycmJeXWqRropebWaKXmlK6iREc2JL8\r
+ Oxi/HVQ6xCjAwajEw/viWZu/EGtiWXFl7iFGSQ4mJVHe1ezt/kJ8SfkplRmJxRnxRaU5qcWH\r
+ GCU4mJVEeP88BirnTUmsrEotyodJSXOwKInzqmm98xMSSE8sSc1OTS1ILYLJynBwKEnwigMj\r
+ WEiwKDU9tSItM6cEIc3EwQkynAdoOC9IDW9xQWJucWY6RP4Uo6KUOO9PDqCEAEgiozQPrheW\r
+ eF4xigO9IswrBNLOA0xacN2vgAYzAQ3mUwC5urgkESEl1cBo8yDVwjU/fb6HVONxhSCHW58N\r
+ P7uaFur1GKhI+3WGv9FRCl8WMX8D4w3eax1d2cuLtz2bphCQGNeY7nnHuDLD8ihT96+Ct73z\r
+ Krg9b/1iOMVhmiK2eOeZRz8LVi399sPnzaNtTZbHLV9+YJgc/y0nI54nTlBp5ZcDbh3njRiv\r
+ fI503pY8/6gSS3FGoqEWc1FxIgAexVgYFwMAAA==\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 12 Apr 2012 16:57:50 -0000\r
+\r
+Quoth Justus Winter on Apr 12 at 11:05 am:\r
+> Quoting Austin Clements (2012-04-01 05:23:23)\r
+> >Quoth Justus Winter on Mar 21 at 1:55 am:\r
+> >> I propose to split the function notmuch_database_close into\r
+> >> notmuch_database_close and notmuch_database_destroy so that long\r
+> >> running processes like alot can close the database while still using\r
+> >> data obtained from queries to that database.\r
+> >\r
+> >Is this actually safe? My understanding of Xapian::Database::close is\r
+> >that, once you've closed the database, basically anything can throw a\r
+> >Xapian exception. A lot of data is retrieved lazily, both by notmuch\r
+> >and by Xapian, so simply having, say, a notmuch_message_t object isn't\r
+> >enough to guarantee that you'll be able to get data out of it after\r
+> >closing the database. Hence, I don't see how this interface could be\r
+> >used correctly.\r
+> \r
+> I do not know how, but both alot and afew (and occasionally the\r
+> notmuch binary) are somehow safely using this interface on my box for\r
+> the last three weeks.\r
+\r
+I see. TL;DR: This isn't safe, but that's okay if we document it.\r
+\r
+The bug report [0] you pointed to was quite informative. At its core,\r
+this is really a memory management issue. To sum up for the record\r
+(and to check my own thinking): It sounds like alot is careful not to\r
+use any notmuch objects after closing the database. The problem is\r
+that, currently, closing the database also talloc_free's it, which\r
+recursively free's everything derived from it. Python later GCs the\r
+wrapper objects, which *also* try to free their underlying objects,\r
+resulting in a double free.\r
+\r
+Before the change to expose notmuch_database_close, the Python\r
+bindings would only talloc_free from destructors. Furthermore, they\r
+prevented the library from recursively freeing things at other times\r
+by internally maintaining a reverse reference for every library talloc\r
+reference (e.g., message is a sub-allocation of query, so the bindings\r
+keep a reference from each message to its query to ensure the query\r
+doesn't get freed). The ability to explicitly talloc_free the\r
+database subverts this mechanism.\r
+\r
+\r
+So, I've come around to thinking that splitting notmuch_database_close\r
+and _destroy is okay. It certainly parallels the rest of the API\r
+better. However, notmuch_database_close needs a big warning similar\r
+to Xapian::Database::close's warning that retrieving information from\r
+objects derived from this database may not work after calling close.\r
+notmuch_database_close is really a specialty interface, and about the\r
+only thing you can guarantee after closing the database is that you\r
+can destroy other objects. This is also going to require a SONAME\r
+major version bump, as mentioned by others. Which, to be fair, would\r
+be a good opportunity to fix some other issues, too, like how\r
+notmuch_database_open can't return errors and how\r
+notmuch_database_get_directory is broken on read-only databases. The\r
+actual bump should be done at release time, but maybe we should drop a\r
+note somewhere (NEWS?) so we don't forget.\r
+\r
+[0] https://github.com/pazz/alot/issues/413\r
projects / notmuch-archives.git / commitdiff