--- /dev/null
+Return-Path: <bgamari.foss@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 3D5FF429E30\r
+ for <notmuch@notmuchmail.org>; Mon, 29 Aug 2011 13:31:03 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+ tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id jD7c1Y5o4M5X for <notmuch@notmuchmail.org>;\r
+ Mon, 29 Aug 2011 13:31:02 -0700 (PDT)\r
+Received: from mail-qy0-f181.google.com (mail-qy0-f181.google.com\r
+ [209.85.216.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 6830B429E28\r
+ for <notmuch@notmuchmail.org>; Mon, 29 Aug 2011 13:31:02 -0700 (PDT)\r
+Received: by qyk34 with SMTP id 34so4441737qyk.5\r
+ for <notmuch@notmuchmail.org>; Mon, 29 Aug 2011 13:31:00 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;\r
+ h=from:to:cc:subject:in-reply-to:references:user-agent:date\r
+ :message-id:mime-version:content-type;\r
+ bh=m7q3XiWrjcGlIlZFvmzrWptslZmVJwa3jX5wUOVoPNc=;\r
+ b=pO8wxAnzakFaL4CYEyMn3bAi5JxmycqgFnAlGC4cnJ1q3vuP+OENhPr9JM4C0cnzbA\r
+ NAmtvnfYOc74dJ5DWyxWCEw2KVwC2Wq83hmKG84TDGrVctngeSO28y+Q3qHphYu9cZ0d\r
+ kN3g2fQO2LOUAVa88dvjD1Guox5SC0WKrrK60=\r
+Received: by 10.229.34.205 with SMTP id m13mr578282qcd.138.1314649860035;\r
+ Mon, 29 Aug 2011 13:31:00 -0700 (PDT)\r
+Received: from localhost (gamari.physics.umass.edu [128.119.56.223])\r
+ by mx.google.com with ESMTPS id o4sm3642286qct.13.2011.08.29.13.30.58\r
+ (version=TLSv1/SSLv3 cipher=OTHER);\r
+ Mon, 29 Aug 2011 13:30:58 -0700 (PDT)\r
+From: Ben Gamari <bgamari.foss@gmail.com>\r
+To: notmuch <notmuch@notmuchmail.org>\r
+Subject: Memory management practices\r
+In-Reply-To: <20110829183010.GA2605@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f>\r
+References: <8739h1pbaq.fsf@gmail.com> <87pqjprzu2.fsf@gmail.com>\r
+ <20110829183010.GA2605@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f>\r
+User-Agent: Notmuch/0.7-37-g5c3c7f6 (http://notmuchmail.org) Emacs/23.2.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Mon, 29 Aug 2011 16:30:57 -0400\r
+Message-ID: <87liucyn7i.fsf@gmail.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Cc: Bertram Felgenhauer <bertram.felgenhauer@googlemail.com>,\r
+ Bart Massey <bart@cs.pdx.edu>\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 29 Aug 2011 20:31:03 -0000\r
+\r
+Hey all,\r
+\r
+Over the last few weeks I've been trying to fix some brokeness in the\r
+notmuch-haskell bindings with respect to memory management.\r
+\r
+In discussion included below, I describe the issue as I approached it.\r
+In short, it appears that GHC's garbage collector is quite liberal with\r
+the order in which it frees resources (which is apparently permitted by\r
+Haskell's FFI specification), allowing, for instance, a Messages object to be\r
+freed before a Query object despite my attempts to hold the proper references in\r
+the Haskell wrapper objects to keep the Query reachable.\r
+\r
+In general, it seems to me that memory management in notmuch bindings is\r
+a little bit harder than it needs to me due to the decision not to\r
+talloc_ref parent objects when a new child object is created. This means\r
+that a bindings author needs to recreate the ownership tree in their\r
+binding, a task which is fairly easily done (except in the case of\r
+Haskell due to the weak GC finalization guarantees) but seems quite\r
+unnecessary. Is there a reason this decision was made? Would a patch be\r
+accepted adding talloc_ref'ing parents in those functions creating\r
+children and talloc_frees in *_destroys?\r
+\r
+Cheers,\r
+\r
+- Ben\r
+\r
+\r
+\r
+On Mon, 29 Aug 2011 20:30:10 +0200, Bertram Felgenhauer <bertram.felgenhauer@googlemail.com> wrote:\r
+> Dear Ben,\r
+> \r
+> Ben Gamari wrote:\r
+> > After looking into this issue in a bit more depth, I'm even more\r
+> > confused. In fact, I would not be surprised if I have stumbled into a\r
+> > bug in the GC.\r
+> [...]\r
+> > MessagesMessage\r
+> > | \r
+> > | msmpp\r
+> > \/\r
+> > QueryMessages\r
+> > |\r
+> > | qmpp\r
+> > \/\r
+> > Query\r
+> > \r
+> > As we can see, each MessagesMessage object in the Messages list\r
+> > resulting from queryMessages holds a reference to the Query object from\r
+> > which it originated. For this reason, I fail to see how it is possible\r
+> > that the RTS would attempt to free the Query before freeing the\r
+> > MessagesPtr.\r
+> \r
+> When a garbage collection is performed, the RTS determines which heap\r
+> objects are still reachable. The rest is then freed _simultaneously_,\r
+> and the corresponding finalizers are run in some random order.\r
+> \r
+> So assuming the application holds a reference to the MessagesMessage\r
+> object for a while and then drops it, the GC will detect unreachability\r
+> of all the three objects at the same time and in the end, the finalizer\r
+> for MessagesMessage may be run before that of Query.\r
+> \r
+> So I think this is not a bug.\r
+> \r
+> To solve this problem properly, libnotmuch should stop imposing order\r
+> constraints on when objects are freed - this would mean tracking\r
+> references using talloc_ref and talloc_unlink instead of\r
+> talloc_free inside the library.\r
+> \r
+> For a bindings author who does not want to touch the library, the best\r
+> idea I have is to add a layer with the sole purpose of tracking those\r
+> implicit references.\r
+> \r
+> Best regards,\r
+> \r
+> Bertram\r
+> \r
+> _______________________________________________\r
+> Glasgow-haskell-users mailing list\r
+> Glasgow-haskell-users@haskell.org\r
+> http://www.haskell.org/mailman/listinfo/glasgow-haskell-users\r
projects / notmuch-archives.git / commitdiff