Date: Thu, 8 Sep 2011 11:15:57 -0400
From: Austin Clements <amdragon@MIT.EDU>
To: Sebastian Spaeth <Sebastian@sspaeth.de>
Subject: Re: Memory management practices
Message-ID: <20110908151557.GM5688@mit.edu>
References: <8739h1pbaq.fsf@gmail.com> <87pqjprzu2.fsf@gmail.com>
	<20110829183010.GA2605@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f>
	<87liucyn7i.fsf@gmail.com> <87aaag3xaf.fsf@gmail.com>
	<CAH-f9WsfHUm_D-+wB89Lt9Wt=hjwDyywvJTK-0NwmHRg0TUsxQ@mail.gmail.com>
	<CAH-f9WveBfvmv2jOF+C81OeeQJt06g6U0q3J_idHrs60DLw7+g@mail.gmail.com>
	<87zkiff8in.fsf@SSpaeth.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87zkiff8in.fsf@SSpaeth.de>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: Bertram Felgenhauer <bertram.felgenhauer@googlemail.com>,
	Bart Massey <bart@cs.pdx.edu>, notmuch <notmuch@notmuchmail.org>
Precedence: list

Quoth Sebastian Spaeth on Sep 08 at  3:50 pm:
> On Wed, 7 Sep 2011 23:05:19 -0400, Austin Clements <amdragon@mit.edu> wrote:
> > Sorry, I went back and re-read your earlier messages and now I see why
> > your references were the way they were.  I stand by the rest of my
> > previous message though.  I think the technique used in the Python
> > bindings only works because Python's GC happens to finalize in a
> > particular order (though I doubt that's guaranteed, and could easily
> > not be the case if you stray into the realm of its cycle collector).
> > In general, it seems like approach is trying to recreate C-like memory
> > management and is fragile as a result, whereas talloc should, I think,
> > allow bindings to express their runtime's memory management rather
> > naturally.
> 
> Mmmh? Why would the method in python be fragile? Each message object
> holds a reference to its parent query object to keep it alive. Are you
> saying cycle collectors could kill off the query object nonetheless?
> (Assume that I know nothing of GCs which comes close to reality)

In general, a garbage collector can't make any guarantees about
finalization order.  When a collection of objects all become
unreachable simultaneously (for example, the last reference to any
Messages object is dropped, causing the Query object and the Message
object to both become unreachable), the garbage collector *could*
finalize the Query first (causing talloc to free the
notmuch_messages_t) and then the Messages object (causing it to
crash).  There's no guarantee in general because, in the presence of
cycles, there is no meaningful finalization order.

That being said, this approach might be (probably is) fine in Python
because Python has an unusual hybrid garbage collector.  Long ago,
Python had only a reference-count based garbage collector.  It now has
a cycle detector layered on top of that [1], but that only kicks in if
there are reference cycles.  Assuming there aren't cycles in the
objects created by the Python bindings, you should get the
deterministic behavior of the reference counted collector.  This isn't
the case in Haskell, which has a generational collector that makes no
guarantees about finalization order (guarantees it couldn't always
keep).


[1] One day a student came to Moon and said: "I understand how to make
a better garbage collector. We must keep a reference count of the
pointers to each cons."

Moon patiently told the student the following story:

    "One day a student came to Moon and said: 'I understand how to
    make a better garbage collector...