Date: Fri, 9 Sep 2011 13:53:28 -0400
From: Austin Clements <amdragon@MIT.EDU>
To: Sebastian Spaeth <Sebastian@sspaeth.de>
Subject: Re: Memory management practices
Message-ID: <20110909175328.GV5688@mit.edu>
References: <8739h1pbaq.fsf@gmail.com> <87pqjprzu2.fsf@gmail.com>
	<20110829183010.GA2605@24f89f8c-e6a1-4e75-85ee-bb8a3743bb9f>
	<87liucyn7i.fsf@gmail.com> <87aaag3xaf.fsf@gmail.com>
	<CAH-f9WsfHUm_D-+wB89Lt9Wt=hjwDyywvJTK-0NwmHRg0TUsxQ@mail.gmail.com>
	<CAH-f9WveBfvmv2jOF+C81OeeQJt06g6U0q3J_idHrs60DLw7+g@mail.gmail.com>
	<87zkiff8in.fsf@SSpaeth.de> <20110908151557.GM5688@mit.edu>
	<8762l22hgk.fsf@SSpaeth.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8762l22hgk.fsf@SSpaeth.de>
User-Agent: Mutt/1.5.20 (2009-06-14)
Cc: Bertram Felgenhauer <bertram.felgenhauer@googlemail.com>,
	Bart Massey <bart@cs.pdx.edu>, notmuch <notmuch@notmuchmail.org>
Precedence: list

Quoth Sebastian Spaeth on Sep 09 at 11:27 am:
> On Thu, 8 Sep 2011 11:15:57 -0400, Austin Clements <amdragon@MIT.EDU> wrote:
> > In general, a garbage collector can't make any guarantees about
> > finalization order.  When a collection of objects all become
> > unreachable simultaneously (for example, the last reference to any
> > Messages object is dropped, causing the Query object and the Message
> > object to both become unreachable), the garbage collector *could*
> > finalize the Query first (causing talloc to free the
> > notmuch_messages_t) and then the Messages object (causing it to
> > crash).  There's no guarantee in general because, in the presence of
> > cycles, there is no meaningful finalization order.
> 
> Right, but that should not pose a problem for python. If e.g. both a
> Query and derived Message objects become unreachable, the python objects
> would not care which object is ditched and deleted first. Currently, it
> seems that we finalize the Messages first, and the Query second. But we
> would not fail if the Query were finalized first. Granted, the
> underlying libnotmuch Message objects were torn away while the python
> Message objects were still around. But they would ultimately also be
> sweeped away, and that would not cause any problems.
> 
> But I am sure that I am missing out something. I'll leave this
> discussion to the pros :-).

Ah, the *Python* objects don't care, but the underlying C objects do.
Suppose the Query were finalized first.  Python calls Query.__del__,
which calls notmuch_query_destroy, which releases the underlying
talloc references to the C notmuch_messages_t objects, causing talloc
to free the notmuch_messages_t.  Messages._msgs now points to freed
memory, so when Python then finalizes the Messages object,
Messages.__del__ will pass this dangling pointer to
notmuch_messages_destroy, which will crash.

Hence my suggestion that, rather than trying to emulate C-style memory
management in bindings, bindings should create an additional talloc
reference to the underlying objects and rather than calling
notmuch_*_destroy during finalization, they should simply unlink this
additional reference.  Any remaining library-created references will
keep the object alive as long as it's still needed by the library.
Then there's also no need to replicate the library's reference
structure in the bindings (though there is a danger of needlessly
delaying free's when the library creates convenience references like
the one from notmuch_query_t to notmuch_messages_t; for these I'd
recommend that the bindings undo such references, which requires a
little knowledge of the library's reference structure, but nothing
beyond what should be documented).