Date: Tue, 24 Nov 2009 16:16:58 +0200
Message-ID: <874ook80th.wl%djcb@djcbsoftware.nl>
From: Dirk-Jan C. Binnema <djcb.bulk@gmail.com>
To: Carl Worth <cworth@cworth.org>
In-Reply-To: <87k4xg634z.fsf@yoom.home.cworth.org>
References: <87aayggsjp.wl%djcb@djcbsoftware.nl>
	<87iqd43wot.fsf@yoom.home.cworth.org>	<87skc6n3yp.wl%djcb@djcbsoftware.nl>
	<877htifa0e.fsf@yoom.home.cworth.org>	<87pr79yaz1.wl%djcb@djcbsoftware.nl>
	<87k4xg634z.fsf@yoom.home.cworth.org>
Mail-Reply-To: djcb@djcbsoftware.nl
User-Agent: Wanderlust/2.15.6 (Almost Unreal) Emacs/23.1 Mule/6.0
	(HANACHIRUSATO)
Organization: DJCBSoftware
MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka")
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: "notmuch@notmuchmail.org" <notmuch@notmuchmail.org>
Subject: Re: [notmuch] interesting project!
Precedence: list
Reply-To: djcb@djcbsoftware.nl

Hi Carl,

>>>>> "Carl" =3D=3D Carl Worth <cworth@cworth.org> writes:

    Carl> I agree that trying to support OOM doesn't make sense without
    Carl> testing. But that's why I want to test notmuch with memory-fault
    Carl> injection. We've been doing this with the cairo library with good
    Carl> success for a while.

    Carl> As for "unlikely that malloc ever returns NULL", that's simply a
    Carl> system-configuration away (just turn off overcommit). And I can i=
magine
    Carl> notmuch being used in lots of places, (netbooks, web servers, etc=
.), so
    Carl> I do want to make it as robust as possible.

That is a very laudable goal! But it's also quite hard to achieve, consider=
ing
that both GMime and Xapian may have some different ideas about that. And at
least in the current code, I see fprintfs in 'malloc-returns-NULL'-cases --
but fprintf itself will probably allocate memory too. Also, at least now, t=
he
bad=C2=A0alloc exceptions for C++ are not caught. Of course, that can be ch=
anged,
but it's just to show that these things are hard to get right.

    Carl> Thanks for mentioning the hash table. The hash table is one of th=
e few
    Carl> things that I *am* using from glib right now in notmuch. It's got=
 a
    Carl> couple of bizarre things about it:

    Carl> 	1. The simpler-appearing g_hash_table_new function is useless
    Carl> 	   for common cases like hashing strings. It will just leak
    Carl> 	   memory. So g_hash_table_new_full is the only one worth using.

Hmmm, I never noticed that behavior. Tf you are using dynamically allocated
strings, GHashTable won't free them for you -- but I can really see how it
could (given that it takes generic pointers), so you have to free those
yourself. But any memleaks beyond that?

    Carl> 	2. There are two lookup functions, g_hash_table_lookup, and
    Carl> 	   g_hash_table_lookup_extended.

    Carl> 	   So, it might make sense if a hash-table interface supported
    Carl> 	   these two modes well. What's bizarre about GHashTable though,
    Carl> 	   is that in the "just a set" case, we only use NULL as the
    Carl> 	   value when inserting. And distinguish "previously inserted
    Carl> 	   with NULL" from "never inserted" is the one thing that
    Carl> 	   g_hash_table_lookup can't do. So I've only found that I could
    Carl> 	   ever use g_hash_table_lookup_extended, (and pass a pair of
    Carl> 	   NULLs for the return arguments I don't need).

Hmmn, well in I found that returning NULL for 'not set' works in many cases,
and it makes it quite easy for that. If you need to distinguish between NULL
and 'not set', you can use either the _extended version as you mention, or =
use
some special NOT_SET static ptr you can compare with (and handle it
appropriately in the destructor).

    Carl> I definitely like the idea of having tiny, focused libraries that=
 do
    Carl> one thing and do it well, (and maybe even some things so tiny that
    Carl> they are actually designed to be copied into the application---li=
ke
    Carl> with gnulib and with Eric's new hash table).

Ok; glib fills the role pretty well for me, and I don't really pay for the
parts that I don't use. But tastes differ, no problem ;-)

    Carl> Thanks for understanding. :-)
    Carl> And I enjoy the conversation,

Same here :)=20

Best wishes,
Dirk.

--=20
Dirk-Jan C. Binnema                  Helsinki, Finland
e:djcb@djcbsoftware.nl           w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C