Date: Fri, 16 Dec 2011 13:43:37 -0800
From: Nathan Edson <nedson@nedson.net>
To: Austin Clements <amdragon@MIT.EDU>
Subject: Re: GLib-ERROR Mesage Running notmuch new
Message-ID: <20111216214337.GB29172@newb>
References: <20111216200714.GA25170@newb> <20111216204529.GA19427@mit.edu>
	<20111216211237.GA27860@newb> <20111216211734.GB19427@mit.edu>
	<20111216212622.GA29172@newb> <20111216213815.GD19427@mit.edu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20111216213815.GD19427@mit.edu>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: notmuch@notmuchmail.org
Precedence: list

On 12/16/11, Austin Clements wrote:
> Quoth Nathan Edson on Dec 16 at  1:26 pm:
> > On 12/16/11, Austin Clements wrote:
> > > Quoth Nathan Edson on Dec 16 at  1:12 pm:
> > > > > Since you're now building notmuch from source, could you build it with
> > > > > make clean && make CFLAGS=-g and run it under gdb again to get a
> > > > > backtrace?  I don't understand why strace didn't show it opening any
> > > > > mail, but with debug symbols the backtrace should hopefully tell us
> > > > > that.
> > > > 
> > > > First, thanks for your help!  Perhaps I am making progress...
> > > > 
> > > > Here is what I get:
> > > > 
> > > > yabai@newb:~$ gdb --args notmuch new
> > > > GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2
> > > > Copyright (C) 2010 Free Software Foundation, Inc.
> > > > License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> > > > This is free software: you are free to change and redistribute it.
> > > > There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> > > > and "show warranty" for details.
> > > > This GDB was configured as "i686-linux-gnu".
> > > > For bug reporting instructions, please see:
> > > > <http://www.gnu.org/software/gdb/bugs/>...
> > > > Reading symbols from /usr/local/bin/notmuch...done.
> > > > (gdb) run
> > > > Starting program: /usr/local/bin/notmuch new
> > > > [Thread debugging using libthread_db enabled]
> > > > 
> > > > GLib-ERROR **: /build/buildd/glib2.0-2.28.6/./glib/gmem.c:239: failed to allocate 2147483648 bytes
> > > > aborting...
> > > > 
> > > > Program received signal SIGABRT, Aborted.
> > > > 0xb7fe1424 in __kernel_vsyscall ()
> > > > (gdb) bt
> > > > #0  0xb7fe1424 in __kernel_vsyscall ()
> > > > #1  0xb7ce3e71 in raise () from /lib/i386-linux-gnu/libc.so.6
> > > > #2  0xb7ce734e in abort () from /lib/i386-linux-gnu/libc.so.6
> > > > #3  0xb7e87f27 in g_logv () from /lib/i386-linux-gnu/libglib-2.0.so.0
> > > > #4  0xb7e87f62 in g_log () from /lib/i386-linux-gnu/libglib-2.0.so.0
> > > > #5  0xb7e85c3c in g_realloc () from /lib/i386-linux-gnu/libglib-2.0.so.0
> > > > #6  0xb7e50965 in ?? () from /lib/i386-linux-gnu/libglib-2.0.so.0
> > > > #7  0xb7e50d5d in g_array_append_vals () from /lib/i386-linux-gnu/libglib-2.0.so.0
> > > > #8  0xb7e520c0 in g_byte_array_append () from /lib/i386-linux-gnu/libglib-2.0.so.0
> > > > #9  0xb7f7fc94 in ?? () from /usr/lib/libgmime-2.4.so.2
> > > > #10 0xb7f80fbf in ?? () from /usr/lib/libgmime-2.4.so.2
> > > > #11 0xb7f81761 in ?? () from /usr/lib/libgmime-2.4.so.2
> > > > #12 0xb7f829ea in g_mime_parser_construct_message () from /usr/lib/libgmime-2.4.so.2
> > > > #13 0xb7fc1168 in _notmuch_message_index_file (message=0x13714078, 
> > > >     filename=0x80d0600 "/home/yabai/Mail/old.mail") at lib/index.cc:437
> > > > #14 0xb7fbef9f in notmuch_database_add_message (notmuch=0x806cd40, 
> > > >     filename=0x80d0600 "/home/yabai/Mail/old.mail", message_ret=0xbffff414) at lib/database.cc:1734
> > > 
> > > Ah!  It did show up in the strace.  /home/yabai/Mail/old.mail appears
> > > to be the offending file.  What's in it?
> > 
> > This is my file full of old mail.  I believe it is in mbox format. It contains approximately 40K emails...could the format of this file affect things?  Orginally, this mail was exported from Apple Mail.
> 
> notmuch doesn't support mbox files, but an mbox file looks enough like
> a maildir message that notmuch will *try* to index it (but as a
> single, gigantic message).  This is certainly the cause of the
> failure.  The immediate solution is to move this file out of the tree
> you're indexing with notmuch.
> 
> Still, notmuch should fail more gracefully and you are probably
> hitting a real gmime bug (given that it's trying to allocate
> (unsigned)-1 bytes of memory).  How big is old.mail?

old.mail is about 5gb.  I guess the file type is the problem...forgive me, I thought notmuch supported mbox.