Date: Sun, 25 Nov 2012 13:05:24 -0500
From: Austin Clements <amdragon@MIT.EDU>
To: Tomi Ollila <tomi.ollila@iki.fi>
Subject: Re: [PATCH 3/3] lib: Reject multi-message mboxes and deprecate
	single-message mbox
Message-ID: <20121125180524.GL4562@mit.edu>
References: <1353824161-31717-1-git-send-email-amdragon@mit.edu>
	<1353824161-31717-3-git-send-email-amdragon@mit.edu>
	<m2y5hpu82l.fsf@guru.guru-group.fi>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <m2y5hpu82l.fsf@guru.guru-group.fi>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: notmuch@notmuchmail.org
Precedence: list

Quoth Tomi Ollila on Nov 25 at  3:26 pm:
> On Sun, Nov 25 2012, Austin Clements <amdragon@MIT.EDU> wrote:
> 
> > Previously, we would treat multi-message mboxes as one giant email,
> > which, besides the obvious incorrect indexing, often led to
> > out-of-memory errors for archival mboxes.  Now we explicitly reject
> > multi-message mboxes.  For historical reasons, we retain support for
> > single-message mboxes, but official deprecate this behavior.
> 
> 
> The series looks good to me -- but I don't know about deprecating
> single-message mboxes:
> 
> * If we someday support (read-only?) mbox format, then single-message
>   mboxes are "normal" again.

If notmuch does gain mbox support, then its handling of single-message
mboxes will *definitely* change because it will stop doing
maildir-like things to them (flag sync, moving from new to cur, etc),
which people may currently be depending on.  This was one of the
motivations for deprecating the current handling of single-message
mboxes.

> * Some naīve mb2md scripts could leave the 'From ' -line intact: for
>   example `formail -bz -s head -3 < $MAIL`(*) can be used to demonstrate this

I would call that "buggy", rather than "naīve".  ]:--8)

> * Some people may have large collection of single-file messages starting
>   with 'From ' currently indexed. If those are to be re-indexed later
>   without "single-message mbox" support that is somewhat of a burden to
>   the users (**)

That's why this only deprecates them (with a warning) and doesn't drop
support for them.  The idea is to keep the historical handling for a
few releases and then we'll have the flexibility to do what we want
with single-message mboxes (including supporting them as real mbox).

It's probably a good idea to include a script or a wiki pointer for
fixing single-message mboxes in the NEWS.  As long as the file name is
kept the same, notmuch won't reindex it.

> (*) my "mb2md" wannabe does gnus-like "$formail" -bz -R 'From ' X-From-Line: ...
> 
> (**) Something like the following could be used to mangle "single-file mboxes"...
>      find . -type f | xargs perl -e 'foreach (@ARGV) { open IO, "+<", $_ or
>      next; sysread IO, $buf, 5; if ($buf eq "From ") { sysseek IO, 0, 0;
>      syswrite IO, "Fro:"; }}' 
>      This breaks the multi-message mbox nicely... >;)
> 
> 
> Tomi