Re: Questions about importing mail (mbox)

diff --git a/2d/7d0703f5ed3d5883429f927cc65d565a6cff02 b/2d/7d0703f5ed3d5883429f927cc65d565a6cff02
new file mode 100644 (file)
index 0000000..94bd101
--- /dev/null
+++ b/2d/7d0703f5ed3d5883429f927cc65d565a6cff02
@@ -0,0 +1,122 @@
+Return-Path: <pieter@praet.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 11D93431FB6\r
+       for <notmuch@notmuchmail.org>; Mon, 21 Mar 2011 07:31:36 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id Ktu6a8Ihivi0 for <notmuch@notmuchmail.org>;\r
+       Mon, 21 Mar 2011 07:31:35 -0700 (PDT)\r
+Received: from mail-wy0-f181.google.com (mail-wy0-f181.google.com\r
+       [74.125.82.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 33234431FB5\r
+       for <notmuch@notmuchmail.org>; Mon, 21 Mar 2011 07:31:35 -0700 (PDT)\r
+Received: by wyf23 with SMTP id 23so5770386wyf.26\r
+       for <notmuch@notmuchmail.org>; Mon, 21 Mar 2011 07:31:31 -0700 (PDT)\r
+Received: by 10.216.56.65 with SMTP id l43mr5079358wec.113.1300717891612;\r
+       Mon, 21 Mar 2011 07:31:31 -0700 (PDT)\r
+Received: from localhost (203.56-242-81.adsl-dyn.isp.belgacom.be\r
+       [81.242.56.203])\r
+       by mx.google.com with ESMTPS id t11sm2579666wes.41.2011.03.21.07.31.27\r
+       (version=TLSv1/SSLv3 cipher=OTHER);\r
+       Mon, 21 Mar 2011 07:31:28 -0700 (PDT)\r
+From: Pieter Praet <pieter@praet.org>\r
+To: Mueen Nawaz <mueen@nawaz.org>, notmuch@notmuchmail.org\r
+Subject: Re: Questions about importing mail (mbox)\r
+In-Reply-To: <87bp15m9oz.fsf@fester.com>\r
+References: <87bp15m9oz.fsf@fester.com>\r
+User-Agent: Notmuch/0.5-49-g4aeb291 (http://notmuchmail.org) Emacs/23.1.50.1\r
+       (x86_64-pc-linux-gnu)\r
+Date: Mon, 21 Mar 2011 15:31:26 +0100\r
+Message-ID:\r
+ <87zkooo88x.fsf@A7GMS.i-did-not-set--mail-host-address--so-tickle-me>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 21 Mar 2011 14:31:36 -0000\r
+\r
+On Sun, 20 Mar 2011 20:30:52 -0700, Mueen Nawaz <mueen@nawaz.org> wrote:\r
+> \r
+> Hi,\r
+> \r
+> I'm trying to experiment with notmuch. \r
+> \r
+> As I understand it, notmuch does not handle mbox for input. The problem\r
+> is that all my mail is currently in mbox format.\r
+> \r
+> So I first tried converting mbox to maildir using mb2md.\r
+> \r
+> It didn't do a good job. When I subsequently tried importing to notmuch,\r
+> notmuch complained about lots of non-mail files - I confirmed that\r
+> indeed mb2md had botched converting those emails.\r
+> \r
+> So then I tried to convert to mh format using Sylpheed. This seemed to\r
+> go well, but then when importing to notmuch, it complained again for\r
+> about 20 emails, and a manual check confirmed that some messages did not\r
+> get converted properly to mh (they don't show up in Sylpheed).\r
+> \r
+> And then I noticed another discrepancy. mutt shows that I started with\r
+> 44473 messages in mbox. When I imported into Sylpheed, it showed 44482\r
+> messages (no idea where the extra 9 came from). However, notmuch is\r
+> reporting that it processed 44482 files, but that it added 35602\r
+> messages.\r
+> \r
+> Why only 35602 (it complained for only about 20 messages)? A search\r
+> confirmed that some messages that show up in both mutt (in mbox) and\r
+> Sylpheed (in mh format) were not indexed.\r
+> \r
+> So I want to know: When you guys switched to notmuch, how did you ensure\r
+> you did not miss any emails. I really, really, really don't want to lose\r
+> any emails in this process!\r
+> \r
+> Thanks.\r
+> \r
+> _______________________________________________\r
+> notmuch mailing list\r
+> notmuch@notmuchmail.org\r
+> http://notmuchmail.org/mailman/listinfo/notmuch\r
+\r
+\r
+It would've been a no-brainer if you'd been using Maildir all along\r
+(mbox is evil incarnate), but...\r
+\r
+I'd suggest keeping your original mbox file safe in git [1], and\r
+consistently commiting every step of the way, so even if messages were\r
+to get lost in translation, you still have a way to get them back, with\r
+negligible storage overhead (just remember to "git gc --aggressive\r
+--prune=now" when you're finished).\r
+\r
+Compacting the mbox file, i.e. purging all stale messages (sync-mailbox\r
+in mutt?) and diffing to HEAD could then possibly give you an indication\r
+as to the origin of the 9 surplus files.\r
+\r
+For the actual conversion to Maildir (and any type of mail fetching in\r
+general), I'd suggest using FDM [2], you'll never look back.\r
+\r
+Regarding the significant discrepancy between processed and added files\r
+in Notmuch: Could be dupes (e.g. mail to/cc/bcc yourself or mailing\r
+lists, ending up in both Inbox and Sent), which are automatically\r
+suppressed by Notmuch.\r
+\r
+\r
+[1] http://git-scm.com/\r
+[2] http://fdm.sourceforge.net/\r