1 Return-Path: <pieter@praet.org>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 3E842431FB6
\r
6 for <notmuch@notmuchmail.org>; Sat, 16 Apr 2011 11:43:33 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id mhcxG++lzjyG for <notmuch@notmuchmail.org>;
\r
16 Sat, 16 Apr 2011 11:43:29 -0700 (PDT)
\r
17 Received: from mail-wy0-f181.google.com (mail-wy0-f181.google.com
\r
18 [74.125.82.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits))
\r
19 (No client certificate requested)
\r
20 by olra.theworths.org (Postfix) with ESMTPS id 71140431FB5
\r
21 for <notmuch@notmuchmail.org>; Sat, 16 Apr 2011 11:43:29 -0700 (PDT)
\r
22 Received: by wyi11 with SMTP id 11so4152797wyi.26
\r
23 for <notmuch@notmuchmail.org>; Sat, 16 Apr 2011 11:43:26 -0700 (PDT)
\r
24 Received: by 10.227.139.149 with SMTP id e21mr3220834wbu.147.1302979406246;
\r
25 Sat, 16 Apr 2011 11:43:26 -0700 (PDT)
\r
26 Received: from localhost (99.28-240-81.adsl-dyn.isp.belgacom.be
\r
27 [81.240.28.99]) by mx.google.com with ESMTPS id
\r
28 bs4sm2301002wbb.1.2011.04.16.11.43.24 (version=TLSv1/SSLv3 cipher=OTHER);
\r
29 Sat, 16 Apr 2011 11:43:25 -0700 (PDT)
\r
30 From: Pieter Praet <pieter@praet.org>
\r
31 To: Mueen Nawaz <mueen@nawaz.org>, notmuch@notmuchmail.org
\r
32 Subject: Re: Questions about importing mail (mbox)
\r
33 In-Reply-To: <87hbavlxoa.fsf@fester.com>
\r
34 References: <87bp15m9oz.fsf@fester.com>
\r
35 <87zkooo88x.fsf@A7GMS.i-did-not-set--mail-host-address--so-tickle-me>
\r
36 <87hbavlxoa.fsf@fester.com>
\r
37 User-Agent: Notmuch/0.5-86-g4875299 (http://notmuchmail.org) Emacs/23.1.50.1
\r
38 (x86_64-pc-linux-gnu)
\r
39 Date: Sat, 16 Apr 2011 20:43:22 +0200
\r
41 <87bp06m3zp.fsf@A7GMS.i-did-not-set--mail-host-address--so-tickle-me>
\r
43 Content-Type: text/plain; charset=us-ascii
\r
44 X-BeenThere: notmuch@notmuchmail.org
\r
45 X-Mailman-Version: 2.1.13
\r
47 List-Id: "Use and development of the notmuch mail system."
\r
48 <notmuch.notmuchmail.org>
\r
49 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
50 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
51 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
52 List-Post: <mailto:notmuch@notmuchmail.org>
\r
53 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
54 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
55 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
56 X-List-Received-Date: Sat, 16 Apr 2011 18:43:33 -0000
\r
58 On Mon, 21 Mar 2011 19:02:45 -0700, Mueen Nawaz <mueen@nawaz.org> wrote:
\r
59 > I think you misunderstood me. A part of me suspects this has something
\r
60 > to do with my not explaining myself, but who's to say?<G>
\r
62 Same here, apparently :D
\r
64 > I'm experimenting with notmuch, and if I can translate everything I
\r
65 > currently do in mutt to notmuch, then I'll just dump mutt. The set of
\r
66 > mboxes I have will remain archived, but for all future incoming email,
\r
67 > I'll switch to MH or MailDir. So I don't actually need to put my old
\r
68 > mboxes under revision control - I just need to save them somewhere.
\r
70 I strongly agree that long term storage choices are a matter of personal
\r
71 opinion, however the intention of my proposition was to simply keep
\r
72 track of what changed in the mbox as a result of the various ops
\r
73 performed, as to gain insight in what gets messed up and where.
\r
75 Non-VCS would be something along the lines of:
\r
76 compact mbox.orig > mbox.comp # (*if* "compact" were a valid command)
\r
77 diff mbox.orig mbox.comp
\r
78 mb2md -s ./mbox.comp -d ./maildir
\r
79 cat ./maildir/new/* >> mbox.conv
\r
80 diff mbox.comp mbox.conv
\r
82 > > For the actual conversion to Maildir (and any type of mail fetching in
\r
83 > > general), I'd suggest using FDM [2], you'll never look back.
\r
85 > Thanks - will take a look.
\r
87 > > Regarding the significant discrepancy between processed and added files
\r
88 > > in Notmuch: Could be dupes (e.g. mail to/cc/bcc yourself or mailing
\r
89 > > lists, ending up in both Inbox and Sent), which are automatically
\r
90 > > suppressed by Notmuch.
\r
92 > It definitely was dupes. I didn't realize that notmuch did not keep
\r
95 > So I wrote a Python script to go through the mboxes and do a count of
\r
96 > only unique messages. Problem? I have over 1000 emails that don't have a
\r
97 > Message-ID header (case invariant search). I could go over why that is,
\r
98 > but suffice it to say that I hate Microsoft.<G>
\r
100 > Once I remove all dupes, I get to within 300-400 of the count that
\r
101 > notmuch provides. The remaining 1000+ emails do contain some dupes, and
\r
102 > I can't find a convenient way to get an accurate count of unique emails
\r
103 > from them, but at least now I'm in the ballpark, and a lot more
\r
106 Sadly, both mb2md and fdm *will* mess things up, since they both split
\r
107 on every single occurence of "^From " [1,2], even if it isn't a
\r
110 Both assume occurences of "^From " in the message body to be already
\r
111 escaped like so: "^>From " [3,4].
\r
113 Even worse, RFC 4155 [5] confirms this to be semi-expected behaviour:
\r
114 >> Many implementations are also known to escape message body lines that
\r
115 >> begin with the character sequence of "From ", so as to prevent
\r
116 >> confusion with overly-liberal parsers that do not search for full
\r
117 >> separator lines. In the common case, a leading Greater-Than symbol
\r
118 >> (0x3E) is used for this purpose (with "From " becoming ">From ").
\r
119 >> However, other implementations are known not to escape such lines
\r
120 >> unless they are immediately preceded by a blank line or if they also
\r
121 >> appear to contain an email address and a timestamp. Other
\r
122 >> implementations are also known to perform secondary escapes against
\r
123 >> these lines if they are already escaped or quoted, while others
\r
124 >> ignore these mechanisms altogether.
\r
126 One way to circumvent this is by making use of the Content-Length header
\r
127 (which is apparently how Mutt does it [6]), but guess what, it suffers
\r
128 the same fate as Message-ID...
\r
130 > Incidentally, one reason I didn't realize dupes were the reason is that
\r
131 > I did a search for a word in one email I had and notmuch did not find
\r
132 > it - so I assumed it had not been indexed. Later on, I realized I had
\r
133 > written a partial word and discovered that notmuch does find it if I
\r
134 > type the full word.
\r
136 > What am I doing wrong? Can't notmuch handle partial word matches? Do I
\r
137 > need to specify an option to get that to work?
\r
139 AFAIK, this depends on how Xapian splits terms, so isn't a Notmuch issue.
\r
140 Globbing helps (sometimes).
\r
142 query: "partia AND from:mueen@nawaz.org"
\r
145 query: "partia* AND from:mueen@nawaz.org"
\r
146 correctly returns this thread.
\r
155 [1] mb2md, line 999 (http://www.linuxkungfu.org/files/scripts/mb2md)
\r
156 [2] fdm, line 461 (http://fdm.cvs.sourceforge.net/viewvc/fdm/fdm/fetch-mbox.c?view=markup)
\r
157 [3] mb2md, line 1342 (http://www.linuxkungfu.org/files/scripts/mb2md)
\r
158 [4] fdm, line 468 (http://fdm.cvs.sourceforge.net/viewvc/fdm/fdm/fetch-mbox.c?view=markup)
\r
159 [5] RFC 4155, section 2, paragraph 5 (http://tools.ietf.org/html/rfc4155)
\r
160 [6] http://www.mail-archive.com/mutt-users@mutt.org/msg21921.html
\r
projects / notmuch-archives.git / blob