--- /dev/null
+Return-Path: <ethan.glasser.camp@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 35E7F431FAF\r
+ for <notmuch@notmuchmail.org>; Thu, 28 Jun 2012 00:39:04 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.798\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.798 tagged_above=-999 required=5\r
+ tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7]\r
+ autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 1MkKDWu5fN7y for <notmuch@notmuchmail.org>;\r
+ Thu, 28 Jun 2012 00:39:02 -0700 (PDT)\r
+Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com\r
+ [209.85.220.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id E9A63431FB6\r
+ for <notmuch@notmuchmail.org>; Thu, 28 Jun 2012 00:39:01 -0700 (PDT)\r
+Received: by vcbf1 with SMTP id f1so1528629vcb.26\r
+ for <notmuch@notmuchmail.org>; Thu, 28 Jun 2012 00:39:00 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+ h=mime-version:in-reply-to:references:date:message-id:subject:from:to\r
+ :cc:content-type;\r
+ bh=mLbRIJZDgr6d1TaGvW5+oCmWHJVufh5MZXpWEh0H90Y=;\r
+ b=B5m+zqwX9HxzFtkqMGI7yjrd4IkOlXZFJYDFlprGcrtFtq6u57eyW5hGKkJmCb4pUV\r
+ J3L4GbBLdePJcRxPFhg3Zr79YEJ2JvsjrwBakgJJTKRtJicRwrmC4WKBVmeruyvvZCH0\r
+ zD8dJ/IwWx/2J5WIoR75kXlag/DOiaALaUHVrbDBmI693/l7lsCgP9xP5nuTOlGLActr\r
+ s5Qncl2LQ7cLBBH7jNW6BYQFW8SCPkodq8MxyhtJXIqvRyN3N3s2yjpxssf6emGdLxIR\r
+ jo6PgtI6we7Nv1CmGx7vKvnzsHKxdGSzF5ICfDJx6VvMODWNpREbr4jxpb0xR/kux2EX\r
+ RNnw==\r
+MIME-Version: 1.0\r
+Received: by 10.52.75.99 with SMTP id b3mr492975vdw.75.1340869140085; Thu, 28\r
+ Jun 2012 00:39:00 -0700 (PDT)\r
+Received: by 10.220.6.3 with HTTP; Thu, 28 Jun 2012 00:39:00 -0700 (PDT)\r
+In-Reply-To: <877gutnmf1.fsf@qmul.ac.uk>\r
+References: <1340656899-5644-1-git-send-email-ethan@betacantrips.com>\r
+ <877gutnmf1.fsf@qmul.ac.uk>\r
+Date: Thu, 28 Jun 2012 03:39:00 -0400\r
+Message-ID:\r
+ <CAOJ+Ob0Kw0Kkhh9C27Xv9gvqtNowzQiNqrLAtvti7fL8NND2+w@mail.gmail.com>\r
+Subject: Re: [RFC PATCH 00/14] modular mail stores based on URIs\r
+From: Ethan <ethan.glasser.camp@gmail.com>\r
+To: Mark Walters <markwalters1009@gmail.com>\r
+Content-Type: multipart/alternative; boundary=bcaec501604bc8fa8004c3836cee\r
+X-Mailman-Approved-At: Thu, 28 Jun 2012 07:51:08 -0700\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 28 Jun 2012 07:39:04 -0000\r
+\r
+--bcaec501604bc8fa8004c3836cee\r
+Content-Type: text/plain; charset=ISO-8859-1\r
+\r
+I sent this at first as a reply-only-to-sender. Oops! Sorry Mark for the\r
+double send.\r
+\r
+On Wed, Jun 27, 2012 at 5:17 AM, Mark Walters <markwalters1009@gmail.com>wrote:\r
+\r
+> > Personally, this isn't my favorite approach, for the following reasons:\r
+> >\r
+> > 1. Notmuch, at some point in its history, chose to store file paths\r
+> > relative to a "mail database", with the intent that if this mail\r
+> > database was moved, filenames would not change and everything would\r
+> > Just Work (tm). The above scheme completely reverses this design\r
+> > decision, and in general completely breaks this relocatability. I\r
+> > don't see any easy way to handle this problem. This isn't just a\r
+> > wishlist feature; at least two things in the test suite (caching of\r
+> > corpus.mail, and the atomicity tests) rely on this behavior.\r
+>\r
+> Why can't the URI just store a relative path, at least for maildir://\r
+> and mbox:// ? It is purely internal to notmuch so it doesn't need to be\r
+> very standard.\r
+>\r
+\r
+Well, relative to where? This is especially relevant now that we can have\r
+multiple mail stores. It sounds like you are suggesting that all mbox://\r
+URIs are relative to an "mbox root", but the fundamental question is how to\r
+pass that information from the configuration into the library.\r
+\r
+Even using configuration itself may be problematic, because only the CLI\r
+uses the configuration, and language bindings like Python and Ruby might\r
+get out of sync! (But note also that the Python bindings currently use\r
+.notmuch-config to find the database path, so maybe it's not a big deal.)\r
+\r
+If I could do whatever I wanted, every mailstore would get registered\r
+somehow and the URIs could use those registered names to specify what\r
+they're relative to: maybe using hostname, such as\r
+maildir://university-mail/some-mail-file, mbox://old-unix-system/some.mbox.\r
+Then changing these names in .notmuch-config would be fine. I just don't\r
+know how to pass that configuration information without an approach like in\r
+the past patch series.\r
+\r
+ > 2. Mail access information, i.e. open connections, etc. can only be\r
+> > stored in variables global to the mailstore code, and cannot be stored\r
+> > as private members of a mailstore object. This is more an aesthetic\r
+> > concern than a functional one.\r
+> >\r
+> > Anyhow, the following (enormous) patch series implement this design. I\r
+> > used uriparser as an external library to parse URIs. The API for this\r
+> > library is a little idiosyncratic. uriparser supports parsing Unicode\r
+> > URIs (strings of wchar_t), but I just used ASCII filenames because I\r
+> > think that's what comes out of Xapian.\r
+>\r
+> Why use a library? Isn't it just a question of does the string contain\r
+> // and, if so, splitting it? I guess that // is a nice separator as I\r
+> think we can assume that a true path does not contain it (since a\r
+> filename cannot contain /).\r
+>\r
+\r
+The URIs are true URIs. Filenames are provided by the "path" segment of the\r
+uri -- everything from the first slash after the hostname up to a ? for\r
+query arguments. My concern was that filenames could (in theory) contain #\r
+or ?, and in practice they contain : (maildir flags). I figured it was\r
+better to do it right.\r
+\r
+ > Patch 11 is borrowed directly from the last patch series.\r
+> >\r
+> > The last four or five patches add mbox support, including a few\r
+> > tests. That part of the series is still very first-draft: I added a\r
+> > new config option to specify URIs to scan, and ">From " lines still\r
+> > need to be unescaped. However, we support scanning mbox files whether\r
+> > messages have content-length or not.\r
+>\r
+> I have an idea that mbox byte-locations change when messages are marked\r
+> as read (amongst other things). It might be worth saying that this\r
+> initial implementation only works for unchanging mboxs (rather than the\r
+> append only condition that you currently say). But I have not got as far\r
+> as applying/testing the series yet.\r
+>\r
+\r
+Yeah, I don't even know how an mbox message gets flagged read and I don't\r
+know how I would support it.\r
+\r
+Ethan\r
+\r
+--bcaec501604bc8fa8004c3836cee\r
+Content-Type: text/html; charset=ISO-8859-1\r
+Content-Transfer-Encoding: quoted-printable\r
+\r
+<div class=3D"gmail_quote">I sent this at first as a reply-only-to-sender. =\r
+Oops! Sorry Mark for the double send.<br><br>On Wed, Jun 27, 2012 at 5:17 A=\r
+M, Mark Walters <span dir=3D"ltr"><<a href=3D"mailto:markwalters1009@gma=\r
+il.com" target=3D"_blank">markwalters1009@gmail.com</a>></span> wrote:<b=\r
+r>\r
+\r
+<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=\r
+x #ccc solid;padding-left:1ex">> Personally, this isn't my favorite =\r
+approach, for the following reasons:<br><div>\r
+><br>\r
+> 1. Notmuch, at some point in its history, chose to store file paths<br=\r
+>\r
+> relative to a "mail database", with the intent that if this =\r
+mail<br>\r
+> database was moved, filenames would not change and everything would<br=\r
+>\r
+> Just Work (tm). The above scheme completely reverses this design<br>\r
+> decision, and in general completely breaks this relocatability. I<br>\r
+> don't see any easy way to handle this problem. This isn't just=\r
+ a<br>\r
+> wishlist feature; at least two things in the test suite (caching of<br=\r
+>\r
+> corpus.mail, and the atomicity tests) rely on this behavior.<br>\r
+<br>\r
+</div>Why can't the URI just store a relative path, at least for maildi=\r
+r://<br>\r
+and mbox:// ? It is purely internal to notmuch so it doesn't need to be=\r
+<br>\r
+very standard.<br></blockquote><div>=A0<br>Well, relative to where? This is=\r
+ especially relevant now that we can\r
+ have multiple mail stores. It sounds like you are suggesting that all=20\r
+mbox:// URIs are relative to an "mbox root", but the fundamental=\r
+=20\r
+question is how to pass that information from the configuration into the\r
+ library.<br>\r
+<br>Even using configuration itself may be problematic, because only the\r
+ CLI uses the configuration, and language bindings like Python and Ruby=20\r
+might get out of sync! (But note also that the Python bindings currently\r
+ use .notmuch-config to find the database path, so maybe it's not a big=\r
+=20\r
+deal.)<br>\r
+<br>If I could do whatever I wanted, every mailstore would get=20\r
+registered somehow and the URIs could use those registered names to=20\r
+specify what they're relative to: maybe using hostname, such as=20\r
+maildir://university-mail/some-mail-file, mbox://old-unix-system/some.mbox.\r
+ Then changing these names in .notmuch-config would be fine. I just=20\r
+don't know how to pass that configuration information without an=20\r
+approach like in the past patch series.<br><div><br></div></div><blockquote=\r
+ class=3D"gmail_quote" style=3D"margin:0pt 0pt 0pt 0.8ex;border-left:1px so=\r
+lid rgb(204,204,204);padding-left:1ex">\r
+<div>\r
+> 2. Mail access information, i.e. open connections, etc. can only be<br=\r
+>\r
+> stored in variables global to the mailstore code, and cannot be stored=\r
+<br>\r
+> as private members of a mailstore object. This is more an aesthetic<br=\r
+>\r
+> concern than a functional one.<br>\r
+><br>\r
+> Anyhow, the following (enormous) patch series implement this design. I=\r
+<br>\r
+> used uriparser as an external library to parse URIs. The API for this<=\r
+br>\r
+> library is a little idiosyncratic. uriparser supports parsing Unicode<=\r
+br>\r
+> URIs (strings of wchar_t), but I just used ASCII filenames because I<b=\r
+r>\r
+> think that's what comes out of Xapian.<br>\r
+<br>\r
+</div>Why use a library? Isn't it just a question of does the string co=\r
+ntain<br>\r
+// and, if so, splitting it? I guess that // is a nice separator as I<br>\r
+think we can assume that a true path does not contain it (since a<br>\r
+filename cannot contain /).<br></blockquote><div><br>The URIs are true URIs=\r
+. Filenames are provided by the "path" segment of=20\r
+the uri -- everything from the first slash after the hostname up to a ?=20\r
+for query arguments. My concern was that filenames could (in theory)=20\r
+contain # or ?, and in practice they contain : (maildir flags). I=20\r
+figured it was better to do it right.<br><br></div><blockquote class=3D"gma=\r
+il_quote" style=3D"margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204,2=\r
+04,204);padding-left:1ex">\r
+<div>\r
+> Patch 11 is borrowed directly from the last patch series.<br>\r
+><br>\r
+> The last four or five patches add mbox support, including a few<br>\r
+> tests. That part of the series is still very first-draft: I added a<br=\r
+>\r
+> new config option to specify URIs to scan, and ">From " l=\r
+ines still<br>\r
+> need to be unescaped. However, we support scanning mbox files whether<=\r
+br>\r
+> messages have content-length or not.<br>\r
+<br>\r
+</div>I have an idea that mbox byte-locations change when messages are mark=\r
+ed<br>\r
+as read (amongst other things). It might be worth saying that this<br>\r
+initial implementation only works for unchanging mboxs (rather than the<br>\r
+append only condition that you currently say). But I have not got as far<br=\r
+>\r
+as applying/testing the series yet.<br></blockquote><div><br>Yeah, I don=\r
+9;t even know how an mbox message gets flagged read and I don't know ho=\r
+w I would support it.<br><br>Ethan<br><span><font color=3D"#888888">\r
+</font></span></div></div><br>\r
+\r
+--bcaec501604bc8fa8004c3836cee--\r
projects / notmuch-archives.git / commitdiff