--- /dev/null
+Return-Path: <mh@glandium.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id A45C1431FBC\r
+ for <notmuch@notmuchmail.org>; Fri, 8 Jan 2010 21:51:26 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id DbYt+p2Rk+Lz for <notmuch@notmuchmail.org>;\r
+ Fri, 8 Jan 2010 21:51:26 -0800 (PST)\r
+Received: from vuizook.err.no (vuizook.err.no [85.19.221.46])\r
+ by olra.theworths.org (Postfix) with ESMTP id A0F88431FAE\r
+ for <notmuch@notmuchmail.org>; Fri, 8 Jan 2010 21:51:25 -0800 (PST)\r
+Received: from cha92-13-88-165-248-19.fbx.proxad.net ([88.165.248.19]\r
+ helo=jigen)\r
+ by vuizook.err.no with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)\r
+ (Exim 4.69) (envelope-from <mh@glandium.org>) id 1NTUEL-0001y1-94\r
+ for notmuch@notmuchmail.org; Sat, 09 Jan 2010 06:51:24 +0100\r
+Received: from mh by jigen with local (Exim 4.71) (envelope-from <mh@jigen>)\r
+ id 1NTUEK-0000x0-7b\r
+ for notmuch@notmuchmail.org; Sat, 09 Jan 2010 06:51:20 +0100\r
+Date: Sat, 9 Jan 2010 06:51:20 +0100\r
+From: Mike Hommey <mh+notmuch@glandium.org>\r
+To: notmuch@notmuchmail.org\r
+Message-ID: <20100109055120.GA3109@glandium.org>\r
+References: <874oo7hex2.fsf@yoom.home.cworth.org>\r
+ <87y6lewqtw.fsf@convex-new.cs.unb.ca>\r
+ <87638i75sz.fsf@home.veldthuis.com> <1260227209-sup-184@riseup.net>\r
+ <874oo22blf.fsf@yoom.home.cworth.org>\r
+ <20100108025620.GB28357@lapse.rw.madduck.net>\r
+ <20100108080636.GA26839@glandium.org>\r
+ <20100108090317.GB735@lapse.rw.madduck.net>\r
+ <20100108092019.GA6671@glandium.org>\r
+ <20100108102631.GB11257@lapse.rw.madduck.net>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <20100108102631.GB11257@lapse.rw.madduck.net>\r
+X-GPG-Fingerprint: A479 A824 265C B2A5 FC54 8D1E DE4B DA2C 54FD 2A58\r
+User-Agent: Mutt/1.5.20 (2009-06-14)\r
+Subject: Re: [notmuch] Quick thoughts on a notmuch daemon\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 09 Jan 2010 05:51:26 -0000\r
+\r
+On Fri, Jan 08, 2010 at 11:26:31PM +1300, martin f krafft wrote:\r
+> also sprach Mike Hommey <mh+notmuch@glandium.org> [2010.01.08.2220 +1300]:\r
+> > FYI, I have a good experience writing fuse filesystems, both with\r
+> > high-level and low-level APIs. I'd avise to use the low-level API,\r
+> > which allows for better performance.\r
+> \r
+> I don't have any experience with FUSE yet, but the examples in\r
+> /usr/share/doc/libfuse-dev/examples/ look trivial. This is where\r
+> I would start, one function at a time. If you have a better\r
+> suggestion, I'd love to hear it; or to clone your repo! ;)\r
+\r
+As I said above, there are 2 sets of APIs in FUSE.\r
+\r
+The high-level API sends the full path for the file being accessed for\r
+every system call. And except for specific cases such as read(), write()\r
+or readdir() you have nothing else to identify the file you are referring\r
+to, which means you have to parse the path, and find the proper file\r
+accordingly.\r
+In notmuch case, that would mean doing a search for most system calls.\r
+Try to imagine how many syscalls that are not read(), write() or\r
+readdir() mutt does when opening a Maildir.\r
+\r
+The low-level API, otoh, uses inode numbers extensively (again, except\r
+for read, write and readdir). The lookup call is responsible for resolving\r
+the paths, given an inode and a name. Its results are cached by the kernel.\r
+So, for example reading foo/bar from your fuse mount point will lookup\r
+foo in the inode 1 (FUSE_ROOT_ID) and then do another lookup for bar in\r
+the first result.\r
+One of the problems with this API is that the inode number type is\r
+unsigned long, which means you can't necessarily map real inode numbers,\r
+which can be 64 bits. And even if it could, afaik, there is no quick way\r
+to get a file from its inode, sadly.\r
+\r
+All in all, in the high-level API case, that means we would need lookups\r
+caching badly, and in the low-level API case, some fast way to map on\r
+one hand virtual directories with inodes numbers, and on the other hand,\r
+real files with inode numbers.\r
+\r
+Some quick thoughts, about the whole thing:\r
+- We will need to be careful about deduplication: if you copy a file\r
+ from one directory to another, you don't want to have the copy in the\r
+ underlying Maildir. But as you won't know until the file is totally\r
+ written and closed...\r
+- We should probably allow extra files to be stored in the virtual\r
+ Maildir (for example, courierimap stores stuff in a Maildir)\r
+- We may not need a client program at all, the "search directories"\r
+ configuration could be handled via extended file attributes.\r
+\r
+I also had another not quite unrelated idea a while ago, that could have\r
+its value here: a generic data store, very much like the git object\r
+database (an idea would be to have the git object datastore be a special\r
+case of this generic data store, for possibly interesting compatibility),\r
+which would allow for better storage of the messages: if the maildir is\r
+exposed via fuse, why would you need a raw maildir for ? It would also\r
+allow easier deduplication of messages that are different but not quite:\r
+- Mailing list replies you get both directly and from the mailing\r
+ list software, their headers have differences, but the files are mostly\r
+ equivalent\r
+- Mail quotes are found in both the original message and its response.\r
+\r
+Mike\r
projects / notmuch-archives.git / commitdiff