From ff07019dc54f77bd8af473e7d522d6cc832c71dc Mon Sep 17 00:00:00 2001 From: Stewart Smith Date: Tue, 14 Aug 2012 14:47:23 +1000 Subject: [PATCH] Re: Alternative (raw) message store (i.e. instead of maildir) --- af/1b677fb25de1d4156f5583f1d7369643f09645 | 116 ++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 af/1b677fb25de1d4156f5583f1d7369643f09645 diff --git a/af/1b677fb25de1d4156f5583f1d7369643f09645 b/af/1b677fb25de1d4156f5583f1d7369643f09645 new file mode 100644 index 000000000..bffff9f0e --- /dev/null +++ b/af/1b677fb25de1d4156f5583f1d7369643f09645 @@ -0,0 +1,116 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 9A35F431FB6 + for ; Mon, 13 Aug 2012 21:47:28 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 0.01 +X-Spam-Level: +X-Spam-Status: No, score=0.01 tagged_above=-999 required=5 + tests=[T_MIME_NO_TEXT=0.01] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id rg+WkSJiGIYj for ; + Mon, 13 Aug 2012 21:47:27 -0700 (PDT) +Received: from kaylee.flamingspork.com (kaylee.flamingspork.com + [74.207.245.61]) + by olra.theworths.org (Postfix) with ESMTP id F31F0431FAE + for ; Mon, 13 Aug 2012 21:47:26 -0700 (PDT) +Received: from willster (localhost [127.0.0.1]) + by kaylee.flamingspork.com (Postfix) with ESMTPS id B59386065; + Tue, 14 Aug 2012 04:49:14 +0000 (UTC) +Received: by willster (Postfix, from userid 1000) + id C4A703081A36; Tue, 14 Aug 2012 14:47:23 +1000 (EST) +From: Stewart Smith +To: Vladimir Marek , + Ciprian Dorin Craciun +Subject: Re: Alternative (raw) message store (i.e. instead of maildir) +In-Reply-To: <20120811094635.GY28321@pub.cz.oracle.com> +References: + + <20120811094635.GY28321@pub.cz.oracle.com> +User-Agent: Notmuch/0.13.2+60~g7ecf77d (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Tue, 14 Aug 2012 14:47:23 +1000 +Message-ID: <874no613ms.fsf@flamingspork.com> +MIME-Version: 1.0 +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha1; protocol="application/pgp-signature" +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 14 Aug 2012 04:47:28 -0000 + +--=-=-= +Content-Transfer-Encoding: quoted-printable + +Vladimir Marek writes: +> Hi, +> +> I have objections against maildir too, but I tried to tackle it from +> different perspective. Store the maildir in zip file and use fuse-zip to +> manage it. It works sort of but it has two major disadvantages: + +huh... this is fairly interesting.... one of the downsides of a million +odd files for mail is that filesystem dump and restore takes a *LOT* +longer than if it's just giant files on disk. Combined with afuse (fuse +automounter) this could be a pretty elegant solution to the problem of +storing archival Maildirs. + +One large archival maildir here went from 6.5GB (du -sh on XFS) to a +2.3GB ZIP archive.... that will never, ever change. Think about the +performance difference between creating 560,000 files for backup/restore +versus copying a single 2.3GB file. + +> - fuse zip stores all changes in memory until unmounted +> - fuse zip (and libzip for that matter) creates new temporary file when +> updating archive, which takes considerable time when the archive is +> very big. + +This isn't much of a hastle if you have maildir per time period and +archive off. Maybe if you sync flags it may be... + +> Of course this solution would have some disadvantages too, but for me +> the advantages would win. At the moment I'm not sure if I want to +> continue working on that. Maybe if there would be more interested guys + +I'm *really* tempted to investigate making this work for archived +mail. Of course, the list of mounted file systems could get insane +depending on granularity I guess... + +=2D-=20 +Stewart Smith + +--=-=-= +Content-Type: application/pgp-signature + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1.4.11 (GNU/Linux) + +iQIcBAEBAgAGBQJQKdhbAAoJEP08notdpDyvrmoP/3Fvn1sLGF94eA+UoFa9+MVR +7XDwVcYGFo/w7WqOFRJDT7lIiYSicN5tYZ5MFwmxngVd7gsKG0J9ZcmN3XCBJ/jw +pTR4H/T2nH3dP00dXujvYGZQ5clm/cXfbDLF2TiPIfL6jFgie6x6NFejdfEtfG1H +QiJJbfwhsL1we6nAd8/HpNA+cYBtXw65oz2qWcrpkBQiTgpo9x9RC5yOG0YZjJFY +1V9ubOUnTYgingMoMTfLuBv4ztmKa2uWyXi+ZSGeLIJlXCFhU/Bcdb5mw6OgHrZX +Jz96/rquDCwYgp5s+KAqZJj11qYVQc8eneYZn3We2/6/mW7moD+36rmmaIMp7H3k +nGJBnRl4BHVKEvwlpNeJNbzxyAwSYbveGOOv3tS0Hhax5uyChcbt168RoiGrdLIE ++DV9Byfv2JtOb5OYi2uumKvsPRWFqHIMq1UcGUIV07M0+6uNjdCpV5/bwGEsZbhU +Ewm7j4iyR7TYduWxUtrpydN4BSFJC5ACIoDmyr3Rzn5/vA1bYlBCnQkWOzIdofG7 +vsTR0Sjk9x9wNfxp7f+DSigkRm4zHe/JliKOQgPx9oYkL0nJ6q7mjFp6NRtlkJ4L +P2lIZAsR0jiVE0x1DKOaHTISkS+bIyWKd9NSycDjeFjBheMOPrgDKOyYlbYMWTC6 +plqF8wQrR7D3Bjw82Ptv +=+dDD +-----END PGP SIGNATURE----- +--=-=-=-- -- 2.26.2