--- /dev/null
+Return-Path: <ciprian.craciun@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id EC31A431FB6\r
+ for <notmuch@notmuchmail.org>; Mon, 13 Aug 2012 12:10:27 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+ tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id NcIPQxOKR3Ca for <notmuch@notmuchmail.org>;\r
+ Mon, 13 Aug 2012 12:10:27 -0700 (PDT)\r
+Received: from mail-wi0-f173.google.com (mail-wi0-f173.google.com\r
+ [209.85.212.173]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 28777431FAE\r
+ for <notmuch@notmuchmail.org>; Mon, 13 Aug 2012 12:10:27 -0700 (PDT)\r
+Received: by wibhm6 with SMTP id hm6so3111131wib.2\r
+ for <notmuch@notmuchmail.org>; Mon, 13 Aug 2012 12:10:24 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+ h=mime-version:in-reply-to:references:date:message-id:subject:from:to\r
+ :cc:content-type;\r
+ bh=SSygKeDfqbu75WG6bTpEw2gqCIcpqdIMP+Tog7Pdric=;\r
+ b=vOEUmf1SKypFFdW6tbJnLLoI5qQaHxSuPZ9Id4fuFr9C5G5t6osPOuUoEd+TRi773z\r
+ 9EeuEmyiXIJCFi7Q2ucUu3BTFL/5kRpfJfKFHf5Kimu/tnJ751EKwX3SbTLj8IIown0f\r
+ n01hmVV+F6e0ffweBJ5lSaRSQu75hiz4Pn5+fhKv4XG348Ei3qViwG5S9RB+Lojtgify\r
+ rhV68V33/WZEXsgVxwe+R77IO1eqTDUjcmrFBgrkegYhquA2WL3butJjhu5mJXfjZ698\r
+ xhbNpEhjRbKZ9/rGWneSSBAaCiVlLWGkQ6+5VafxjaIDmrjcgFr+p05ZJDZ9gFqVbk0J\r
+ iuQw==\r
+MIME-Version: 1.0\r
+Received: by 10.216.195.143 with SMTP id p15mr6906724wen.167.1344885024096;\r
+ Mon, 13 Aug 2012 12:10:24 -0700 (PDT)\r
+Received: by 10.180.104.196 with HTTP; Mon, 13 Aug 2012 12:10:23 -0700 (PDT)\r
+In-Reply-To: <87mx21rw4h.fsf@servo.finestructure.net>\r
+References:\r
+ <CA+Tk8fwq2thNeKHgfG-EX0hgR7uyqrSce0ZMOhEJBsz1RVtRqg@mail.gmail.com>\r
+ <20120811094635.GY28321@pub.cz.oracle.com>\r
+ <CA+Tk8fzRp0DFfTnLaN3Biq_vEcBGzLg-=-af64Rb5DSM6yw25g@mail.gmail.com>\r
+ <87mx21rw4h.fsf@servo.finestructure.net>\r
+Date: Mon, 13 Aug 2012 22:10:23 +0300\r
+Message-ID:\r
+ <CA+Tk8fySXQN8Qv6QuLM-cJ4E+W9WBkws1B7RWa46C-_NKJeOGQ@mail.gmail.com>\r
+Subject: Re: Alternative (raw) message store (i.e. instead of maildir)\r
+From: Ciprian Dorin Craciun <ciprian.craciun@gmail.com>\r
+To: Jameson Graef Rollins <jrollins@finestructure.net>\r
+Content-Type: text/plain; charset=UTF-8\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 13 Aug 2012 19:10:28 -0000\r
+\r
+On Sat, Aug 11, 2012 at 11:50 PM, Jameson Graef Rollins\r
+<jrollins@finestructure.net> wrote:\r
+> On Sat, Aug 11 2012, Ciprian Dorin Craciun <ciprian.craciun@gmail.com> wrote:\r
+>> My problem with it is that it doesn't scale... And I don't mean\r
+>> this in a theoretical sense, I mean it in the concrete one: I have\r
+>> about 661k emails... And a single `notmuch sync` takes a few tens of\r
+>> seconds...\r
+>\r
+> Hey, Ciprian. That sounds really slow, which makes me wonder if there\r
+> are other things going on here.\r
+> I have 155k messages, but notmuch new\r
+> takes a fraction of a second for me. This initial indexing certainly\r
+> takes a long time (hours potentially), but additions after that should\r
+> be really fast. What version of notmuch are you using? What version of\r
+> xapian?\r
+\r
+\r
+ Don't think there is anything wrong here... Its just drags with\r
+the file system...\r
+\r
+ So just to give a complete info:\r
+ * hardware: Core i5, 8GiB RAM (7.5GiB of which is the FS cache),\r
+SSD (about 175MiB raw disk access);\r
+ * `notmuch --version`: 0.13 (built from sources on latest ArchLinux);\r
+ * `notmuch count`: 701820;\r
+ * `notmuch new` (after adding 5925 new emails, at touching others):\r
+~~~~\r
+Processed 7017 total files in 3m 19s (35 files/sec.).\r
+Added 6061 new messages to the database. Detected 1116 file renames.\r
+~~~~\r
+ * actually the entire thing took almost 5 minutes, but the first\r
+two it didn't display anything just acesing the disk;\r
+ * `notmuch new` (another go, but this time I've `time`-d it):\r
+~~~~\r
+No new mail.\r
+real 0m40.546s\r
+user 0m4.523s\r
+sys 0m17.506s\r
+~~~~\r
+ * `notmuch new` (yet another go, no change):\r
+~~~~\r
+No new mail.\r
+real 0m39.190s\r
+user 0m4.229s\r
+sys 0m17.697s\r
+~~~~\r
+ * just to `du` the maildir (there are also 40k other files in\r
+other maildirs not included in this count):\r
+~~~~\r
+8.7G ......\r
+real 0m22.229s\r
+user 0m1.023s\r
+sys 0m7.890s\r
+~~~~\r
+ * on `new` no hooks are run;\r
+ * the file system in cause is JFS;\r
+\r
+\r
+ As such I doubt the problem is with notmuch itself, and I guess\r
+it's the file system interaction...\r
+\r
+ Now I know I have a really obscure corner case, and I'm positively\r
+amazed on how good notmuch handles this situation. I just wandered if\r
+I could have fixed my problem by moving to an embedded DB, thus\r
+skipping all that syscall overhead...\r
+\r
+ Ciprian.\r
projects / notmuch-archives.git / commitdiff