3a/ccc6b6be9e6aea449bc886ebf2689bace1e76f

   1 Return-Path: <amdragon@mit.edu>\r
   2 X-Original-To: notmuch@notmuchmail.org\r
   3 Delivered-To: notmuch@notmuchmail.org\r
   4 Received: from localhost (localhost [127.0.0.1])\r
   5         by olra.theworths.org (Postfix) with ESMTP id 19D2F431FBD\r
   6         for <notmuch@notmuchmail.org>; Thu, 24 Jul 2014 07:32:26 -0700 (PDT)\r
   7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
   8 X-Spam-Flag: NO\r
   9 X-Spam-Score: -2.3\r
  10 X-Spam-Level: \r
  11 X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
  12         tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
  13 Received: from olra.theworths.org ([127.0.0.1])\r
  14         by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
  15         with ESMTP id heHREkDtyM3p for <notmuch@notmuchmail.org>;\r
  16         Thu, 24 Jul 2014 07:32:22 -0700 (PDT)\r
  17 Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu\r
  18         [18.9.25.12])\r
  19         (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
  20         (No client certificate requested)\r
  21         by olra.theworths.org (Postfix) with ESMTPS id 78DDB431FAF\r
  22         for <notmuch@notmuchmail.org>; Thu, 24 Jul 2014 07:32:22 -0700 (PDT)\r
  23 X-AuditID: 1209190c-f79ef6d000005dd6-01-53d118f5d638\r
  24 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
  25         (using TLS with cipher AES256-SHA (256/256 bits))\r
  26         (Client did not present a certificate)\r
  27         by dmz-mailsec-scanner-1.mit.edu (Symantec Messaging Gateway) with SMTP\r
  28         id 83.9F.24022.5F811D35; Thu, 24 Jul 2014 10:32:21 -0400 (EDT)\r
  29 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
  30         by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s6OEWJfg008537; \r
  31         Thu, 24 Jul 2014 10:32:20 -0400\r
  32 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
  33         (authenticated bits=0)\r
  34         (User authenticated as amdragon@ATHENA.MIT.EDU)\r
  35         by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6OEWGXb032203\r
  36         (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
  37         Thu, 24 Jul 2014 10:32:18 -0400\r
  38 Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
  39         (envelope-from <amdragon@mit.edu>)\r
  40         id 1XAK48-000700-0E; Thu, 24 Jul 2014 10:32:16 -0400\r
  41 Date: Thu, 24 Jul 2014 10:32:14 -0400\r
  42 From: Austin Clements <amdragon@MIT.EDU>\r
  43 To: Dmitry Bogatov <KAction@gnu.org>\r
  44 Subject: Re: Notmuch new speed degradation\r
  45 Message-ID: <20140724143214.GA13893@mit.edu>\r
  46 References: <20140724081916.GA32474@localhost>\r
  47 MIME-Version: 1.0\r
  48 Content-Type: text/plain; charset=us-ascii\r
  49 Content-Disposition: inline\r
  50 In-Reply-To: <20140724081916.GA32474@localhost>\r
  51 User-Agent: Mutt/1.5.21 (2010-09-15)\r
  52 X-Brightmail-Tracker:\r
  53  H4sIAAAAAAAAA+NgFupjleLIzCtJLcpLzFFi42IR4hRV1v0qcTHYoOu4rsX+S19ZLK7fnMns\r
  54         wOTRNs3M49mqW8wBTFFcNimpOZllqUX6dglcGSdmRRfMZqv4c2URewPjM5YuRk4OCQETiacd\r
  55         55ggbDGJC/fWs3UxcnEICcxmktjz+AuUs5FRouPtYhYI5zSTxMeJT6AySxgl/t2/xgrSzyKg\r
  56         KnGjZQLYLDYBDYlt+5czgtgiAioSZ26sZgexmQWkJb79bgarERbQkph45QXYHbwCOhL7zv0E\r
  57         iwsJ6Em8fnCSCSIuKHFy5hMWiF4tiRv/XgLFOcDmLP/HARLmFNCXWPayF2yVKNCqKSe3sU1g\r
  58         FJqFpHsWku5ZCN0LGJlXMcqm5Fbp5iZm5hSnJusWJyfm5aUW6Rrq5WaW6KWmlG5iBAe1JM8O\r
  59         xjcHlQ4xCnAwKvHwduw9HyzEmlhWXJl7iFGSg0lJlHfL2QvBQnxJ+SmVGYnFGfFFpTmpxYcY\r
  60         JTiYlUR4RXkuBgvxpiRWVqUW5cOkpDlYlMR531pbBQsJpCeWpGanphakFsFkZTg4lCR4RYHR\r
  61         KyRYlJqeWpGWmVOCkGbi4AQZzgM03AOkhre4IDG3ODMdIn+KUZdj0f6X3UxCLHn5ealS4rxW\r
  62         IEUCIEUZpXlwc2DJ6BWjONBbwrwfxIGqeICJDG7SK6AlTEBLXiWcB1lSkoiQkmpg7F9e5Hm/\r
  63         S8os4tLZgr8RKzf3/ilZ9/WLYIzwP432ZW83Xrry4tOO/X89TyzaWXRmkWD2R+ZTVpGHPyXO\r
  64         1r3ZUrj1ovRTr89Xd2pl6NcKrJr5KPVrG9+imK78wypXfYVm3NW7aqRlY/e5mrEyZf0qVZlH\r
  65         6/yW3Lr3KejX1jViZ84rNYd/VVpvEKjEUpyRaKjFXFScCACXZLJjIQMAAA==\r
  66 Cc: notmuch@notmuchmail.org\r
  67 X-BeenThere: notmuch@notmuchmail.org\r
  68 X-Mailman-Version: 2.1.13\r
  69 Precedence: list\r
  70 List-Id: "Use and development of the notmuch mail system."\r
  71         <notmuch.notmuchmail.org>\r
  72 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
  73         <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
  74 List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
  75 List-Post: <mailto:notmuch@notmuchmail.org>\r
  76 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
  77 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
  78         <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
  79 X-List-Received-Date: Thu, 24 Jul 2014 14:32:26 -0000\r
  80 \r
  81 Quoth Dmitry Bogatov on Jul 24 at 12:19 pm:\r
  82 > Hello!\r
  83 > \r
  84 > I have ~ 3 000 000 mails. I wanted to index them.\r
  85 > First 1 000 000 took several hours, next 200 000 took several days.\r
  86 > \r
  87 > And now, even with libeatmydata, it takes ~ 4 sec for a file.\r
  88 > \r
  89 > Is it any way I can improve perfomance?\r
  90 > \r
  91 > PS. Please, keep me in CC.\r
  92 \r
  93 Hi Dmitry.  My guess is that's you've exceeded your OS buffer cache\r
  94 size by enough that most B-tree reads are going to disk at least once.\r
  95 How big is your database (du -h $MAIL/.notmuch/xapian) and what does\r
  96 free -h report on that computer?  Also, is this on an SSD or an HDD?\r
  97 \r
  98 You could try running notmuch compact.  That should shrink the\r
  99 database, and, more importantly, pack more into the active page set\r
 100 and, I think, also linearize the database.\r