Return-Path: X-Original-To: notmuch@notmuchmail.org Delivered-To: notmuch@notmuchmail.org Received: from localhost (localhost [127.0.0.1]) by olra.theworths.org (Postfix) with ESMTP id 8A3834196F2 for ; Mon, 12 Apr 2010 06:33:43 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at olra.theworths.org X-Spam-Flag: NO X-Spam-Score: -0.5 X-Spam-Level: X-Spam-Status: No, score=-0.5 tagged_above=-999 required=5 tests=[BAYES_05=-0.5] autolearn=ham Received: from olra.theworths.org ([127.0.0.1]) by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CxwnwRjthB8q for ; Mon, 12 Apr 2010 06:33:42 -0700 (PDT) Received: from homiemail-a21.g.dreamhost.com (caiajhbdcbbj.dreamhost.com [208.97.132.119]) by olra.theworths.org (Postfix) with ESMTP id D230D431FC1 for ; Mon, 12 Apr 2010 06:33:42 -0700 (PDT) Received: from sspaeth.de (mtec-hg-docking-1-dhcp-204.ethz.ch [129.132.133.204]) by homiemail-a21.g.dreamhost.com (Postfix) with ESMTPA id 8068C300074 for ; Mon, 12 Apr 2010 06:33:39 -0700 (PDT) Received: by sspaeth.de (sSMTP sendmail emulation); Mon, 12 Apr 2010 15:33:35 +0200 From: "Sebastian Spaeth" To: "Notmuch list" Subject: please eat my data! Date: Mon, 12 Apr 2010 15:33:35 +0200 Message-ID: <87633wlrrk.fsf@SSpaeth.de> User-Agent: notmuch version 0.1.1 (Emacs 23.1.1/x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Apr 2010 13:33:43 -0000 fsync is really killing xapian (and notmuch). What suffers, are the boolean prefixes (tag, id, and thread). Using libeatmydata (which disables fsync) shows a 10x speedup for tagging. The speedup is only factor 2 for e.g. from: searches. This is ext4 on recent stock Ubuntu. Given that search by tag and thread are performed really often (each time I advance a thread, for example), that really hurts. With a warm file cache and a thread containing 11 messages: --------------------------------------------------- time notmuch tag +test -- thread:0000000000000f4e real 0m0.677s user 0m0.030s sys 0m0.020s --------------------------------------------------- time LD_PRELOAD=./libeatmydata.so notmuch tag +test -- thread:0000000000000f4e real 0m0.040s user 0m0.020s sys 0m0.020s --------------------------------------------------- However tagging ~850 messages based on a from search is "ONLY" factor 2: ------------------------------------------------------ time notmuch tag +test -- from:"sebastian@sspaeth.de" real 0m2.355s user 0m1.240s sys 0m0.040s --------------------------------------------------- time LD_PRELOAD=./libeatmydata.so notmuch tag +test -- from:"sebastian@sspaeth.de" real 0m1.286s user 0m1.230s sys 0m0.010s ---------------------------------------------------