From bae4fc48fa820fd29bb4239f774023c283fad1ff Mon Sep 17 00:00:00 2001 From: Ethan Glasser-Camp Date: Sat, 13 Oct 2012 12:28:36 +2000 Subject: [PATCH] Re: More ideas about logging. --- f9/f7ab06d33929c66177ee8a50c96865dfe45fdf | 91 +++++++++++++++++++++++ 1 file changed, 91 insertions(+) create mode 100644 f9/f7ab06d33929c66177ee8a50c96865dfe45fdf diff --git a/f9/f7ab06d33929c66177ee8a50c96865dfe45fdf b/f9/f7ab06d33929c66177ee8a50c96865dfe45fdf new file mode 100644 index 000000000..d99b152e3 --- /dev/null +++ b/f9/f7ab06d33929c66177ee8a50c96865dfe45fdf @@ -0,0 +1,91 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 7A041431FAF + for ; Fri, 12 Oct 2012 09:28:46 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.799 +X-Spam-Level: +X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id qpvbgY0WlFRd for ; + Fri, 12 Oct 2012 09:28:46 -0700 (PDT) +Received: from mail-qc0-f181.google.com (mail-qc0-f181.google.com + [209.85.216.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 89BA0431FB6 + for ; Fri, 12 Oct 2012 09:28:45 -0700 (PDT) +Received: by mail-qc0-f181.google.com with SMTP id x40so2673074qcp.26 + for ; Fri, 12 Oct 2012 09:28:44 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=from:to:cc:subject:in-reply-to:references:user-agent:date + :message-id:mime-version:content-type; + bh=HX3IhTZ+UaoBgoFPPCeEiI81s707BL3CzA4xEMp9NCg=; + b=l3mdlLPN2I1b21vgrwEKOkT3SQarLbYKJVl5oyamplCwaW05CA2WZmHZNPxbaFgmP+ + oehUDy+Pdvlo26Jy+S5AoLnIzVEX0eL2orCYVfRyTYe++ZvreaU3wXYRO25KekYCuQ9P + wIiNaGxbV0ZDipWJaX2Pv1uA09IIRwfMz0TB12Kvdfxi9sSbklwK547g5ZfvLO5AvlGq + huA1YdCUETiafelySQ0PPb2plLxfyikZE9HD42ERRW22QmbIIJlTjsoTujYM3HLRMQvD + NGyXez14WADmY59N5h9zb5hU9YQ6yegNcp/olcW6jJjKwHcfeqF3aczBEzYABCHhGox1 + Td4A== +Received: by 10.224.60.17 with SMTP id n17mr8242158qah.63.1350059324414; + Fri, 12 Oct 2012 09:28:44 -0700 (PDT) +Received: from smtp.gmail.com (p70-80.acedsl.com. [66.114.70.80]) + by mx.google.com with ESMTPS id jw1sm6781053qeb.13.2012.10.12.09.28.41 + (version=TLSv1/SSLv3 cipher=OTHER); + Fri, 12 Oct 2012 09:28:42 -0700 (PDT) +From: Ethan Glasser-Camp +To: Austin Clements , David Bremner +Subject: Re: More ideas about logging. +In-Reply-To: <20111216040722.GC12245@mit.edu> +References: <87obv9i7y3.fsf@zancas.localnet> <20111216040722.GC12245@mit.edu> +User-Agent: Notmuch/0.14+45~g6ea9330 (http://notmuchmail.org) Emacs/23.3.1 + (x86_64-pc-linux-gnu) +Date: Fri, 12 Oct 2012 12:28:36 -0400 +Message-ID: <87y5jbac3v.fsf@betacantrips.com> +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Cc: Olly Betts , Notmuch Mail +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Fri, 12 Oct 2012 16:28:46 -0000 + +Austin Clements writes: + +> The trouble with this approach is that the OS doesn't have to flush +> logfile to the disk platters in any particular order relative to the +> updates to Xapian. So, after someone trips over your plug, you could +> come back with Xapian saying you have 500 log entries when your +> logfile comes back with only 20. The only way I know of to fix this +> is to fsync after the logfile write, which would obviously have +> performance issues. But maybe there are cleverer ways? + +Sorry to jump in almost a year after the fact, but.. + +How bad do you think those performance issues are going to be? I don't +see them as prohibitive, even in the case where you write a log entry +for every message being tagged. Xapian's doing an fsync each time we +commit, isn't it? (Or is there some cute trick where it rename()s the +database?) + +If, instead of truncating, you replay logged operations, I think you can +get away with just writing log entries for user-level operations (like +"notmuch tag +mytag to:somequery") which could touch a lot of +messages. This would then only require one fsync on the log file before +doing a lot of tag updates. + +Ethan -- 2.26.2