From: Peter Wang Date: Mon, 26 Nov 2012 12:05:05 +0000 (+1100) Subject: Re: [PATCH v2 05/20] insert: copy stdin to Maildir tmp file X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=6a3ac24226b4702fcb004d6d6ec1e314b5a6e7e2;p=notmuch-archives.git Re: [PATCH v2 05/20] insert: copy stdin to Maildir tmp file --- diff --git a/51/be72b432008463cb17e55d340280693fd24a4c b/51/be72b432008463cb17e55d340280693fd24a4c new file mode 100644 index 000000000..7713a4da9 --- /dev/null +++ b/51/be72b432008463cb17e55d340280693fd24a4c @@ -0,0 +1,114 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 57BED431FAF + for ; Mon, 26 Nov 2012 04:05:15 -0800 (PST) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.799 +X-Spam-Level: +X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id Nris0RKcXYEW for ; + Mon, 26 Nov 2012 04:05:13 -0800 (PST) +Received: from mail-da0-f53.google.com (mail-da0-f53.google.com + [209.85.210.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 8FF6D431FAE + for ; Mon, 26 Nov 2012 04:05:13 -0800 (PST) +Received: by mail-da0-f53.google.com with SMTP id x6so2879236dac.26 + for ; Mon, 26 Nov 2012 04:05:11 -0800 (PST) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=date:message-id:from:to:cc:subject:in-reply-to:references + :mime-version:content-type:content-disposition + :content-transfer-encoding; + bh=WCtQ2ic/6gZmX8BLR4JP6EFuEHW5EAzDGumtAb1J9Jg=; + b=mG567wbtdaC3fa3VBr2UOTiDiCuKlkRff2Lp+amUo/Ay7TeiGYPsZJPswUgUtD2f7N + bRYBuoOgpX5Qxy81Hd3Hqv++/AtIhh80vhjtFlnKBmmrdm+fJPLfoozp9I39nxknp/Na + DYM8SBZXIo8vDl3yB/GSg3Nsh3ualWzdsa40zr0ZxfAGy+4/wQcrnos6H3zh338lWFSY + AmWsCVyBohVEQyek2xUGk9otvZw4CMh7UgwoWO5KTMl95UaM4896/iGZe2bhMez4Iqqr + Kq3Gum4RPST/e/qQ+hAUONHE1PEieVVDuw3eDvpdZXD4CRJ29FbxdF4g6SQiK0BaXOuQ + wTOQ== +Received: by 10.66.87.42 with SMTP id u10mr25517846paz.62.1353931511628; + Mon, 26 Nov 2012 04:05:11 -0800 (PST) +Received: from localhost (215.42.233.220.static.exetel.com.au. + [220.233.42.215]) + by mx.google.com with ESMTPS id vk5sm8686958pbc.34.2012.11.26.04.05.08 + (version=TLSv1/SSLv3 cipher=OTHER); + Mon, 26 Nov 2012 04:05:10 -0800 (PST) +Date: Mon, 26 Nov 2012 23:05:05 +1100 +Message-ID: <20121126230505.GA2095@hili.localdomain> +From: Peter Wang +To: Tomi Ollila +Subject: Re: [PATCH v2 05/20] insert: copy stdin to Maildir tmp file +In-Reply-To: +References: <1353806206-29133-1-git-send-email-novalazy@gmail.com> + <1353806206-29133-6-git-send-email-novalazy@gmail.com> + +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Disposition: inline +Content-Transfer-Encoding: 8bit +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Mon, 26 Nov 2012 12:05:15 -0000 + +On Mon, 26 Nov 2012 11:39:41 +0200, Tomi Ollila wrote: +> On Sun, Nov 25 2012, Peter Wang wrote: +> +> > Read the new message from standard input into the Maildir tmp file. +> > --- +> +> There are a few issues that gort my attention in this particular function: +> +> +> > notmuch-insert.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++---- +> > 1 file changed, 47 insertions(+), 4 deletions(-) +> > +> > diff --git a/notmuch-insert.c b/notmuch-insert.c +> > index 371fb47..88e8533 100644 +> > --- a/notmuch-insert.c +> > +++ b/notmuch-insert.c +> > @@ -94,6 +94,47 @@ maildir_open_tmp_file (void *ctx, const char *dir, +> > return fd; +> > } +> > +> > +/* Copy the contents of fdin into fdout. */ +> > +static notmuch_bool_t +> > +copy_fd_data (int fdin, int fdout) +> > +{ +> > + char buf[4096]; +> +> Copying in 4k blocks is slow when at least when doing file to file copy. +> Also socket buffers can often hold much more data. When reading from +> network and saving to file (in low-load machine) this is OK, but otherwise +> something like 64k buffer works better(*). +> +> (*) Now that I said it I have to measure this yet another time ;) + +I get up to a whopping 10 ms faster copy of a 100 MiB file by increasing +the buffer size to 8192 bytes or more (standalone program, stdin to +disk, no fsync, no xapian). Feel free to fiddle with it after it's +pushed, if you think it's worthwhile. + +> You're claiming in function name & and its description that this is more +> "generic" copy function -- yet error message speaks about 'standard input'. + +I'll rename it. + +Peter