From eb44ecb3cf288280e857672b81f20b926af6ad2f Mon Sep 17 00:00:00 2001 From: Franz Fellner Date: Wed, 24 Jul 2013 07:39:48 +0200 Subject: [PATCH] Re: Performance-issues with large threads --- 7f/64a3f0abd28060893bfbc9991ffb2d2fd81b8a | 102 ++++++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 7f/64a3f0abd28060893bfbc9991ffb2d2fd81b8a diff --git a/7f/64a3f0abd28060893bfbc9991ffb2d2fd81b8a b/7f/64a3f0abd28060893bfbc9991ffb2d2fd81b8a new file mode 100644 index 000000000..7f83f973c --- /dev/null +++ b/7f/64a3f0abd28060893bfbc9991ffb2d2fd81b8a @@ -0,0 +1,102 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id E2829431FBC + for ; Tue, 23 Jul 2013 22:38:10 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.799 +X-Spam-Level: +X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id qbX-Ry1zQc4I for ; + Tue, 23 Jul 2013 22:38:06 -0700 (PDT) +Received: from mail-bk0-f47.google.com (mail-bk0-f47.google.com + [209.85.214.47]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id D03CB431FB6 + for ; Tue, 23 Jul 2013 22:38:05 -0700 (PDT) +Received: by mail-bk0-f47.google.com with SMTP id jg1so3262375bkc.20 + for ; Tue, 23 Jul 2013 22:38:03 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=from:to:subject:date:user-agent:mime-version:message-id:in-reply-to + :references:content-type:content-transfer-encoding; + bh=nyB11t58/jEX24xX0GDUGsKyy5YpDHWhbF7uJXVp8MM=; + b=Vz9gGtyT/k4/6W/MSffnerZLi6v7Ro2tGJLcNGA+hl/g5h3tm1/ii8h4TeNYAEsDJh + t8OQPjPvPKi2GsEHLEMlmuQ/IMuPpW3x1TbvRALbzQY5JtBU/JLZlG3VRnj4MF+MQf3O + ELUKDerOWcjPx6izM1XJfVJ6s2a7bIVebA7FiLLdau6Qtpo3cFOkUCAWmJu3Y18QygPa + 6bzeJmNxAIKqM344jopAE8aA3w55+fD6nPWknSucct68WTVmBX7qYyqbrjphRTllkvys + 3v7dYWXTtaR4uLoB4nNvMrkiir4p8nRhHv6OcRER4u9H5crrfOW76E/LdkWIvQY6yE6z + wbdQ== +X-Received: by 10.204.225.73 with SMTP id ir9mr5069096bkb.45.1374644281886; + Tue, 23 Jul 2013 22:38:01 -0700 (PDT) +Received: from localhost (p4FD0C5B1.dip0.t-ipconnect.de. [79.208.197.177]) + by mx.google.com with ESMTPSA id pc8sm4281328bkb.10.2013.07.23.22.38.00 + for + (version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128); + Tue, 23 Jul 2013 22:38:01 -0700 (PDT) +From: Franz Fellner +To: +Subject: Re: Performance-issues with large threads +Date: Wed, 24 Jul 2013 07:39:48 +0200 +User-Agent: trojita/v0.3.93-100-gbbf4022-dirty; Qt/4.8.4; X11; Linux; +MIME-Version: 1.0 +Message-ID: <418182bd-9a04-4e1d-8392-ce16d64681c0@gmail.com> +In-Reply-To: <87wqoh77zh.fsf@qmul.ac.uk> +References: <027633b9-200d-4174-a8b5-72daff923b93@gmail.com> + <87wqoh77zh.fsf@qmul.ac.uk> +Content-Type: text/plain; charset=utf-8; format=flowed +Content-Transfer-Encoding: quoted-printable +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Wed, 24 Jul 2013 05:38:11 -0000 + +Hej,=0A=0A> How long does =0A>=0A> notmuch show --format=3Djson thread:= +thread-id > /dev/null=0A>=0A> take? Is it much faster after the first = +run? What about the same with=0A> the added cli option --body=3Dfalse?=0A= +=0A[7:23] $ time notmuch show --format=3Djson thread:00000000000002d1 >= +|/dev/null=0Anotmuch show --format=3Djson thread:00000000000002d1 >| /d= +ev/null 0,33s user 0,09s system 8% cpu 4,571 total=0A[7:25] $ time not= +much show --format=3Djson thread:00000000000002d1 >|/dev/null=0Anotmuch= + show --format=3Djson thread:00000000000002d1 >| /dev/null 0,12s user = +0,02s system 99% cpu 0,139 total=0A[7:25] $ time notmuch show --format=3D= +json --body=3Dfalse thread:00000000000002d1 >|/dev/null=0Anotmuch show = +--format=3Djson --body=3Dfalse thread:00000000000002d1 >| /dev/null 0,= +30s user 0,08s system 8% cpu 4,446 total=0A[7:25] $ time notmuch show -= +-format=3Djson --body=3Dfalse thread:00000000000002d1 >|/dev/null=0Anot= +much show --format=3Djson --body=3Dfalse thread:00000000000002d1 >| /de= +v/null 0,10s user 0,01s system 99% cpu 0,119 total=0A[7:26] $=0A=0ASo = +fetching without cache takes 4.5 secs, with only 0.1 -> The issue could= + be in parsing the result inside notmuch-vim/alot/...=0A=0A> Do the mes= +sages have html parts?=0AYes, most of them do (unless you reply by mail= + and only use plaintext).=0A=0A> Also what disk are you using (and does= + it have any other significant=0A> activity happening)? And what versio= +n of notmuch are you using?=0AI use a HDD with 5400 RPM. The most inter= +esting activity is ~80-100% CPU usage during loading, all the 13 secs -= + quite a battery killer ;)=0AI regularly build from latest git master (= +due to the vim-client).=0A=0A>=0A> Best wishes=0A>=0A> Mark=0A>=0A>=0A>= +=0A>=0A>=0A>=0A> Franz Fellner writes:=0A>=0A= +>> Hi,=0A>> =0A>> "Large" in the context of this issue is >20 replies, = +I have to =0A>> wait ~1 sec until the thread shows up. With larger =0A>= +> conversations it get's worse (a bug report on bugs.kde.org with =0A>>= + ~400 replies takes ~13 secs. to load).=0A>> This is due to the fact, t= +hat notmuch-{vim,emacs} load the =0A>> complete mail including the body= +.=0A>> ner does a great job here, it loads all headers but only the =0A= +>> body of the first unread mail, resulting in instant loading =0A>> ev= +en of gigantic conversations. It would be terrific to see =0A>> somethi= +ng like that, at least in my favored vim-client ;)=0A>> =0A>> Thx ...=0A= +>=0A>=0A -- 2.26.2