From c85217069963c4bf851cb9efab5b458e85d6819b Mon Sep 17 00:00:00 2001 From: Jameson Graef Rollins Date: Wed, 20 Jun 2012 11:52:01 +1700 Subject: [PATCH] Re: Notmuch Pick --- df/c64725e1ce181a05a6accfd6d843749381b963 | 185 ++++++++++++++++++++++ 1 file changed, 185 insertions(+) create mode 100644 df/c64725e1ce181a05a6accfd6d843749381b963 diff --git a/df/c64725e1ce181a05a6accfd6d843749381b963 b/df/c64725e1ce181a05a6accfd6d843749381b963 new file mode 100644 index 000000000..38734a1eb --- /dev/null +++ b/df/c64725e1ce181a05a6accfd6d843749381b963 @@ -0,0 +1,185 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 343D4431FB6 + for ; Tue, 19 Jun 2012 11:52:09 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -2.29 +X-Spam-Level: +X-Spam-Status: No, score=-2.29 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_MED=-2.3, T_MIME_NO_TEXT=0.01] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id tZmcVK9aGbCF for ; + Tue, 19 Jun 2012 11:52:08 -0700 (PDT) +Received: from outgoing-mail.its.caltech.edu (outgoing-mail.its.caltech.edu + [131.215.239.19]) + by olra.theworths.org (Postfix) with ESMTP id 75601431FAF + for ; Tue, 19 Jun 2012 11:52:08 -0700 (PDT) +Received: from earth-doxen.imss.caltech.edu (localhost [127.0.0.1]) + by earth-doxen-postvirus (Postfix) with ESMTP id 1A87E66E00FE; + Tue, 19 Jun 2012 11:52:08 -0700 (PDT) +X-Spam-Scanned: at Caltech-IMSS on earth-doxen by amavisd-new +Received: from finestructure.net (gwave-111.ligo.caltech.edu + [131.215.114.111]) (Authenticated sender: jrollins) by earth-doxen-submit + (Postfix) with ESMTP id 7FA9166E008D; Tue, 19 Jun 2012 11:52:04 -0700 (PDT) +Received: by finestructure.net (Postfix, from userid 1000) + id 49D8293D; Tue, 19 Jun 2012 11:52:04 -0700 (PDT) +From: Jameson Graef Rollins +To: Mark Walters , notmuch@notmuchmail.org +Subject: Re: Notmuch Pick +In-Reply-To: <87mx3zok3d.fsf@qmul.ac.uk> +References: <87395ump0d.fsf@qmul.ac.uk> + <87zk80ilt9.fsf@servo.finestructure.net> + <87mx3zok3d.fsf@qmul.ac.uk> +User-Agent: Notmuch/0.13.2+53~g1567997 (http://notmuchmail.org) Emacs/23.4.1 + (x86_64-pc-linux-gnu) +Date: Tue, 19 Jun 2012 11:52:01 -0700 +Message-ID: <87zk7z2kz2.fsf@servo.finestructure.net> +MIME-Version: 1.0 +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha256; protocol="application/pgp-signature" +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 19 Jun 2012 18:52:09 -0000 + +--=-=-= +Content-Transfer-Encoding: quoted-printable + +On Tue, Jun 19 2012, Mark Walters wrote: +> As you say several tests do need fixing: I +> thought I would leave that until people said they were basically happy +> with the change.=20 + +Yeah, but failing tests are basically a non-starter for me. You can +maybe get away with not adding new tests during development, but I'm +unlikely to even try out the patches if a bunch of the existing tests +are failing. + +> I am not sure about sanity tests for pick: how would that work while it +> lives in contrib? Obviously it would need some tests before coming in to +> proper mainline. + +Since you're not actually submitting these patches yet, does it really +need to live in contrib? The goal is for it to ultimately *not* live in +contrib, so I would say just go ahead and make them apply in the main +source. + +An example of a sanity test I'm talking about is simply display the pick +thread for something and test that it looks like expected. Sort of a +zeroth order thing. There are other emacs tests that do the same sort +of thing so it shouldn't be too hard to add. + +>> Would it also be useful to make this same change for the search out, for +>> consistency? I notice the search output now uses newlines between all +>> fields, which should help for asynchronous processing, but it might be +>> nice to put newline separators between the initial and final brackets as +>> well. +> +> Right.=20 + +I would say go ahead and send to the list any patches that come up in +development that are generally useful. Especially if they're small like +this they are likely to applied before this series, so it will be +smaller and easier to read when the times comes. + +>> df97df62b70b884a1cd367360ed6ff7eda0e8af6 +>> cli: add --headers_only option to notmuch-show.c +>> +>> Your comment in this patch is very interesting: +>> +>> This is used by notmuch-pick.el (although not needed) because it giv= +es a +>> speed-up of at least a factor of a two; moreover it reduces the memo= +ry +>> usage in emacs hugely. +>> +>> The only difference between the regular show json output and the +>> --headers-only output, as far as I can tell, is the presence of the +>> content of text/plain parts if they exist in the message. We previously +>> had a discussion about the show output not including any part bodies at +>> all, but we decided that the inclusion of text/plain bodies shouldn't +>> affect anything, so why not include them. If they actually do, then I +>> argue we should just move to having show json not include any body parts +>> at all by default, and just have them be retrieved individually like we +>> do currently for non-text/plain parts. This would make things cleaner, +>> and would get rid of the need to have this extra option, which really +>> doesn't produce a significantly different output. +> +> For one use of pick (displaying the structure of a single thread) this +> is not important (and the asynchronous stuff is irrelevant too). For +> another use of displaying the thread structure of a whole "folder" of +> messages it is important. For example the output of=20 +> +> notmuch show tag:notmuch +> +> is about 70MB on my system, whereas with the --headers-only option this +> drops to about 7MB. Note Emacs uses substantially more than this much +> memory to actually process the JSON, and on a low-powered laptop this +> is enough to cause a swap-storm. +> +> Your suggestion of just not including the text/plain is nice for +> notmuch-pick, and is very simple (a single line change). However, it +> does seem to slow the emacs show mode down noticeably on large threads +> (something like 2s to 4s on a thread with 180 messages) so I worry that +> this change might annoy people. What do you think? + +Threads that long are already a bit of a pain to deal with. But I think +this reveals a weakness in the way we're displaying threads more than +anything. It seems silly to me that we would retrieve all 180 messages +From=20such a thread, and format all of them, only to then hide all but +the one that the reader is seeing. I think this "pick" series +illustrates this nicely. Maybe it would be better to refactor the +current emacs show mode to only retrieve parts for messages that are +being displayed. I'm not sure how difficult that would be, though. + +I would also like to have access to more of the message headers in show +mode. For instance, in emacs I want to access more headers with +notmuch-message-headers than are currently available. Having json +always include all headers might be a bit of a performance hit for some +messages with lots of headers, but it would allow callers access to +headers they don't currently have access to at all. + +So I think it would be nice and clean if the default json output +included all the message headers and the message structure, and then +part contents could be retrieved individually. I imagine show mode +could be restructured such that it could use this efficiently. + +jamie. + +PS. where did the name "pick" come from? It doesn't seem to fit with +the functionality to me. + +--=-=-= +Content-Type: application/pgp-signature + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1.4.12 (GNU/Linux) + +iQIcBAEBCAAGBQJP4MpSAAoJEO00zqvie6q8HWIP/3zuS2RWqZVQyvq7SPhwCtFB +9zmWoYvHtTwmeGqgBCAbfVKZVzczSSVW7JIltDL5hT91IDnBY5VT5vifz0fecBz+ +1U9H0DT5fdU84WB8kzGgvzptHVEbO0QMWodiPRbFjfBk1WovB7EVN1OPfBQE27lw +x7okLDEKg0wAXLdFnXC1swoPCxSYVv2Zr+joJwLBmyLBW85Hm+cEmP/ZrMAVrmBe +zbl9phzD7XbSudU5UE0sg1AjKfcYbZpeqmgKqdXn38e+0JHQCZsWptBfu/8SDz09 +9A6+dYSIVKJ8Gv25JkdI9gIKHgh8exc05DH+8KgDEa+L4srpbHpG3DfvedS1WpBy +n4xQchF1DXcTk89G228pP0EaNLHR6eF82TqJ2vIfe+csTU0VnaxFsTbXS6Skup1F +UvTedyC005el3Ej7quLI43JQa39CktsVT8sEUiqdi9HQpcdmC/PiU/hHdloLjEvS +ym1qy45Vfi/hKmoLKVCERmkMVn+i0lkW76m7FVX1oDdM5LjqIYWRDF2JdhP5/whW +VT2YCDbwdejKMbB906fZk9tMpvi/ui477gCOYMwf056kJ0iftsj/Jxolc08aHTs5 +uyRd+5XWdUjCMdGb6NuuxSpoG1B7k7M5iZmSBaKmyDKR+MMPuGUzHBykbehTU4EF +R3hVLYoSH5if5SkBFJv5 +=5XeH +-----END PGP SIGNATURE----- +--=-=-=-- -- 2.26.2