From 7d64bff484de3ad073f262174d27917637953d70 Mon Sep 17 00:00:00 2001 From: Matt Armstrong Date: Fri, 5 Aug 2016 10:12:07 +1700 Subject: [PATCH] Re: notmuch.el: controlling what does and doesn't get expanded in searches --- 0e/34a4da85c07557b53dce71c97c18f76674d0f5 | 209 ++++++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 0e/34a4da85c07557b53dce71c97c18f76674d0f5 diff --git a/0e/34a4da85c07557b53dce71c97c18f76674d0f5 b/0e/34a4da85c07557b53dce71c97c18f76674d0f5 new file mode 100644 index 000000000..7815478a6 --- /dev/null +++ b/0e/34a4da85c07557b53dce71c97c18f76674d0f5 @@ -0,0 +1,209 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id 79AA96DE0356 + for ; Thu, 4 Aug 2016 10:12:21 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.819 +X-Spam-Level: +X-Spam-Status: No, score=-0.819 tagged_above=-999 required=5 tests=[AWL=0.012, + DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, + SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id SogO1YgbpPcu for ; + Thu, 4 Aug 2016 10:12:13 -0700 (PDT) +Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com + [209.85.220.54]) + by arlo.cworth.org (Postfix) with ESMTPS id 3B0016DE02B5 + for ; Thu, 4 Aug 2016 10:12:13 -0700 (PDT) +Received: by mail-pa0-f54.google.com with SMTP id iw10so84388112pac.2 + for ; Thu, 04 Aug 2016 10:12:13 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; + s=20120113; + h=from:to:subject:in-reply-to:references:user-agent:date:message-id + :mime-version; bh=+VMXkONMBoQw++SK5PR6GyUCtfE620smFTCaOcz6RHw=; + b=HPfDXr+OS7g3FXntfrki1qd8ii1oYZHsMunOjkwZaZL4h3hFQ3Az3/KUJJDpBGzgks + l5HNi6MX+5N6QVlXQ/ALKdtCgriZcESCfG4KipJzULMRCn/AwN41RnSISqqzPDLM6d6W + ZIxUrpEoKcExdE7ZTVn7ViGOqy/qT3iqSQ7kijw/lAKinRQyhx4MCpQGFGUuET5T/q+N + 81YRyRQOxytTBc0MzUFkEPkIsbiDWXuNR2rntHUPMWR1DIUoRnrqix3eVk/yyW2POJZ6 + pHYsBNmo1ghXTf/MD0KyeqsIXMttYHlAcErsZe1TKaVfoQqnYz9WJuug2A5JSRVwqbLW LQAg== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:from:to:subject:in-reply-to:references + :user-agent:date:message-id:mime-version; + bh=+VMXkONMBoQw++SK5PR6GyUCtfE620smFTCaOcz6RHw=; + b=Cgn8qcXlUroG2+6b6uhnhbWC3IDLDxC3qvclFYjAzwz1uwOtxJCPYJocVQw2uNozo2 + NUupLSP+RdCR5m1uvD9GH0cieyn56L6x0ZfQslR1hZe0foPZmkFSoUEDlWYk32NsVRos + +aFP9ggJYPXwpVaeu1332vsOmDd3UZMocbQVR0RtIt38MWqoIbw+HNfDEzna/L6920x2 + AJ7tt2gFyAWl2fhw7ODL6/wa5XP0QIBOKX2toaZYFno7jyFrGaMZvRYA2wuVv5/wfW1N + ALJzsd2lwueDC+pdTXlsWN+HvloBU3ulDezsK8LMNx+HkR7qtz+B/V4+srTp18QYAD7B + 6MRw== +X-Gm-Message-State: + AEkoouurubDgfCTcrB0Q7RgBWxUtRezZm8IQchuR3odhbXkOeLZYcauxrAvmnwXhQAeNsFnk +X-Received: by 10.66.62.226 with SMTP id b2mr127671998pas.119.1470330732305; + Thu, 04 Aug 2016 10:12:12 -0700 (PDT) +Received: from marmstrong-linux.kir.corp.google.com + ([2620:0:1008:11:9d01:ef49:41c8:1902]) + by smtp.gmail.com with ESMTPSA id sy7sm4708834pac.42.2016.08.04.10.12.09 + (version=TLS1_2 cipher=AES128-SHA bits=128/128); + Thu, 04 Aug 2016 10:12:09 -0700 (PDT) +From: Matt Armstrong +To: Jani Nikula , notmuch@notmuchmail.org +Subject: Re: notmuch.el: controlling what does and doesn't get expanded in + searches +In-Reply-To: <87a8gsv787.fsf@nikula.org> +References: + <87a8gsv787.fsf@nikula.org> +User-Agent: Notmuch/0.22.1+62~g2a7b11b (https://notmuchmail.org) Emacs/24.3.1 + (x86_64-pc-linux-gnu) +Date: Thu, 04 Aug 2016 10:12:07 -0700 +Message-ID: +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 04 Aug 2016 17:12:21 -0000 + +Jani Nikula writes: + +> On Thu, 04 Aug 2016, Matt Armstrong wrote: +>> This question pertains to notmuch built from recent git HEAD, using +>> notmuch.el in show mode (i.e. not tree mode). +>> +>> I sometimes read a thread with a bunch of messages and notmuch.el +>> collapses a bunch of them (even if unread and the search matches tags +>> in every message). I can't figure out the heuristic notmuch is +>> applying here. +> +> The idea is that all the messages matching the query are expanded, the +> others collapsed. Each expanded message must fully match the query. The +> unread or inbox tags are not special in this regard. +> +> I am not saying this is ideal, but this is how it's supposed to +> work. (Indeed I'd personally like to define e.g. saved search specific +> tags or queries to use for deciding which messages to expand.) + +Thank you. + +Yes, I find the query semantics with respect to tags and threads a bit +confusing at times. This is not a problem specific to notmuch, as I +find the same kinds of issues in GMail. Usually the problem occurs at +the semantic border between per-message tags and thread-based +operations. + +I can now explain what I am seeing. In my notmuch-post-hook I tag +messages according to various categories (mailing lists, etc.). Every +time I do this I also tag them with 'filtered'. My +`notmuch-saved-searches` has about 15 sub-queries of the form "tag:inbox +AND tag:". + +I also have a saved search to catch the "unfiltered" stuff: + + tag:inbox AND tag:unfiltered + +So this occurred: + +1) One message was sent to a foo-announce mailing list. This was not + caught by my filters, so it was simply tagged 'inbox'. + +2) All replies were sent to the main "foo" mailing list, which *was* + caught by my filters and tagged 'inbox' and 'foo' and 'filtered'. + +This is bad for two reasons: + +a) If I observed this thread by searching for 'tag:inbox AND tag:foo', + the initial foo-announce message would not be expanded. But, as the + first message in the thread, it is the most important! + +b) If I observed this thread by searching for 'tag:inbox AND not + tag:filtered' (as I do to find all the "uncategorized" stuff in my + inbox), then the foo-announce mail is expanded but none of the + others. + +This problem isn't specific to my 'filtered' tag approach. I think it +generalizes to any approach that attempts to split incoming mail. + +I've been seeing this problem quite frequently because I'm in an +environment where messages are cc:'d to different mailing lists all the +time. It is common for threads to be cc'd to new mailing lists +mid-thread, or for people to pull lists off the cc: list mid-thread +(sometimes into private per-person distribution). + +In this environment, different messages in those threads area expanded +depending on which query I use to find them. This is undesirable +because, generally, I want to read every message I have not yet read in +these threads. + + +>> In particular, pressing SPC does not seem to navigate to the collapsed +>> messages (again, even if they are unread). +> +> SPC and n and p are supposed to navigate expanded messages only. N and P +> navigate all messages (but do not expand by default). Again, the tags +> the messages have do not matter. You can manually expand/collapse +> messages, and that'll affect the navigation. + +Note that SPC also archives, and when it does, it archives the entire +thread, not just the expanded messages (i.e. those that match the +current search). So, this gave rise to the above situation, where I +pressed SPC twice and archived a 40 message thread, with most messages +still unread. + + +>> Worst case: only the first messages is initially expanded and all +>> subsequent are collapsed. I press SPC and the cursor goes to the end +>> of search results. SPC again all the entire thread is archived. +>> +>> This behavior has caused me to accidentally skip messages. First +>> step for me is understanding what is going on so I can fix it. +> +> Yes, let's first check that notmuch behaves as it is expected, and +> then figure out how to improve it. + +I think the semantics make coherent sense for ad-hoc searches. Things +break down for me when considering an inbox processing workflow heavily +based on archiving. + +If I return to a thread after reading the first N messages I need not +see those messages expanded. I have already read them, so I'd prefer +they be collapsed. Not much usually does this for me because I archive +aggressively, but I don't always archive. In these cases I think I'd +prefer expansion instead be based on whether I've read the message +(tag:unread). + +Also, I do want threads consisting entirely of read messages to appear +in my inbox searches, so I do not want to simply add "AND tag:unread" to +all of my `notmuch-saved-searches`. + +Additionally, if messages that don't match the query are pulled into +threads that don't match the query, and are "unread", I'd like to see +them. Such messages are quite likely to provide important context. + +I can think of two ideas: + + i) notmuch could have an "also expand tags" feature, where thread based + results would auto expand matching tags. I would set this to + "unread". + + ii) notmuch could have an "expand query" feature, where thread based + results would use an entirely different query to decide, within a + thread, which messages to expand. I would set this query to + "tag:unread". + +This would be most natural with the current archive semantics in +notmuch.el, which apply tags to entire threads on the assumption that +SPC advances over them in meaningful ways. -- 2.26.2