--- /dev/null
+Return-Path: <marmstrong@google.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 79AA96DE0356\r
+ for <notmuch@notmuchmail.org>; Thu, 4 Aug 2016 10:12:21 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.819\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.819 tagged_above=-999 required=5 tests=[AWL=0.012,\r
+ DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01,\r
+ SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id SogO1YgbpPcu for <notmuch@notmuchmail.org>;\r
+ Thu, 4 Aug 2016 10:12:13 -0700 (PDT)\r
+Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com\r
+ [209.85.220.54])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 3B0016DE02B5\r
+ for <notmuch@notmuchmail.org>; Thu, 4 Aug 2016 10:12:13 -0700 (PDT)\r
+Received: by mail-pa0-f54.google.com with SMTP id iw10so84388112pac.2\r
+ for <notmuch@notmuchmail.org>; Thu, 04 Aug 2016 10:12:13 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;\r
+ s=20120113;\r
+ h=from:to:subject:in-reply-to:references:user-agent:date:message-id\r
+ :mime-version; bh=+VMXkONMBoQw++SK5PR6GyUCtfE620smFTCaOcz6RHw=;\r
+ b=HPfDXr+OS7g3FXntfrki1qd8ii1oYZHsMunOjkwZaZL4h3hFQ3Az3/KUJJDpBGzgks\r
+ l5HNi6MX+5N6QVlXQ/ALKdtCgriZcESCfG4KipJzULMRCn/AwN41RnSISqqzPDLM6d6W\r
+ ZIxUrpEoKcExdE7ZTVn7ViGOqy/qT3iqSQ7kijw/lAKinRQyhx4MCpQGFGUuET5T/q+N\r
+ 81YRyRQOxytTBc0MzUFkEPkIsbiDWXuNR2rntHUPMWR1DIUoRnrqix3eVk/yyW2POJZ6\r
+ pHYsBNmo1ghXTf/MD0KyeqsIXMttYHlAcErsZe1TKaVfoQqnYz9WJuug2A5JSRVwqbLW LQAg==\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:from:to:subject:in-reply-to:references\r
+ :user-agent:date:message-id:mime-version;\r
+ bh=+VMXkONMBoQw++SK5PR6GyUCtfE620smFTCaOcz6RHw=;\r
+ b=Cgn8qcXlUroG2+6b6uhnhbWC3IDLDxC3qvclFYjAzwz1uwOtxJCPYJocVQw2uNozo2\r
+ NUupLSP+RdCR5m1uvD9GH0cieyn56L6x0ZfQslR1hZe0foPZmkFSoUEDlWYk32NsVRos\r
+ +aFP9ggJYPXwpVaeu1332vsOmDd3UZMocbQVR0RtIt38MWqoIbw+HNfDEzna/L6920x2\r
+ AJ7tt2gFyAWl2fhw7ODL6/wa5XP0QIBOKX2toaZYFno7jyFrGaMZvRYA2wuVv5/wfW1N\r
+ ALJzsd2lwueDC+pdTXlsWN+HvloBU3ulDezsK8LMNx+HkR7qtz+B/V4+srTp18QYAD7B\r
+ 6MRw==\r
+X-Gm-Message-State:\r
+ AEkoouurubDgfCTcrB0Q7RgBWxUtRezZm8IQchuR3odhbXkOeLZYcauxrAvmnwXhQAeNsFnk\r
+X-Received: by 10.66.62.226 with SMTP id b2mr127671998pas.119.1470330732305;\r
+ Thu, 04 Aug 2016 10:12:12 -0700 (PDT)\r
+Received: from marmstrong-linux.kir.corp.google.com\r
+ ([2620:0:1008:11:9d01:ef49:41c8:1902])\r
+ by smtp.gmail.com with ESMTPSA id sy7sm4708834pac.42.2016.08.04.10.12.09\r
+ (version=TLS1_2 cipher=AES128-SHA bits=128/128);\r
+ Thu, 04 Aug 2016 10:12:09 -0700 (PDT)\r
+From: Matt Armstrong <marmstrong@google.com>\r
+To: Jani Nikula <jani@nikula.org>, notmuch@notmuchmail.org\r
+Subject: Re: notmuch.el: controlling what does and doesn't get expanded in\r
+ searches\r
+In-Reply-To: <87a8gsv787.fsf@nikula.org>\r
+References: <qf54m70o7h5.fsf@marmstrong-linux.kir.corp.google.com>\r
+ <87a8gsv787.fsf@nikula.org>\r
+User-Agent: Notmuch/0.22.1+62~g2a7b11b (https://notmuchmail.org) Emacs/24.3.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Thu, 04 Aug 2016 10:12:07 -0700\r
+Message-ID: <qf57fbw4fx4.fsf@marmstrong-linux.kir.corp.google.com>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 04 Aug 2016 17:12:21 -0000\r
+\r
+Jani Nikula <jani@nikula.org> writes:\r
+\r
+> On Thu, 04 Aug 2016, Matt Armstrong <marmstrong@google.com> wrote:\r
+>> This question pertains to notmuch built from recent git HEAD, using\r
+>> notmuch.el in show mode (i.e. not tree mode).\r
+>>\r
+>> I sometimes read a thread with a bunch of messages and notmuch.el\r
+>> collapses a bunch of them (even if unread and the search matches tags\r
+>> in every message). I can't figure out the heuristic notmuch is\r
+>> applying here.\r
+>\r
+> The idea is that all the messages matching the query are expanded, the\r
+> others collapsed. Each expanded message must fully match the query. The\r
+> unread or inbox tags are not special in this regard.\r
+>\r
+> I am not saying this is ideal, but this is how it's supposed to\r
+> work. (Indeed I'd personally like to define e.g. saved search specific\r
+> tags or queries to use for deciding which messages to expand.)\r
+\r
+Thank you.\r
+\r
+Yes, I find the query semantics with respect to tags and threads a bit\r
+confusing at times. This is not a problem specific to notmuch, as I\r
+find the same kinds of issues in GMail. Usually the problem occurs at\r
+the semantic border between per-message tags and thread-based\r
+operations.\r
+\r
+I can now explain what I am seeing. In my notmuch-post-hook I tag\r
+messages according to various categories (mailing lists, etc.). Every\r
+time I do this I also tag them with 'filtered'. My\r
+`notmuch-saved-searches` has about 15 sub-queries of the form "tag:inbox\r
+AND tag:<something>".\r
+\r
+I also have a saved search to catch the "unfiltered" stuff:\r
+\r
+ tag:inbox AND tag:unfiltered\r
+\r
+So this occurred:\r
+\r
+1) One message was sent to a foo-announce mailing list. This was not\r
+ caught by my filters, so it was simply tagged 'inbox'.\r
+\r
+2) All replies were sent to the main "foo" mailing list, which *was*\r
+ caught by my filters and tagged 'inbox' and 'foo' and 'filtered'.\r
+\r
+This is bad for two reasons:\r
+\r
+a) If I observed this thread by searching for 'tag:inbox AND tag:foo',\r
+ the initial foo-announce message would not be expanded. But, as the\r
+ first message in the thread, it is the most important!\r
+\r
+b) If I observed this thread by searching for 'tag:inbox AND not\r
+ tag:filtered' (as I do to find all the "uncategorized" stuff in my\r
+ inbox), then the foo-announce mail is expanded but none of the\r
+ others.\r
+\r
+This problem isn't specific to my 'filtered' tag approach. I think it\r
+generalizes to any approach that attempts to split incoming mail.\r
+\r
+I've been seeing this problem quite frequently because I'm in an\r
+environment where messages are cc:'d to different mailing lists all the\r
+time. It is common for threads to be cc'd to new mailing lists\r
+mid-thread, or for people to pull lists off the cc: list mid-thread\r
+(sometimes into private per-person distribution).\r
+\r
+In this environment, different messages in those threads area expanded\r
+depending on which query I use to find them. This is undesirable\r
+because, generally, I want to read every message I have not yet read in\r
+these threads.\r
+\r
+\r
+>> In particular, pressing SPC does not seem to navigate to the collapsed\r
+>> messages (again, even if they are unread).\r
+>\r
+> SPC and n and p are supposed to navigate expanded messages only. N and P\r
+> navigate all messages (but do not expand by default). Again, the tags\r
+> the messages have do not matter. You can manually expand/collapse\r
+> messages, and that'll affect the navigation.\r
+\r
+Note that SPC also archives, and when it does, it archives the entire\r
+thread, not just the expanded messages (i.e. those that match the\r
+current search). So, this gave rise to the above situation, where I\r
+pressed SPC twice and archived a 40 message thread, with most messages\r
+still unread.\r
+\r
+\r
+>> Worst case: only the first messages is initially expanded and all\r
+>> subsequent are collapsed. I press SPC and the cursor goes to the end\r
+>> of search results. SPC again all the entire thread is archived.\r
+>>\r
+>> This behavior has caused me to accidentally skip messages. First\r
+>> step for me is understanding what is going on so I can fix it.\r
+>\r
+> Yes, let's first check that notmuch behaves as it is expected, and\r
+> then figure out how to improve it.\r
+\r
+I think the semantics make coherent sense for ad-hoc searches. Things\r
+break down for me when considering an inbox processing workflow heavily\r
+based on archiving.\r
+\r
+If I return to a thread after reading the first N messages I need not\r
+see those messages expanded. I have already read them, so I'd prefer\r
+they be collapsed. Not much usually does this for me because I archive\r
+aggressively, but I don't always archive. In these cases I think I'd\r
+prefer expansion instead be based on whether I've read the message\r
+(tag:unread).\r
+\r
+Also, I do want threads consisting entirely of read messages to appear\r
+in my inbox searches, so I do not want to simply add "AND tag:unread" to\r
+all of my `notmuch-saved-searches`.\r
+\r
+Additionally, if messages that don't match the query are pulled into\r
+threads that don't match the query, and are "unread", I'd like to see\r
+them. Such messages are quite likely to provide important context.\r
+\r
+I can think of two ideas:\r
+\r
+ i) notmuch could have an "also expand tags" feature, where thread based\r
+ results would auto expand matching tags. I would set this to\r
+ "unread".\r
+\r
+ ii) notmuch could have an "expand query" feature, where thread based\r
+ results would use an entirely different query to decide, within a\r
+ thread, which messages to expand. I would set this query to\r
+ "tag:unread".\r
+\r
+This would be most natural with the current archive semantics in\r
+notmuch.el, which apply tags to entire threads on the assumption that\r
+SPC advances over them in meaningful ways.\r
projects / notmuch-archives.git / commitdiff