From: Austin Clements Date: Mon, 5 Aug 2013 18:59:47 +0000 (+2000) Subject: Re: unread message appear `folded' X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=713884af04c0ee8d47831316fab4a67048c789e8;p=notmuch-archives.git Re: unread message appear `folded' --- diff --git a/38/9109bf5c7e006f071278e56c56b8ce5c383305 b/38/9109bf5c7e006f071278e56c56b8ce5c383305 new file mode 100644 index 000000000..2a733aa0f --- /dev/null +++ b/38/9109bf5c7e006f071278e56c56b8ce5c383305 @@ -0,0 +1,155 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 6A9BE431FB6 + for ; Mon, 5 Aug 2013 12:00:03 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id zt7tlyITOaMD for ; + Mon, 5 Aug 2013 11:59:57 -0700 (PDT) +Received: from dmz-mailsec-scanner-5.mit.edu (dmz-mailsec-scanner-5.mit.edu + [18.7.68.34]) + by olra.theworths.org (Postfix) with ESMTP id EF31C431FAE + for ; Mon, 5 Aug 2013 11:59:56 -0700 (PDT) +X-AuditID: 12074422-b7ef78e000000935-5a-51fff62a96fd +Received: from mailhub-auth-4.mit.edu ( [18.7.62.39]) + by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP + id 16.8C.02357.A26FFF15; Mon, 5 Aug 2013 14:59:54 -0400 (EDT) +Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) + by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id r75IxrMW006141; + Mon, 5 Aug 2013 14:59:54 -0400 +Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91]) + (authenticated bits=0) + (User authenticated as amdragon@ATHENA.MIT.EDU) + by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r75Ixosb022754 + (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); + Mon, 5 Aug 2013 14:59:52 -0400 +Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80) + (envelope-from ) + id 1V6Q0S-0003q6-Sa; Mon, 05 Aug 2013 14:59:49 -0400 +Date: Mon, 5 Aug 2013 14:59:47 -0400 +From: Austin Clements +To: Ramakrishnan Muthukrishnan +Subject: Re: unread message appear `folded' +Message-ID: <20130805185947.GD7794@mit.edu> +References: + + +MIME-Version: 1.0 +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +In-Reply-To: + +User-Agent: Mutt/1.5.21 (2010-09-15) +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFmpkleLIzCtJLcpLzFFi42IRYrdT19X69j/QYHqrhcX1mzOZLdZ+TXJg + 8ni26hazx6I96QFMUVw2Kak5mWWpRfp2CVwZTU1TmQoeSlW8nLWDqYGxXbSLkZNDQsBE4tmE + G4wQtpjEhXvr2UBsIYF9jBJfzvt2MXIB2RsYJV7f388G4Zxikrja8JYFwlnCKPF/8z0gh4OD + RUBF4m+XMEg3m4CGxLb9y8GmiggYSVxrucIKYjMLSEt8+93MBGILC2hLXGpeAFbDC2SfW3Gc + CWLmckaJeRtWMEMkBCVOznzCAtGsJXHj30smkF0gg5b/4wAJcwpESkzf9x2sXBTohCknt7FN + YBSahaR7FpLuWQjdCxiZVzHKpuRW6eYmZuYUpybrFicn5uWlFuma6uVmluilppRuYgSHtIvS + DsafB5UOMQpwMCrx8CZc/R8oxJpYVlyZe4hRkoNJSZR3/xegEF9SfkplRmJxRnxRaU5q8SFG + CQ5mJRHe+VuAcrwpiZVVqUX5MClpDhYlcd5nT88GCgmkJ5akZqemFqQWwWRlODiUJHh5vwI1 + ChalpqdWpGXmlCCkmTg4QYbzAA1nBanhLS5IzC3OTIfIn2JUlBLn/QtykQBIIqM0D64XlnJe + MYoDvSLM+xmkigeYruC6XwENZgIabPLzL8jgkkSElFQDo++CUMGY029+bWZPYb44M+XaqxRb + 9V0FzXu+n8p0/pTtxd5RujtmabnyhX8HWe5u/z4t80DGv48VczT4le/1cnTs0/5+ninAhOVv + ePWP8P3KfX8vqBmU7/qx8pxqt/lExdwcnSB17jt3Nv12jfdPDtIomrVw6nk982Vn5h+fub9O + 7tUGvouqS5VYijMSDbWYi4oTAbViH3IUAwAA +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Mon, 05 Aug 2013 19:00:03 -0000 + +Quoth Ramakrishnan Muthukrishnan on Aug 05 at 11:35 pm: +> Ramakrishnan Muthukrishnan writes: +> +> > Hi, +> > +> > Pardon me for using non-standard terms here because of lack of knowledge +> > of the language used in the email world. +> > +> > I have this thread in which some messages are tagged unread. When the +> > thread is opened, I see that those unread messages are not shown in full +> > in the threaded view. Instead they appear `folded' with only the summary +> > appearing in the thread, as if they are read. +> > +> > Here is a screenshot: http://i.imgur.com/Eh8SKe6.png +> > +> > I am attaching the messages in the thread shown in the screenshot as a +> > tar.bz2 file. +> +> I just forgot to mention that I see this only when I do a +> notmuch-search with `*' and not when I search with a tag. + +'*' turns out to be the key. When showing a thread, +notmuch-show-build-buffer constructs a new query from the thread ID +and the original search query of the form + + thread:X and (original query) + +If the original query is '*', this is + + thread:X and (*) + +But * isn't part of the Xapian query syntax. Notmuch specially +handles queries that exactly match "*" before passing the query to +Xapian. When * is embedded in a larger query, this special casing +doesn't apply. In fact, Xapian parses this query as + + (Tmail AND (and:(pos=1) FILTER GX)) + +The "and" in the query turned into a plain search term (the "AND" is a +proper boolean operator, but is unrelated to the provided query). + +This is a symptom of a general problem where we assume queries are +textually composable, when they are not. We have the same problem at +least in notmuch-search-filter and in notmuch tag query optimization. +In this particular case, the * causes Xapian's query grammar to fail +to parse the query, which Xapian handles by re-parsing the entire +query will all query features disabled (which includes disabling +support for boolean operators). Unfortunately, just handling * better +isn't really a solution because it's just one of many things that +violates query composability. + +Some solutions I can see are: + +1) Switch to a composable query syntax (which would include *). This, + obviously, requires a custom query parser, but is the most + localized change I can think of and keeps queries as strings. + +2) Never construct queries by pasting strings together. This would + require changes to both the libnotmuch and CLI interfaces and + queries could no longer be strings, but in the words of Alan + Perlis, the string is a stark data structure. (In the case of + show, I would actually love it if we could specify separate search + and match queries because that would eliminate --entire-thread as + well as the fallback in notmuch-show-build-buffer when search + returns nothing, but I digress.) + +3) Keep queries as plain strings, but switch to some hybrid syntax + that lets us combine Xapian queries with composable operators + parsed by notmuch. When we need to combine queries, do it using + the composable operators. This actually may not be a bad way to + transition to a full custom query parser; I think it would be + relatively easy to take over parsing Xapian's boolean syntax, but + leave the "prob" parsing to Xapian. + +I lean towards 3 because it seems like the least disruptive and offers +a smooth transition to 1.