--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 6A9BE431FB6\r
+ for <notmuch@notmuchmail.org>; Mon, 5 Aug 2013 12:00:03 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id zt7tlyITOaMD for <notmuch@notmuchmail.org>;\r
+ Mon, 5 Aug 2013 11:59:57 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-5.mit.edu (dmz-mailsec-scanner-5.mit.edu\r
+ [18.7.68.34])\r
+ by olra.theworths.org (Postfix) with ESMTP id EF31C431FAE\r
+ for <notmuch@notmuchmail.org>; Mon, 5 Aug 2013 11:59:56 -0700 (PDT)\r
+X-AuditID: 12074422-b7ef78e000000935-5a-51fff62a96fd\r
+Received: from mailhub-auth-4.mit.edu ( [18.7.62.39])\r
+ by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 16.8C.02357.A26FFF15; Mon, 5 Aug 2013 14:59:54 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+ by mailhub-auth-4.mit.edu (8.13.8/8.9.2) with ESMTP id r75IxrMW006141; \r
+ Mon, 5 Aug 2013 14:59:54 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r75Ixosb022754\r
+ (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+ Mon, 5 Aug 2013 14:59:52 -0400\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1V6Q0S-0003q6-Sa; Mon, 05 Aug 2013 14:59:49 -0400\r
+Date: Mon, 5 Aug 2013 14:59:47 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Ramakrishnan Muthukrishnan <ram@rkrishnan.org>\r
+Subject: Re: unread message appear `folded'\r
+Message-ID: <20130805185947.GD7794@mit.edu>\r
+References:\r
+ <m2iozkf26n.fsf@ramakrmu-mac.i-did-not-set--mail-host-address--so-tickle-me>\r
+ <m2fvuof1yj.fsf@ramakrmu-mac.i-did-not-set--mail-host-address--so-tickle-me>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To:\r
+ <m2fvuof1yj.fsf@ramakrmu-mac.i-did-not-set--mail-host-address--so-tickle-me>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFmpkleLIzCtJLcpLzFFi42IRYrdT19X69j/QYHqrhcX1mzOZLdZ+TXJg\r
+ 8ni26hazx6I96QFMUVw2Kak5mWWpRfp2CVwZTU1TmQoeSlW8nLWDqYGxXbSLkZNDQsBE4tmE\r
+ G4wQtpjEhXvr2UBsIYF9jBJfzvt2MXIB2RsYJV7f388G4Zxikrja8JYFwlnCKPF/8z0gh4OD\r
+ RUBF4m+XMEg3m4CGxLb9y8GmiggYSVxrucIKYjMLSEt8+93MBGILC2hLXGpeAFbDC2SfW3Gc\r
+ CWLmckaJeRtWMEMkBCVOznzCAtGsJXHj30smkF0gg5b/4wAJcwpESkzf9x2sXBTohCknt7FN\r
+ YBSahaR7FpLuWQjdCxiZVzHKpuRW6eYmZuYUpybrFicn5uWlFuma6uVmluilppRuYgSHtIvS\r
+ DsafB5UOMQpwMCrx8CZc/R8oxJpYVlyZe4hRkoNJSZR3/xegEF9SfkplRmJxRnxRaU5q8SFG\r
+ CQ5mJRHe+VuAcrwpiZVVqUX5MClpDhYlcd5nT88GCgmkJ5akZqemFqQWwWRlODiUJHh5vwI1\r
+ ChalpqdWpGXmlCCkmTg4QYbzAA1nBanhLS5IzC3OTIfIn2JUlBLn/QtykQBIIqM0D64XlnJe\r
+ MYoDvSLM+xmkigeYruC6XwENZgIabPLzL8jgkkSElFQDo++CUMGY029+bWZPYb44M+XaqxRb\r
+ 9V0FzXu+n8p0/pTtxd5RujtmabnyhX8HWe5u/z4t80DGv48VczT4le/1cnTs0/5+ninAhOVv\r
+ ePWP8P3KfX8vqBmU7/qx8pxqt/lExdwcnSB17jt3Nv12jfdPDtIomrVw6nk982Vn5h+fub9O\r
+ 7tUGvouqS5VYijMSDbWYi4oTAbViH3IUAwAA\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 05 Aug 2013 19:00:03 -0000\r
+\r
+Quoth Ramakrishnan Muthukrishnan on Aug 05 at 11:35 pm:\r
+> Ramakrishnan Muthukrishnan <ram@rkrishnan.org> writes:\r
+> \r
+> > Hi,\r
+> >\r
+> > Pardon me for using non-standard terms here because of lack of knowledge\r
+> > of the language used in the email world.\r
+> >\r
+> > I have this thread in which some messages are tagged unread. When the\r
+> > thread is opened, I see that those unread messages are not shown in full\r
+> > in the threaded view. Instead they appear `folded' with only the summary\r
+> > appearing in the thread, as if they are read. \r
+> >\r
+> > Here is a screenshot: http://i.imgur.com/Eh8SKe6.png\r
+> >\r
+> > I am attaching the messages in the thread shown in the screenshot as a\r
+> > tar.bz2 file.\r
+> \r
+> I just forgot to mention that I see this only when I do a\r
+> notmuch-search with `*' and not when I search with a tag. \r
+\r
+'*' turns out to be the key. When showing a thread,\r
+notmuch-show-build-buffer constructs a new query from the thread ID\r
+and the original search query of the form\r
+\r
+ thread:X and (original query)\r
+\r
+If the original query is '*', this is\r
+\r
+ thread:X and (*)\r
+\r
+But * isn't part of the Xapian query syntax. Notmuch specially\r
+handles queries that exactly match "*" before passing the query to\r
+Xapian. When * is embedded in a larger query, this special casing\r
+doesn't apply. In fact, Xapian parses this query as\r
+\r
+ (Tmail AND (and:(pos=1) FILTER GX))\r
+\r
+The "and" in the query turned into a plain search term (the "AND" is a\r
+proper boolean operator, but is unrelated to the provided query).\r
+\r
+This is a symptom of a general problem where we assume queries are\r
+textually composable, when they are not. We have the same problem at\r
+least in notmuch-search-filter and in notmuch tag query optimization.\r
+In this particular case, the * causes Xapian's query grammar to fail\r
+to parse the query, which Xapian handles by re-parsing the entire\r
+query will all query features disabled (which includes disabling\r
+support for boolean operators). Unfortunately, just handling * better\r
+isn't really a solution because it's just one of many things that\r
+violates query composability.\r
+\r
+Some solutions I can see are:\r
+\r
+1) Switch to a composable query syntax (which would include *). This,\r
+ obviously, requires a custom query parser, but is the most\r
+ localized change I can think of and keeps queries as strings.\r
+\r
+2) Never construct queries by pasting strings together. This would\r
+ require changes to both the libnotmuch and CLI interfaces and\r
+ queries could no longer be strings, but in the words of Alan\r
+ Perlis, the string is a stark data structure. (In the case of\r
+ show, I would actually love it if we could specify separate search\r
+ and match queries because that would eliminate --entire-thread as\r
+ well as the fallback in notmuch-show-build-buffer when search\r
+ returns nothing, but I digress.)\r
+\r
+3) Keep queries as plain strings, but switch to some hybrid syntax\r
+ that lets us combine Xapian queries with composable operators\r
+ parsed by notmuch. When we need to combine queries, do it using\r
+ the composable operators. This actually may not be a bad way to\r
+ transition to a full custom query parser; I think it would be\r
+ relatively easy to take over parsing Xapian's boolean syntax, but\r
+ leave the "prob" parsing to Xapian.\r
+\r
+I lean towards 3 because it seems like the least disruptive and offers\r
+a smooth transition to 1.\r
projects / notmuch-archives.git / commitdiff