[PATCH 4/4] Update NEWS for user.other_name

[notmuch-archives.git] / 8a / f3530b7e26dcc5a34bf90560825feaf26f908d

Return-Path: <pieter@praet.org>\r
X-Original-To: notmuch@notmuchmail.org\r
Delivered-To: notmuch@notmuchmail.org\r
Received: from localhost (localhost [127.0.0.1])\r
        by olra.theworths.org (Postfix) with ESMTP id C913A431FD0\r
        for <notmuch@notmuchmail.org>; Sat, 16 Jul 2011 08:07:18 -0700 (PDT)\r
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
X-Spam-Flag: NO\r
X-Spam-Score: -0.69\r
X-Spam-Level: \r
X-Spam-Status: No, score=-0.69 tagged_above=-999 required=5\r
        tests=[RCVD_IN_DNSWL_LOW=-0.7, T_MIME_NO_TEXT=0.01] autolearn=disabled\r
Received: from olra.theworths.org ([127.0.0.1])\r
        by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
        with ESMTP id 4itz8bn-Mnyi for <notmuch@notmuchmail.org>;\r
        Sat, 16 Jul 2011 08:07:18 -0700 (PDT)\r
Received: from mail-wy0-f181.google.com (mail-wy0-f181.google.com\r
        [74.125.82.181]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
        (No client certificate requested)\r
        by olra.theworths.org (Postfix) with ESMTPS id D65EE431FB6\r
        for <notmuch@notmuchmail.org>; Sat, 16 Jul 2011 08:07:17 -0700 (PDT)\r
Received: by wyh22 with SMTP id 22so466768wyh.26\r
        for <notmuch@notmuchmail.org>; Sat, 16 Jul 2011 08:07:16 -0700 (PDT)\r
Received: by 10.227.195.209 with SMTP id ed17mr4126119wbb.13.1310828836389;\r
        Sat, 16 Jul 2011 08:07:16 -0700 (PDT)\r
Received: from localhost (130.40-242-81.adsl-dyn.isp.belgacom.be\r
        [81.242.40.130])\r
        by mx.google.com with ESMTPS id gd1sm1922019wbb.10.2011.07.16.08.07.14\r
        (version=TLSv1/SSLv3 cipher=OTHER);\r
        Sat, 16 Jul 2011 08:07:15 -0700 (PDT)\r
From: Pieter Praet <pieter@praet.org>\r
To: Austin Clements <amdragon@MIT.EDU>\r
Subject: Re: [PATCH v2] emacs: bad regexp @ `notmuch-search-process-filter'\r
In-Reply-To: <20110713185721.GI25558@mit.edu>\r
References: <20110705214234.GA15360@mit.edu>\r
        <1310416993-31031-1-git-send-email-pieter@praet.org>\r
        <20110711210532.GC25558@mit.edu> <878vs28dvo.fsf@praet.org>\r
        <20110713185721.GI25558@mit.edu>\r
User-Agent: Notmuch/0.6-60-ga0910f1 (http://notmuchmail.org) Emacs/23.1.50.1\r
        (x86_64-pc-linux-gnu)\r
Date: Sat, 16 Jul 2011 17:07:12 +0200\r
Message-ID: <87oc0u6z8f.fsf@praet.org>\r
MIME-Version: 1.0\r
Content-Type: multipart/mixed; boundary="=-=-="\r
Cc: Notmuch Mail <notmuch@notmuchmail.org>, David Edmondson <dme@dme.org>\r
X-BeenThere: notmuch@notmuchmail.org\r
X-Mailman-Version: 2.1.13\r
Precedence: list\r
List-Id: "Use and development of the notmuch mail system."\r
        <notmuch.notmuchmail.org>\r
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
        <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
List-Post: <mailto:notmuch@notmuchmail.org>\r
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
        <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
X-List-Received-Date: Sat, 16 Jul 2011 15:07:18 -0000\r
\r
--=-=-=\r
\r
On Wed, 13 Jul 2011 14:57:21 -0400, Austin Clements <amdragon@MIT.EDU> wrote:\r
> Quoth Pieter Praet on Jul 13 at  4:16 pm:\r
> > On Mon, 11 Jul 2011 17:05:32 -0400, Austin Clements <amdragon@MIT.EDU> wrote:\r
> > > Quoth Pieter Praet on Jul 11 at 10:43 pm:\r
> > > > TL;DR: I can haz regex pl0x?\r
> > > \r
> > > Oof, what a pain.  I'm happy to change the output format of search; I\r
> > > hadn't realized how difficult it would be to parse.  In fact, I'm not\r
> > > sure it's even parsable by regexp, because the message ID's themselves\r
> > > could contain parens.\r
> > > \r
> > > So what would be a good format?  One possibility would be to\r
> > > NULL-delimit the query part; as distasteful as I find that, this part\r
> > > of the search output isn't meant for user consumption.  Though I fear\r
> > > this is endemic to the dual role the search output currently plays as\r
> > > both user and computer readable.\r
> > > \r
> > > I've also got the code to do everything using document ID's instead of\r
> > > message ID's.  As a side-effect, it makes the search output clean and\r
> > > readily parsable since document ID's are just numbers.  Hence, there\r
> > > are no quoting or escaping issues (plus the output is much more\r
> > > compact).  I haven't sent this to the list yet because I haven't had a\r
> > > chance to benchmark it and determine if the performance benefits make\r
> > > exposing document ID's worthwhile.\r
> > \r
> > Jamie Zawinski once said/wrote [1]:\r
> >   'Some people, when confronted with a problem, think "I know,\r
> >   I'll use regular expressions." Now they have two problems.'\r
> > \r
> > With this in mind, I set out to get rid of this whole regex mess altogether,\r
> > by populating the search buffer using Notmuch's JSON output instead of doing\r
> > brittle text matching tricks.\r
> > \r
> > Looking for some documentation, I stumbled upon a long-forgotten gem [2].\r
> > \r
> > David's already done pretty much all of the work for us!\r
> \r
> Yes, similar thoughts were running through my head as I futzed with\r
> the formatting for this.  My concern with moving to JSON for search\r
> buffers is that parsing it is about *30 times slower* than the current\r
> regexp-based approach (0.6 seconds versus 0.02 seconds for a mere 1413\r
> result search buffer).  I think JSON makes a lot of sense for show\r
> buffers because there's generally less data and it has a lot of\r
> complicated structure.  Search results, on the other hand, have a very\r
> simple, regular, and constrained structure, so JSON doesn't buy us\r
> nearly as much.\r
\r
That seems about right. Using the entire Notmuch mailing list archive,\r
processing JSON ends up taking 23x longer (see test in att).\r
\r
> JSON is hard to parse because, like the text search output, it's\r
> designed for human consumption (of course, unlike the text search\r
> output, it's also designed for computer consumption).  There's\r
> something to be said for the debuggability and generality of this and\r
> JSON is very good for exchanging small objects, but it's a remarkably\r
> inefficient way to exchange large amounts of data between two\r
> programs.\r
> \r
> I guess what I'm getting at, though it pains me to say it, is perhaps\r
> search needs a fast, computer-readable interchange format.  The\r
> structure of the data is so simple and constrained that this could be\r
> altogether trivial.\r
\r
I guess that's our only option then. Could you implement it for me?\r
I'll make sure to rebase my patch series in an acceptable time frame.\r
\r
An extra output format shouldn't be that much of a problem though, if we\r
further compartmentalize the code. What are your thoughts on (in the\r
long term) moving to a plugin-based architecture? Eg. enable something\r
like this:\r
\r
  ./input/{Maildir, ...}\r
  ./output/{plain, JSON, ...}\r
  ./filters/{crypto, ...}\r
  ./backends/(Xapian, ...)\r
  ./uis/{Emacs, VIM, web, ...}\r
\r
> Or maybe I need a faster computer.\r
\r
That's what M$ Tech Support would want you to believe :)\r
What we need is slower computers, so devs are forced to count cycles again.\r
The rise of netbooks has thankfully done wonders in this respect.\r
\r
> If anyone is curious, here's how I timed the parsing.\r
> \r
> (defmacro time-it (code)\r
>   `(let ((start-time (get-internal-run-time)))\r
>      ,code\r
>      (float-time (time-subtract (get-internal-run-time) start-time))))\r
> \r
> (with-current-buffer "json"\r
>   (goto-char (point-min))\r
>   (time-it (json-read)))\r
> \r
> (with-current-buffer "text"\r
>   (goto-char (point-min))\r
>   (time-it\r
>    (while (re-search-forward "^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$" nil t))))\r
\r
\r
Peace\r
\r
-- \r
Pieter\r
\r
--=-=-=\r
Content-Type: application/octet-stream\r
Content-Disposition: attachment; filename=regexp-vs-json.org\r
Content-Transfer-Encoding: base64\r
\r
KiBQYXJzaW5nIHBsYWluLXRleHQgdy8gcmVnZXhwIHZzLiBwYXJzaW5nIEpTT04uCgogICMrU09V\r
UkNFOiB0aW1lci1tYWNybwogICMrQkVHSU5fU1JDIGVtYWNzLWxpc3AKICAgIChkZWZtYWNybyB0\r
aW1lLWl0IChjb2RlKQogICAgICBgKGxldCAoKHN0YXJ0LXRpbWUgKGdldC1pbnRlcm5hbC1ydW4t\r
dGltZSkpKQogICAgICAgICAsY29kZQogICAgICAgICAoZmxvYXQtdGltZSAodGltZS1zdWJ0cmFj\r
dCAoZ2V0LWludGVybmFsLXJ1bi10aW1lKSBzdGFydC10aW1lKSkpKQogICMrRU5EX1NSQwoKICAj\r
K1NPVVJDRTogY291bnQtbXNncwogICMrQkVHSU5fU1JDIHNoCiAgICBub3RtdWNoIGNvdW50IC0t\r
IHRhZzp4L25vdG11Y2gKICAjK0VORF9TUkMKCiAgIytTT1VSQ0U6IHRpbWUtdGV4dAogICMrQkVH\r
SU5fU1JDIGVtYWNzLWxpc3AgOm5vd2ViIHllcwogICAgPDx0aW1lci1tYWNybz4+CiAgICAod2l0\r
aC10ZW1wLWJ1ZmZlcgogICAgICAoY2FsbC1wcm9jZXNzICJub3RtdWNoIiBuaWwgdCBuaWwgInNl\r
YXJjaCIgIi0tZm9ybWF0PXRleHQiICItLSIgInRhZzp4L25vdG11Y2giKQogICAgICAoZ290by1j\r
aGFyIChwb2ludC1taW4pKQogICAgICAodGltZS1pdAogICAgICAgKHdoaWxlIChyZS1zZWFyY2gt\r
Zm9yd2FyZCAiXlxcKHRocmVhZDpbMC05QS1GYS1mXSpcXCkgXFwoW15dW10qXFwpIFxcKFxcW1sw\r
LTkvXSpcXF1cXCkgXFwoW147XSpcXCk7CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg\r
ICBcXCguKlxcKSAoXFwoW14oKV0qXFwpKSQiIG5pbCB0KSkpKQogICMrRU5EX1NSQwoKICAjK1NP\r
VVJDRTogdGltZS1qc29uCiAgIytCRUdJTl9TUkMgZW1hY3MtbGlzcCA6bm93ZWIgeWVzCiAgICA8\r
PHRpbWVyLW1hY3JvPj4KICAgICh3aXRoLXRlbXAtYnVmZmVyCiAgICAgIChjYWxsLXByb2Nlc3Mg\r
Im5vdG11Y2giIG5pbCB0IG5pbCAic2VhcmNoIiAiLS1mb3JtYXQ9anNvbiIgIi0tIiAidGFnOngv\r
bm90bXVjaCIpCiAgICAgIChnb3RvLWNoYXIgKHBvaW50LW1pbikpCiAgICAgICh0aW1lLWl0IChq\r
c29uLXJlYWQpKSkKICAjK0VORF9TUkMKCiAgIytUQkxOQU1FOiByZXN1bHRzCiAgfC0tLS0tLS0t\r
LS0rLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS0tLSstLS0tLS0tLS0tfAogIHwgbXNnY291bnQgfCB0\r
aW1lKHRleHQpIHwgdGltZShqc29uKSB8ICUgc2xvd2VyIHwKICB8LS0tLS0tLS0tLSstLS0tLS0t\r
LS0tLS0rLS0tLS0tLS0tLS0tKy0tLS0tLS0tLS18CiAgfCAgICAgNTI5NCB8ICAgICAgIDAuMDEg\r
fCAgICAgICAwLjIzIHwgICAgMjMwMC4gfAogIHwtLS0tLS0tLS0tKy0tLS0tLS0tLS0tLSstLS0t\r
LS0tLS0tLS0rLS0tLS0tLS0tLXwKICAjK1RCTEZNOiAkMT0nKHNiZSAiY291bnQtbXNncyIpJyA6\r
OiAkMj0nKHNiZSAidGltZS10ZXh0IiknIDo6ICQzPScoc2JlICJ0aW1lLWpzb24iKScgOjogJDQ9\r
KCQzLyQyKSoxMDAK\r
--=-=-=--\r