From: Patrick Totzke Date: Sat, 13 Oct 2012 16:58:51 +0000 (+0100) Subject: Re: nbook: a notmuch based address book written in python X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=7281821dbac865fead0a2d1a48768325dbc012f8;p=notmuch-archives.git Re: nbook: a notmuch based address book written in python --- diff --git a/88/1b0df88e072918e08f281e4034dc1140096a3a b/88/1b0df88e072918e08f281e4034dc1140096a3a new file mode 100644 index 000000000..c424734dd --- /dev/null +++ b/88/1b0df88e072918e08f281e4034dc1140096a3a @@ -0,0 +1,242 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id AC568431FAF + for ; Sat, 13 Oct 2012 09:58:58 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 1.7 +X-Spam-Level: * +X-Spam-Status: No, score=1.7 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, + FREEMAIL_FROM=0.001, FREEMAIL_REPLY=2.499, RCVD_IN_DNSWL_LOW=-0.7] + autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id ALU8TD6-Lb9y for ; + Sat, 13 Oct 2012 09:58:57 -0700 (PDT) +Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com + [209.85.212.179]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 120C1431FAE + for ; Sat, 13 Oct 2012 09:58:56 -0700 (PDT) +Received: by mail-wi0-f179.google.com with SMTP id hq7so474734wib.2 + for ; Sat, 13 Oct 2012 09:58:54 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=content-type:mime-version:content-disposition:from:user-agent:to + :references:in-reply-to:message-id:subject:date; + bh=8JtzhEy/zL//eJsxSac+ZULSPhwHzhHgqdikRBckbBE=; + b=zsJbFd9AWA4L5uU2tz9nwCS0RZZ/4kIityrY36ubAPba4TOrryuSwp6vBjkGihThRo + lPFkLwULUwxm1mNfoSi4MGqeQjytaHkdyCL4l0nm34PbBn0oq1u16e6mdsscw1gXYrBX + utO96MC2X8sgNxPRnYcoUZDbjYkXngSlAhSAR6RYlgw+K187DCYLf1hiSGmALoOErOQJ + 4JOTfmEGmJ8MMIO93vqsodoO5NGH1BkSqVnGzf8iDH4eqzr7pQ/+0wddbL9RV1YTMWnF + AsIuG+1OtyPL1arv1xgsQnoFiOSEQ6CEFxJT0HYy+9yG2gCQr8HLRsSMJ5KnwyNAeJwr + c8tQ== +Received: by 10.216.227.133 with SMTP id d5mr4665672weq.194.1350147534338; + Sat, 13 Oct 2012 09:58:54 -0700 (PDT) +Received: from localhost (cpc6-sgyl27-2-0-cust46.sgyl.cable.virginmedia.com. + [82.32.140.47]) + by mx.google.com with ESMTPS id dm3sm4558286wib.3.2012.10.13.09.58.52 + (version=TLSv1/SSLv3 cipher=OTHER); + Sat, 13 Oct 2012 09:58:53 -0700 (PDT) +Content-Type: multipart/signed; protocol="application/pgp-signature"; + micalg="pgp-sha1"; boundary="===============3247913046672548219==" +MIME-Version: 1.0 +Content-Disposition: inline +From: Patrick Totzke +User-Agent: alot/0.3.3+ +To: Suvayu Ali , notmuch@notmuchmail.org +References: <20120924082646.GA10577@kuru.dyndns-at-home.com> + <20120925104457.12264.30350@megatron> + <20121008093429.GC4534@kuru.dyndns-at-home.com> +In-Reply-To: <20121008093429.GC4534@kuru.dyndns-at-home.com> +Message-ID: <20121013165851.29671.29869@brick.lan> +Subject: Re: nbook: a notmuch based address book written in python +Date: Sat, 13 Oct 2012 17:58:51 +0100 +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Sat, 13 Oct 2012 16:58:58 -0000 + +--===============3247913046672548219== +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: quoted-printable + +Quoting Suvayu Ali (2012-10-08 10:34:29) +> Hi Patrick, +> = + +> Sorry for the very late reply; I got distracted with some personal +> matters. +> = + +> On Tue, Sep 25, 2012 at 11:44:57AM +0100, Patrick Totzke wrote: +> > Hey Suvayu, welcome to notmuch! +> > = + +> > I hope you are aware that there are already a few search based abook to= +ols +> > around for notmuch (listed in the wiki, albeit hidden in the emacs docs= +): +> > http://notmuchmail.org/emacstips/#index14h2 +> > I personally use nottoomuch-addresses.sh, which apparently does some ad= +vanced +> > caching voodoo for speed. +> > = + +> = + +> I wasn't aware of either of them, thanks for pointing them out. I'll +> take a look for inspiration and ideas. +> = + +> > But to your tool; practice test: +> > I wasn't able to use wildcards or simply prefixes of names. This is ess= +ential +> > if you want to use it for tabcompleting contacts in a MUA. +> = + +> Since the idea was inspired by the completion on the Gmail web +> interface, I already do a partial search so wildcards should not be +> necessary. + +Not sure what you mean here: If I compose a mail using gmails web interface +and type a prefix of someone's name I will get this contect as a suggestion. +My point was that using your tool, I did not get a contact suggested +for all prefixes. + +> > The time lookups take seems to depend on how many matches there are: +> > = + +> > ------------------------------- +> > time nbook Suvayu +> > 1 unique email addresses found for `Suvayu' +> > fatkasuvayu+linux@gmail.com Suvayu Ali +> > = + +> > nbook Suvayu 0.04s user 0.01s system 95% cpu 0.050 total +> > ------------------------------- +> > time nbook Justus +> > ... +> > = + +> > nbook Justus 0.21s user 0.07s system 11% cpu 2.484 total +> > ------------------------------- +> = + +> Yes, I noticed this too when I searched for the more common names. Not +> sure how to get around this though. + +I think this is a conceptual problem with your algorithm: +You look up *all* messages and add a name to your result-list +if it matches. This means you go through some condidate +as often as you index contains mails from/to him. +What one really wants is to ask the database to do something like + "SELECT name,email from RECIPIENTS_OR_SENDER" +where RECIPIENTS_OR_SENDER is some imaginary list that stores +a set of contacts. + +Bottom line: One would have to change the layout of the underlying +database (not likely) or do regularly update some cache +and only work on that. This is what some of the mentioned tools do if i'm n= +ot mistaken. + +> > And If I look for my own name, this takes over a minute, +> > eventually dying. This could be an issue with libnotmuch though. +> > Possibly, your algorithm takes very long and then reads from an initial= +ly +> > opened Database object again, which was invalidated by concurrent write= +s of other processes.. +> > = + +> > ------------------------------- +> > [~] time nbook Patrick = + +> > = + +> > Error opening /home/pazz/mail/gmail/[Google Mail].All Mail/cur/13306822= +70_0.12958.megatron,U=3D8766,FMD5=3D66ff6a8bc18a8a3ac4b311daa93d358a:2,S: T= +oo many open files +> > Traceback (most recent call last): +> > File "/home/pazz/bin/nbook", line 167, in +> > File "/home/pazz/bin/nbook", line 71, in __init__ +> > File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.p= +y", line 233, in get_header +> > notmuch.errors.NullPointerError +> > Error in sys.excepthook: +> > Traceback (most recent call last): +> > File "/usr/lib/python2.7/dist-packages/apport_python_hook.py", line 6= +6, in apport_excepthook +> > ImportError: No module named fileutils +> > = + +> > Original exception was: +> > Traceback (most recent call last): +> > File "/home/pazz/bin/nbook", line 167, in +> > File "/home/pazz/bin/nbook", line 71, in __init__ +> > File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.p= +y", line 233, in get_header +> > notmuch.errors.NullPointerError +> > nbook Patrick 3.20s user 5.47s system 12% cpu 1:11.65 total +> > ------------------------------------ +> > = + +> = + +> Yes someone else pointed this out too. Again I'm not sure how to +> proceed here. I had a quick look at this last week and it seemed to me +> the limitation comes from within the python bindings for notmuch. Do +> you have any ideas? + +As mentioned before, I think you invalidate the Database object concurrently +while your long-running algorithm goes through all messages. +Xapian doesn't handle concurrent access to the index like a normal=E2=84=A2= + database would. +This means you are notified by this error that some changes were detected. +Maybe the error message should be more telling here though. Teythoon? + +> > Anyway, have fun hacking notmuch! If you are looking for a related proj= +ect to bring in your python skills +> > I could think of one or two :D +> = + +> That would be wonderful. To give you my background, I'm a graduate +> student in physics and I have to do a lot of C/C++ and python +> programming for my research. Contributing to FOSS projects seems like a +> wonderful way to learn to collaborate and clean programming (we +> physicists tend to be sloppy programmers :-p). + +https://github.com/teythoon/afew +https://github.com/pazz/alot +http://excess.org/urwid/ + +I'm sure patches will be welcome to any of the above :) +Best, +/p + +--===============3247913046672548219== +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Description: signature +Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii" + +-----BEGIN PGP SIGNATURE----- +Version: GnuPG v1.4.11 (GNU/Linux) + +iEYEABECAAYFAlB5ncsACgkQlDQDZ9fWxaofuwCbBIrFTCAEoimDW+oZLkLIOp5+ +hFsAnjPfXjLw2idZX33ykZMrhQ5KXSp/ +=1H/w +-----END PGP SIGNATURE----- + +--===============3247913046672548219==--