Re: nbook: a notmuch based address book written in python

diff --git a/c3/4b98b27c55f101e341d05660ff581a939e3ca2 b/c3/4b98b27c55f101e341d05660ff581a939e3ca2
new file mode 100644 (file)
index 0000000..0616a05
--- /dev/null
+++ b/c3/4b98b27c55f101e341d05660ff581a939e3ca2
@@ -0,0 +1,254 @@
+Return-Path: <fatkasuvayu+linux@gmail.com>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id DDD26431FBC\r
+       for <notmuch@notmuchmail.org>; Tue, 16 Oct 2012 07:55:12 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.799\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.799 tagged_above=-999 required=5\r
+       tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+       FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id 38H4adXAWhIE for <notmuch@notmuchmail.org>;\r
+       Tue, 16 Oct 2012 07:55:10 -0700 (PDT)\r
+Received: from mail-wi0-f179.google.com (mail-wi0-f179.google.com\r
+       [209.85.212.179]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 7751B431FB6\r
+       for <notmuch@notmuchmail.org>; Tue, 16 Oct 2012 07:55:10 -0700 (PDT)\r
+Received: by mail-wi0-f179.google.com with SMTP id hq7so2923337wib.2\r
+       for <notmuch@notmuchmail.org>; Tue, 16 Oct 2012 07:55:08 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;\r
+       h=sender:date:from:to:subject:message-id:mail-followup-to:references\r
+       :mime-version:content-type:content-disposition\r
+       :content-transfer-encoding:in-reply-to:user-agent;\r
+       bh=BJhNgeEvF4Du1+GeH7pci0NDb7w7CuRZCoGthXiZZ1M=;\r
+       b=Q1kTKKVckQE47xaKElMw3AopEf6QMJ4hHlSgOKm9+iDxHPqP7U6D7bjfha3NjDVsyW\r
+       XAlWLXFugnQDG6CPyxnNoOVA/T7SEiLjWE+myuH0T3D//YHDnET83KwcKlzzc1xihclM\r
+       jcPYWxt6sPhZIdVB7Z/GhaE6bFym5giRe8TFoJlHEKCgFnUmia+1sTkza5X0EPE4aobc\r
+       3M0s9SAYzV6VOjxFQPZXnO0sIFhLom0AY006GfHuq1PipXa9GnJcg4NryoYS0Iv+eDf6\r
+       kg3JvpK+sZK7yT2QXiDtjG+yobHQgMK2q3oEabDIiInl6DayUY8p9cG9vWVpYTQqP7Vv\r
+       vQEw==\r
+Received: by 10.180.80.104 with SMTP id q8mr32528965wix.6.1350399307879;\r
+       Tue, 16 Oct 2012 07:55:07 -0700 (PDT)\r
+Received: from kuru.dyndns-at-home.com (pb-d-128-141-52-183.cern.ch.\r
+       [128.141.52.183])\r
+       by mx.google.com with ESMTPS id f1sm19474364wiy.2.2012.10.16.07.55.05\r
+       (version=TLSv1/SSLv3 cipher=OTHER);\r
+       Tue, 16 Oct 2012 07:55:06 -0700 (PDT)\r
+Sender: suvayu ali <fatkasuvayu@gmail.com>\r
+Date: Tue, 16 Oct 2012 16:55:03 +0200\r
+From: Suvayu Ali <fatkasuvayu+linux@gmail.com>\r
+To: notmuch@notmuchmail.org\r
+Subject: Re: nbook: a notmuch based address book written in python\r
+Message-ID: <20121016145503.GC11488@kuru.dyndns-at-home.com>\r
+Mail-Followup-To: notmuch@notmuchmail.org\r
+References: <20120924082646.GA10577@kuru.dyndns-at-home.com>\r
+       <20120925104457.12264.30350@megatron>\r
+       <20121008093429.GC4534@kuru.dyndns-at-home.com>\r
+       <20121013165851.29671.29869@brick.lan>\r
+       <20121015105830.12412.43278@thinkbox.jade-hamburg.de>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=utf-8\r
+Content-Disposition: inline\r
+Content-Transfer-Encoding: 8bit\r
+In-Reply-To: <20121015105830.12412.43278@thinkbox.jade-hamburg.de>\r
+User-Agent: Mutt/1.5.21 (2011-07-01)\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 16 Oct 2012 14:55:13 -0000\r
+\r
+Hi Justus,\r
+\r
+I finally had time to go through your response carefully.\r
+\r
+On Mon, Oct 15, 2012 at 12:58:30PM +0200, Justus Winter wrote:\r
+> \r
+> > > > -------------------------------\r
+> > > > [~] time nbook Patrick                     \r
+> > > > \r
+> > > > Error opening /home/pazz/mail/gmail/[Google Mail].All Mail/cur/1330682270_0.12958.megatron,U=8766,FMD5=66ff6a8bc18a8a3ac4b311daa93d358a:2,S: Too many open files\r
+> > > > Traceback (most recent call last):\r
+> > > >   File "/home/pazz/bin/nbook", line 167, in <module>\r
+> > > >   File "/home/pazz/bin/nbook", line 71, in __init__\r
+> > > >   File "/home/pazz/.local/lib/python2.7/site-packages/notmuch/message.py", line 233, in get_header\r
+> > > > notmuch.errors.NullPointerError\r
+\r
+[...]\r
+\r
+> > As mentioned before, I think you invalidate the Database object concurrently\r
+> > while your long-running algorithm goes through all messages.\r
+> > Xapian doesn't handle concurrent access to the index like a normal™ database would.\r
+> > This means you are notified by this error that some changes were detected.\r
+> > Maybe the error message should be more telling here though. Teythoon?\r
+> \r
+> The reason for this error is exactly what the error message says, you\r
+> are opening to many files. Check out this limit using ulimit -n:\r
+> \r
+> % ulimit -n\r
+> 4096\r
+> \r
+> This problem is subtle. Here is a minimal test case:\r
+> \r
+> ~~~ snip ~~~\r
+> import notmuch\r
+> \r
+> with notmuch.Database() as db:\r
+>     query = notmuch.Query(db, 'a').search_messages()\r
+>     for msg in query:\r
+>         msg.get_header('from')\r
+> \r
+> with notmuch.Database() as db:\r
+>     query = notmuch.Query(db, 'a').search_messages()\r
+>     for msg in list(query):\r
+>         msg.get_header('from')\r
+> ~~~ snap ~~~\r
+> \r
+> % python test.py\r
+> Error opening /home/teythoon/Maildir/.lists.notmuch/cur/1323251462.M53044P18514.thinkbox,S=7306,W=7466:2,: Too many open files\r
+> Traceback (most recent call last):\r
+>   File "test.py", line 11, in <module>\r
+>     msg.get_header('from')\r
+>   File "/home/teythoon/.local/lib/python2.7/site-packages/notmuch/message.py", line 237, in get_header\r
+>     raise NullPointerError()\r
+> notmuch.errors.NullPointerError\r
+> \r
+> Observe that it blows up in line 11, the first version works. The only\r
+> difference is that the second version creates a list from the notmuch\r
+> query. This prevents the garbage collector from collecting the message\r
+> objects and thus closing the file handles. So here's your fix:\r
+> \r
+> ~~~ snip ~~~\r
+> diff --git a/nbook b/nbook\r
+> index 387c71d..b3d4fd6 100755\r
+> --- a/nbook\r
+> +++ b/nbook\r
+> @@ -173,7 +173,7 @@ class AddressHeaders(object):\r
+>  # Search\r
+>  db = Database()\r
+>  query = Query(db, 'from:"{0}" or to:"{0}"'.format(querystr))\r
+> -msgs = list(query.search_messages())\r
+> +msgs = query.search_messages()\r
+>  \r
+>  addresses = AddressHeaders(msgs, querystr)\r
+>  print addresses\r
+> ~~~ snap ~~~\r
+> \r
+\r
+This explanation helped me a lot, thanks!\r
+\r
+> A few more comments:\r
+> \r
+> > from notmuch import *\r
+> \r
+> Please avoid * imports, they prevent tools like pyflakes from checking\r
+> whether you accidentally misspelled any identifiers.\r
+> \r
+\r
+Point taken.  I'll be more careful in the future.  :)\r
+\r
+> > pyversion = float('%d.%d' % (sys.version_info.major, sys.version_info.minor))\r
+> > if pyversion < 2.7:\r
+> \r
+> Converting this to float feels wrong. Consider doing sth like\r
+> \r
+> if sys.version_info.major > 2 or (sys.version_info.major == 2 and sys.version_info.minor >= 7):\r
+> \r
+\r
+I incorporated these suggestions too.\r
+\r
+> >     print '`nbook\' needs Python 2.7 or higher for argparse'\r
+> \r
+> Note that in py3k print is a function and not a statement, so you need\r
+> to use braces. Consider dropping this at the beginning of all your\r
+> python files to make py2.7 use the new features:\r
+> \r
+> from __future__ import print_function, absolute_import, unicode_literals\r
+> \r
+> >     exit(-1)\r
+> \r
+> exit is not a builtin function. You have to use sys.exit. Tools like\r
+> pyflakes can spot this kind of mistakes. Also, sys.exit also accepts a\r
+> string as argument which it prints to stderr before exiting with an\r
+> error code.\r
+> \r
+\r
+I will read-up some more about the above suggestions and update\r
+accordingly.\r
+\r
+> >         self.__fromhdr__ += ',' + msg.get_header('from')\r
+> \r
+> Hm, this is somewhat unpythonic. It used to be the case that building\r
+> strings this way was a lot slower than building a list and then\r
+> joining it on a delimiter of your choice\r
+> (i.e. ','.join(from_headers)). This is (was?) because strings are\r
+> immutable in python and constantly creating strings just to throw them\r
+> away in the next iteration puts a lot of pressure on the memory\r
+> management system. Somewhat recent discussion here:\r
+> \r
+> http://stackoverflow.com/questions/1316887/what-is-the-most-efficient-string-concatenation-method-in-python\r
+> \r
+\r
+I had a commit with ','.join(..) in a private branch, but thanks for\r
+pointing out the reasons and the links to the discussion.  This was very\r
+helpful.\r
+\r
+> >     def print_addrs(self, fmtstr='', query=''):\r
+> >         if '' == fmtstr: fmtstr = '%s    %s\n'\r
+> \r
+> Ok, several things here:\r
+> \r
+> * The comparison looks weird, you are using the string constant as the\r
+>   first operand. While this is technically not wrong, it is somewhat\r
+>   unpythonic b/c if you read it out loud (''if the empty string is\r
+>   equal to fmtstr'') it somewhat bends the 1:1 mapping of the semantic\r
+>   of your program and the English sentence. It looks like this c hack\r
+>   that is actually unnecessary in python b/c you cannot use the\r
+>   assignment operator as a value (except for a=b=c=0 style\r
+>   assignments).\r
+> \r
+\r
+Yes you are correct, I'm more used to C/C++ and the reason you mention\r
+is why I tend to write comparisons like that.  I'll retrain my fingers\r
+for python from now on.\r
+\r
+> * Please don't put multiple statements in one line.\r
+> \r
+\r
+I will keep that in mind for the future.\r
+\r
+> * This can be written shorter and more idiomatic (yay keyword\r
+>   arguments):\r
+> \r
+>     def print_addrs(self, fmtstr='%s    %s\n', query=''):\r
+>         [...]\r
+> \r
+\r
+That was silly of me not to do that in the first place! :-p\r
+\r
+> Happy hacking :)\r
+> Justus\r
+\r
+Thank you soo much for this incredibly informative response.  I learned\r
+a lot.\r
+\r
+Cheers,\r
+\r
+-- \r
+Suvayu\r
+\r
+Open source is the future. It sets us free.\r