--- /dev/null
+Return-Path: <tomi.ollila@iki.fi>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 99689431FBC\r
+ for <notmuch@notmuchmail.org>; Thu, 30 May 2013 09:06:55 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+ autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id S8n+lLFKrtT0 for <notmuch@notmuchmail.org>;\r
+ Thu, 30 May 2013 09:06:47 -0700 (PDT)\r
+Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])\r
+ by olra.theworths.org (Postfix) with ESMTP id 85B7D431FAE\r
+ for <notmuch@notmuchmail.org>; Thu, 30 May 2013 09:06:47 -0700 (PDT)\r
+Received: from guru.guru-group.fi (localhost [IPv6:::1])\r
+ by guru.guru-group.fi (Postfix) with ESMTP id 475A8100093;\r
+ Thu, 30 May 2013 19:06:43 +0300 (EEST)\r
+From: Tomi Ollila <tomi.ollila@iki.fi>\r
+To: Justus Winter <4winter@informatik.uni-hamburg.de>,\r
+ Julian Berman <Julian@GrayVines.com>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] Fix shared library loading in Python bindings.\r
+In-Reply-To: <20130530122608.20099.26290@thinkbox.jade-hamburg.de>\r
+References: <1369540418-94177-1-git-send-email-Julian@GrayVines.com>\r
+ <20130530122608.20099.26290@thinkbox.jade-hamburg.de>\r
+User-Agent: Notmuch/0.15.2+115~g12cf6af (http://notmuchmail.org) Emacs/24.3.1\r
+ (x86_64-unknown-linux-gnu)\r
+X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL\r
+ $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F\r
+ !)g;OY^,BjTbr)Np:%c_o'jj,Z\r
+Date: Thu, 30 May 2013 19:06:42 +0300\r
+Message-ID: <m28v2wcuzx.fsf@guru.guru-group.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 30 May 2013 16:06:55 -0000\r
+\r
+On Thu, May 30 2013, Justus Winter <4winter@informatik.uni-hamburg.de> wrote:\r
+\r
+> Hi Julian :)\r
+>\r
+> Quoting Julian Berman (2013-05-26 05:53:38)\r
+>> Specifically, fixes loading on OS X, where libnotmuch will be\r
+>> a dylib.:\r
+>> ---\r
+>> bindings/python/notmuch/globals.py | 3 ++-\r
+>> 1 file changed, 2 insertions(+), 1 deletion(-)\r
+>> \r
+>> diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py\r
+>> index c7632c3..5e08e73 100644\r
+>> --- a/bindings/python/notmuch/globals.py\r
+>> +++ b/bindings/python/notmuch/globals.py\r
+>> @@ -18,11 +18,12 @@ Copyright 2010 Sebastian Spaeth <Sebastian@SSpaeth.de>\r
+>> """\r
+>> \r
+>> from ctypes import CDLL, Structure, POINTER\r
+>> +from ctypes.util import find_library\r
+>> \r
+>> #-----------------------------------------------------------------------------\r
+>> #package-global instance of the notmuch library\r
+>> try:\r
+>> - nmlib = CDLL("libnotmuch.so.3")\r
+>> + nmlib = CDLL(find_library("libnotmuch"))\r
+>\r
+> Does this work for you on Darwin? On my box (Debian/Linux) I have to\r
+> use "notmuch" instead of "libnotmuch" to get anything from\r
+> find_library:\r
+>\r
+> In [5]: print ctypes.util.find_library("notmuch")\r
+> libnotmuch.so.3\r
+>\r
+> In [6]: print ctypes.util.find_library("libnotmuch")\r
+> None\r
+>\r
+> Then again, find_library is different for every architecture under the\r
+> sun...\r
+>\r
+>> except:\r
+>> raise ImportError("Could not find shared 'notmuch' library.")\r
+>\r
+> I'm leaning towards declining the patch in it's current form. If the\r
+> bindings do not work on OS X, we need to find another solution. There\r
+> are two reasons for this:\r
+>\r
+> 1. find_library was once used, but was removed since it is (has?) been\r
+> problematic wrt to LD_LIBRARY_PATH usage:\r
+>\r
+> % git show f378f458\r
+> commit f378f45893bb4263402008b2abd8aab522901fb6\r
+> Author: Cedric Cabessa <ced@ryick.net>\r
+> Date: Mon Apr 5 03:03:51 2010 +0200\r
+>\r
+> find_library does not read LD_LIBRARY_PATH, but CDLL does.\r
+>\r
+> diff --git a/cnotmuch/globals.py b/cnotmuch/globals.py\r
+> index ef2686f..fa20ae8 100644\r
+> --- a/cnotmuch/globals.py\r
+> +++ b/cnotmuch/globals.py\r
+> @@ -3,17 +3,17 @@ from ctypes.util import find_library\r
+> \r
+> #-----------------------------------------------------------------------------\r
+> #package-global instance of the notmuch library\r
+> -#TODO: lazy load this on first access?\r
+> -so = find_library('notmuch')\r
+> -if so is None:\r
+> - raise ImportError("Could not find shared 'notmuch' library.")\r
+> -nmlib = CDLL(so)\r
+> +try:\r
+> + nmlib = CDLL("libnotmuch.so.1")\r
+> +except:\r
+> + raise ImportError("Could not find shared 'notmuch' library.")\r
+> [...]\r
+>\r
+> As a heavy LD_LIBRARY_PATH user I don't want to break this. So I tried\r
+> to test whether find_library now respects LD_LIBRARY_PATH or not:\r
+>\r
+> teythoon@thinkbox ~ % echo $LD_LIBRARY_PATH\r
+> /home/teythoon/.local/lib\r
+> teythoon@thinkbox ~ % strace -e trace=open python -c 'import ctypes; ctypes.CDLL("libnotmuch.so.3")' 2>&1 | grep notmuch\r
+> open("/home/teythoon/.local/lib/libnotmuch.so.3", O_RDONLY) = 3\r
+>\r
+> That's how it's done now and indeed it finds my libnotmuch.\r
+>\r
+> teythoon@thinkbox ~ % strace -f -e trace=stat,open python -c 'import ctypes.util; ctypes.util.find_library("notmuch")' 2>&1 | grep notmuch\r
+>\r
+> Nothing. Funny. Let's see:\r
+>\r
+> teythoon@thinkbox ~ % strace -f -e trace=fork,execve,clone python -c 'import ctypes.util; print ctypes.util.find_library("notmuch")'\r
+> execve("/usr/bin/python", ["python", "-c", "import ctypes.util; print ctypes"...], [/* 63 vars */]) = 0\r
+> clone(Process 12000 attached\r
+> child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f0734a649d0) = 12000\r
+> [pid 12000] execve("/bin/sh", ["sh", "-c", "/sbin/ldconfig -p 2>/dev/null"], [/* 63 vars */]) = 0\r
+> [pid 12000] clone(Process 12001 attached\r
+> child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f119d5479d0) = 12001\r
+> [pid 12001] execve("/sbin/ldconfig", ["/sbin/ldconfig", "-p"], [/* 63 vars */]) = 0\r
+> Process 12001 detached\r
+> [pid 12000] --- SIGCHLD (Child exited) @ 0 (0) ---\r
+> Process 12000 detached\r
+> --- SIGCHLD (Child exited) @ 0 (0) ---\r
+> libnotmuch.so.3\r
+>\r
+> So it also prints libnotmuch.so.3, but only because the version\r
+> installed from the Debian archive is also libnotmuch.so.3:\r
+>\r
+> teythoon@thinkbox ~ % /sbin/ldconfig -p | grep notmuch\r
+> libnotmuch.so.3 (libc6,x86-64) => /usr/lib/libnotmuch.so.3\r
+>\r
+> So I guess *if* I had a libnotmuch.so.2 in my ldconfig(8) cache (-p\r
+> prints the cache), find_library would have returned libnotmuch.so.2\r
+> and not my libnotmuch.so.3. So your patch would most likely break this\r
+> kind of setup.\r
+>\r
+> 2. Uh. I actually looked at /usr/lib/python2.7/ctypes/util.py and I\r
+> almost got eye cancer from just looking briefly at it... The\r
+> implementation of find_library differs for various operating systems\r
+> and most (all?) of them use fork/exec some program (even gcc in some\r
+> cases) to look up the libraries. I'd like to avoid this if possible,\r
+> maybe even if this means handling OS X as a special case in our\r
+> bindings. For reference, our current solution does not require any\r
+> fork/exec calls:\r
+>\r
+> teythoon@thinkbox ~ % strace -f -e trace=execve,fork,clone python -c 'import ctypes; ctypes.CDLL("libnotmuch.so.3")' \r
+> execve("/usr/bin/python", ["python", "-c", "import ctypes; ctypes.CDLL(\"libn"...], [/* 63 vars */]) = 0\r
+>\r
+> Thoughts anyone?\r
+\r
+Tried this:\r
+\r
+>>> from ctypes import CDLL\r
+>>> x = CDLL("foobnar.so")\r
+Traceback (most recent call last):\r
+ File "<stdin>", line 1, in <module>\r
+ File "/usr/lib/python2.6/ctypes/__init__.py", line 353, in __init__\r
+ self._handle = _dlopen(self._name, mode)\r
+OSError: foobnar.so: cannot open shared object file: No such file or\r
+directory\r
+\r
+Looked into /usr/lib/python2.6/ctypes/__init__.py -- _dlopen() uses\r
+dlopen() from _ctypes module, which is in \r
+/usr/lib/python2.6/lib-dynload/_ctypes.so\r
+\r
+I'd guess this does pretty much the same as dlopen(3) does in Linux\r
+(or Mac -- whatever it does there)\r
+\r
+Could something like\r
+\r
+try:\r
+ nmlib = CDLL("libnotmuch.so.3")\r
+except OSError:\r
+ try:\r
+ nmlib = CDLL("something_macosx_understands")\r
+ except:\r
+ ImportError("Could not find shared 'notmuch' library.")\r
+\r
+\r
+be used ?\r
+\r
+Just before sending this email I came up with this page:\r
+\r
+https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/dlopen.3.html\r
+\r
+The 'SEARCHING' section has quite a few words how the file is found.\r
+\r
+> Cheers,\r
+> Justus\r
+\r
+\r
+Tomi\r
projects / notmuch-archives.git / commitdiff