From: Jed Brown Date: Tue, 9 Apr 2013 14:57:05 +0000 (+1900) Subject: Re: [RFC/PATCH] python: search parent lib directory for libnotmuch.so X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=3cfef71417ca4a3761d74af3e7f4d1c7dbec2b7f;p=notmuch-archives.git Re: [RFC/PATCH] python: search parent lib directory for libnotmuch.so --- diff --git a/45/b51aed6dfe59939a7c991d92b610db354a4c41 b/45/b51aed6dfe59939a7c991d92b610db354a4c41 new file mode 100644 index 000000000..e6e87d3c4 --- /dev/null +++ b/45/b51aed6dfe59939a7c991d92b610db354a4c41 @@ -0,0 +1,148 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id CB32C431FBC + for ; Tue, 9 Apr 2013 07:57:08 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: 0.301 +X-Spam-Level: +X-Spam-Status: No, score=0.301 tagged_above=-999 required=5 + tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=1, + FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 2X54zkeGnGSx for ; + Tue, 9 Apr 2013 07:57:08 -0700 (PDT) +Received: from mail-ie0-f177.google.com (mail-ie0-f177.google.com + [209.85.223.177]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 33E79431FAE + for ; Tue, 9 Apr 2013 07:57:08 -0700 (PDT) +Received: by mail-ie0-f177.google.com with SMTP id tp5so8620857ieb.36 + for ; Tue, 09 Apr 2013 07:57:07 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; + h=x-received:sender:from:to:cc:subject:in-reply-to:references + :user-agent:date:message-id:mime-version:content-type; + bh=23WQoIPTgSk0tuJozkAUgP5/DJEnd+i86uLP3bjQiXs=; + b=YwC5qlK/ezEGSPAZE0WOg1G+bVNw7tWd20e5h51Qs3MaBXgEqvFZ/3xLtmzBNTTJIh + NNfj1HSNCITlh7+3MoIlh4ciZXIlVm7SWZYgiUfRVonEhRr5/YFQsV3+tD6AvvJIEJXr + ABOTE33GIjhqkeM6h+0PUc6Ywi6tV8+u7+awyLcdErDiQygNgzjmWrIO9cg42rqGxGJ+ + VZWBiiKdvAI2ETOE1iaSdu/2ChOkX4Iousu0NAzKRwI1Z5zxfMOLI/wvtXmBPI9InVS0 + 0i0gwHMnLPHrcl20B5U72siFYEo9Q2pg6DVHkb1rs8lsie0FY5VRjgywFIeTJk3DhWiy + brDQ== +X-Received: by 10.50.7.69 with SMTP id h5mr10905593iga.69.1365519426923; + Tue, 09 Apr 2013 07:57:06 -0700 (PDT) +Received: from localhost (vis-v410v070.mcs.anl-external.org. [130.202.17.70]) + by mx.google.com with ESMTPS id + ip2sm22891004igc.5.2013.04.09.07.57.05 + (version=TLSv1.2 cipher=RC4-SHA bits=128/128); + Tue, 09 Apr 2013 07:57:06 -0700 (PDT) +Sender: Jed Brown +From: Jed Brown +To: Justus Winter <4winter@informatik.uni-hamburg.de>, notmuch@notmuchmail.org +Subject: Re: [RFC/PATCH] python: search parent lib directory for libnotmuch.so +In-Reply-To: <20130409141333.7736.41695@thinkbox.jade-hamburg.de> +References: <1365475646-22926-1-git-send-email-jed@59A2.org> + <20130409141333.7736.41695@thinkbox.jade-hamburg.de> +User-Agent: Notmuch/0.15.2+78~g5404ac5 (http://notmuchmail.org) Emacs/24.3.1 + (x86_64-unknown-linux-gnu) +Date: Tue, 09 Apr 2013 09:57:05 -0500 +Message-ID: <87obdn7nwe.fsf@mcs.anl.gov> +MIME-Version: 1.0 +Content-Type: text/plain +Cc: Sebastian Spaeth +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 09 Apr 2013 14:57:09 -0000 + +Justus Winter <4winter@informatik.uni-hamburg.de> writes: +> +> May I ask why you cannot use LD_LIBRARY_PATH? I too install libnotmuch +> to a non-standard location as unprivileged user and to make this +> library available I add its path to LD_LIBRARY_PATH. + +See libdir_in_ldconfig testing in configure: we make a significant +effort to set RPATH appropriately when installing to a location that is +not already searched (perhaps via LD_LIBRARY_PATH). This currently does +not apply to the Python bindings, so while you can install without +LD_LIBRARY_PATH and still run the notmuch executable fine, you must set +LD_LIBRARY_PATH to use the Python bindings. That is the inconsistency I +wanted to fix here. + +>> This is sort of a hack, but I don't know a less intrusive way to get +>> libnotmuch.so from somewhere dlopen(3) doesn't already search. +>> +>> The absolute path version won't do the right thing in case of 'setup.py +>> develop', otherwise we could use it in all cases. It may still make +>> sense to make the absolute path version take precedence. +> +> Well, if something like this is necessary and wanted (opinions +> anyone?) at least let's not hardcode the assumption about the +> directory layout but just walk up the tree until we find notmuch.so.3 +> or lib/notmuch.so.3. This way the bindings will find the correct +> library even when they are included directly from within the source +> tree. + +I actually wrote the more permissive version below, then decided I +preferred the stricter behavior because there was less chance of +accidentally finding a stale libnotmuch.so.3. Note that in the source +tree, notmuch-shared already has RPATH pointing to the install location +so it's not valid without install. The strict version of my patch has +similar behavior in that Python bindings are only valid when installed. +If you want to run them from the source tree, you'd have to add +/path/to/notmuch/lib to LD_LIBRARY_PATH. + +diff --git a/bindings/python/notmuch/globals.py b/bindings/python/notmuch/globals.py +index c7632c3..2fd383f 100644 +--- a/bindings/python/notmuch/globals.py ++++ b/bindings/python/notmuch/globals.py +@@ -24,7 +24,16 @@ from ctypes import CDLL, Structure, POINTER + try: + nmlib = CDLL("libnotmuch.so.3") + except: +- raise ImportError("Could not find shared 'notmuch' library.") ++ import os.path ++ path = os.path.abspath(__file__) ++ while True: ++ path = os.path.dirname(path) ++ try: ++ nmlib = CDLL(os.path.join(path, 'libnotmuch.so.3')) ++ break ++ except: ++ if path == '/': ++ raise ImportError("Could not find shared 'notmuch' library.") + + from .compat import Python3StringMixIn, encode_utf8 as _str + + +> Otoh, adding such behavior might be 'surprising' and lead to many +> problems down the road like spurious bug reports just because the +> magic library finder locates a rogue libnotmuch.so.3 somewhere. +> +>> An alternative would be to find libnotmuch.so using the notmuch +>> executable. +> +> How would you do that? fork'ing and exec'ing is not an option (and I'm +> not sure what one could achieve by exec'ing it, but how else would you +> talk to the dynamic linker?) , and poking around in binaries to get +> their rpath isn't either in my opinion. And you would have to locate +> the 'right' binary in the first place? + +I don't like the indirection either, but the binary is compiled with +knowledge of prefix/RPATH, so if we wanted a single canonical location +to specify this information, I would make it the binary. + +If you don't want to trust Python install directory hierarchy, we could +have 'setup.py install' write some info about RPATH.