From: Jani Nikula Date: Tue, 3 May 2016 16:31:24 +0000 (+0300) Subject: Re: [PATCH] Use the Xapian::DB_RETRY_LOCK flag when available X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=fc7620c395f5c91a54a5b780b79bb856a92488c1;p=notmuch-archives.git Re: [PATCH] Use the Xapian::DB_RETRY_LOCK flag when available --- diff --git a/a6/012fffcac04d0ae7019f31a52a172d0114b8bd b/a6/012fffcac04d0ae7019f31a52a172d0114b8bd new file mode 100644 index 000000000..371e0485b --- /dev/null +++ b/a6/012fffcac04d0ae7019f31a52a172d0114b8bd @@ -0,0 +1,206 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id 345BB6DE01F7 + for ; Tue, 3 May 2016 09:32:57 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.562 +X-Spam-Level: +X-Spam-Status: No, score=-0.562 tagged_above=-999 required=5 tests=[AWL=0.158, + DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, + RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id NFAGvLXyCcbF for ; + Tue, 3 May 2016 09:32:49 -0700 (PDT) +Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com + [74.125.82.68]) by arlo.cworth.org (Postfix) with ESMTPS id A15346DE00DF for + ; Tue, 3 May 2016 09:32:48 -0700 (PDT) +Received: by mail-wm0-f68.google.com with SMTP id n129so4752904wmn.1 + for ; Tue, 03 May 2016 09:32:48 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=nikula-org.20150623.gappssmtp.com; s=20150623; + h=from:to:subject:in-reply-to:references:user-agent:date:message-id + :mime-version; bh=w7NKSYTHtyjmaTWC6Th3leenh5MlW/bZPpTo96BIm1c=; + b=qb47JWfUQamVHyrfQ3h/6+gDqEGSvls+Y7di9irAwtRbgqCz9hxhPeZL9foWM27m51 + h1zZ9uNN6Oltcu+8yriQFPPVbkkkvRVFj3z+cnFc8ER6BUiUEDflEfpn8p7cE6nClaNn + CawvmWeEWf/r8iD3P4/uSScgqy+t0GwgHTsY5I0a7AmfKtC+eko80O3Y63D8KHjqmgl8 + d1PjYoth0gZwU6J63ZHvxFJCJ8Udhc9gxV9z7mV5OXS35mUsDFUU0YwvfyU9t/jznym0 + /eY2UawpL7s0p4Z4f3wp9JLgVFaoVDAy+jSjbBO529G4McJn7p+bdXtbySfQ9EIgUwfi + ecfA== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20130820; + h=x-gm-message-state:from:to:subject:in-reply-to:references + :user-agent:date:message-id:mime-version; + bh=w7NKSYTHtyjmaTWC6Th3leenh5MlW/bZPpTo96BIm1c=; + b=PCNS2wgKW8UpU5Vz/cRDYocEu0tdTiE70itIyL/04hwgVtGinri4hpeSdY77mcSG3u + IKJlilbs7tH6R5evx3FcsIAqmznTaEGXCoQ69iUx0rqe+C0kOgrjL2msOgatW4EK+k9d + ZR6VSmJZ7CEtlY1EtnTeUQTA6BISWh6FzqsmdxK2rlsUSPKZUlUn9vDoirmKcQm6mzM1 + b0vOgh+ctXkL+rB57kO45h/qQmnB652aOJEr5WqFi1Ov21U70dbbvAhRgwyN++UddO0A + k93f5tqS/HuP7yUf7RtqJBzNwH0/Cq8KAWPaA/Ltgb5AYWSa2NthsfxPe1xZVbciXcnM + WbNw== +X-Gm-Message-State: + AOPr4FUfV9JsYRRHqty7NB61QxfrBMOcjemuuiV7GuKAhq1pjN4bL1OGd9JpPSe/X3YLxg== +X-Received: by 10.28.47.206 with SMTP id v197mr4510161wmv.87.1462293167223; + Tue, 03 May 2016 09:32:47 -0700 (PDT) +Received: from localhost (mobile-access-bcee7f-102.dhcp.inet.fi. + [188.238.127.102]) + by smtp.gmail.com with ESMTPSA id jq1sm4769437wjc.28.2016.05.03.09.32.46 + (version=TLSv1/SSLv3 cipher=OTHER); + Tue, 03 May 2016 09:32:46 -0700 (PDT) +From: Jani Nikula +To: Istvan Marko , notmuch@notmuchmail.org +Subject: Re: [PATCH] Use the Xapian::DB_RETRY_LOCK flag when available +In-Reply-To: +References: +User-Agent: Notmuch/0.22+9~g73339ad (http://notmuchmail.org) Emacs/24.4.1 + (x86_64-pc-linux-gnu) +Date: Tue, 03 May 2016 19:31:24 +0300 +Message-ID: <87k2jbgl2b.fsf@nikula.org> +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 03 May 2016 16:32:57 -0000 + +On Tue, 03 May 2016, Istvan Marko wrote: +> Xapian 1.3 has introduced the DB_RETRY_LOCK flag (Xapian bug +> 275). Detect it in configure and use it if available. With this flag +> commands that need the write lock will wait for their turn instead of +> aborting when it's not immediately available. +> --- +> configure | 25 ++++++++++++++++++++++++- +> lib/database.cc | 5 +++++ +> 2 files changed, 29 insertions(+), 1 deletion(-) +> +> diff --git a/configure b/configure +> index 6231d2b..0c1d5bc 100755 +> --- a/configure +> +++ b/configure +> @@ -371,6 +371,21 @@ if [ ${have_xapian} = "1" ]; then +> esac +> fi +> +> +# DB_RETRY_LOCK is only supported on Xapian > 1.3.2 +> +have_xapian_db_retry_lock=0 +> +if [ ${have_xapian} = "1" ]; then +> + printf "Checking for Xapian lock retry support... " +> + case "${xapian_version}" in +> + 0.*|1.[012].*|1.3.[0-2]) +> + printf "No (only available with Xapian > 1.3.2).\n" ;; +> + [1-9]*.[0-9]*.[0-9]*) +> + have_xapian_db_retry_lock=1 +> + printf "Yes.\n" ;; +> + *) +> + printf "Unknown version.\n" ;; +> + esac +> +fi +> + +> default_xapian_backend="" +> if [ ${have_xapian} = "1" ]; then +> printf "Testing default Xapian backend... " +> @@ -998,6 +1013,9 @@ HAVE_D_TYPE = ${have_d_type} +> # Whether the Xapian version in use supports compaction +> HAVE_XAPIAN_COMPACT = ${have_xapian_compact} +> +> +# Whether the Xapian version in use supports DB_RETRY_LOCK +> +HAVE_XAPIAN_DB_RETRY_LOCK = ${have_xapian_db_retry_lock} +> + +> # Whether the getpwuid_r function is standards-compliant +> # (if not, then notmuch will #define _POSIX_PTHREAD_SEMANTICS +> # to enable the standards-compliant version -- needed for Solaris) +> @@ -1072,6 +1090,7 @@ CONFIGURE_CFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ +> -DSTD_GETPWUID=\$(STD_GETPWUID) \\ +> -DSTD_ASCTIME=\$(STD_ASCTIME) \\ +> -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT) \\ +> + -DHAVE_XAPIAN_DB_RETRY_LOCK=\$(HAVE_XAPIAN_DB_RETRY_LOCK) \\ +> -DUTIL_BYTE_ORDER=\$(UTIL_BYTE_ORDER) +> +> CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ +> @@ -1086,6 +1105,7 @@ CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\ +> -DSTD_GETPWUID=\$(STD_GETPWUID) \\ +> -DSTD_ASCTIME=\$(STD_ASCTIME) \\ +> -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT) \\ +> + -DHAVE_XAPIAN_DB_RETRY_LOCK=\$(HAVE_XAPIAN_DB_RETRY_LOCK) \\ +> -DUTIL_BYTE_ORDER=\$(UTIL_BYTE_ORDER) +> +> CONFIGURE_LDFLAGS = \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(ZLIB_LDFLAGS) \$(XAPIAN_LDFLAGS) +> @@ -1097,7 +1117,10 @@ cat > sh.config < # script of notmuch. +> +> # Whether the Xapian version in use supports compaction +> -NOTMUCH_HAVE_XAPIAN_COMPACT=${have_xapian_compact} +> +NOTMUCH_HAVE_XAPIAN_ =${have_xapian_compact} + +Ooops? + +> + +> +# Whether the Xapian version in use supports DB_RETRY_LOCK +> +NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK=${have_xapian_db_retry_lock} + +You don't need to update sh.config unless you need this in a shell +script, typically in the tests. Which brings us to the question, how can +you test this? + +> +> # Which backend will Xapian use by default? +> NOTMUCH_DEFAULT_XAPIAN_BACKEND=${default_xapian_backend} +> diff --git a/lib/database.cc b/lib/database.cc +> index c8c5e26..4b503a2 100644 +> --- a/lib/database.cc +> +++ b/lib/database.cc +> @@ -929,8 +929,13 @@ notmuch_database_open_verbose (const char *path, +> string last_mod; +> +> if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE) { +> + #if HAVE_XAPIAN_DB_RETRY_LOCK +> + notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path, +> + Xapian::DB_CREATE_OR_OPEN|Xapian::DB_RETRY_LOCK); +> + #else +> notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path, +> Xapian::DB_CREATE_OR_OPEN); +> + #endif + +Please no #ifdefs like this in the middle of the code. + +A better alternative is to define something like this above the +function: + +#if HAVE_XAPIAN_DB_RETRY_LOCK +#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK) +#else +#define DB_ACTION Xapian::DB_CREATE_OR_OPEN +#endif + +and use that in new Xapian::WritableDatabase(). + + +BR, +Jani. + + +> } else { +> notmuch->xapian_db = new Xapian::Database (xapian_path); +> } +> -- +> 2.4.10 +> +> +> -- +> Istvan +> _______________________________________________ +> notmuch mailing list +> notmuch@notmuchmail.org +> https://notmuchmail.org/mailman/listinfo/notmuch