Re: [PATCH] Use the Xapian::DB_RETRY_LOCK flag when available
authorJani Nikula <jani@nikula.org>
Tue, 3 May 2016 16:31:24 +0000 (19:31 +0300)
committerW. Trevor King <wking@tremily.us>
Sat, 20 Aug 2016 23:21:45 +0000 (16:21 -0700)
a6/012fffcac04d0ae7019f31a52a172d0114b8bd [new file with mode: 0644]

diff --git a/a6/012fffcac04d0ae7019f31a52a172d0114b8bd b/a6/012fffcac04d0ae7019f31a52a172d0114b8bd
new file mode 100644 (file)
index 0000000..371e048
--- /dev/null
@@ -0,0 +1,206 @@
+Return-Path: <jani@nikula.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 345BB6DE01F7\r
+ for <notmuch@notmuchmail.org>; Tue,  3 May 2016 09:32:57 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.562\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.562 tagged_above=-999 required=5 tests=[AWL=0.158,\r
+  DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7,\r
+ RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id NFAGvLXyCcbF for <notmuch@notmuchmail.org>;\r
+ Tue,  3 May 2016 09:32:49 -0700 (PDT)\r
+Received: from mail-wm0-f68.google.com (mail-wm0-f68.google.com\r
+ [74.125.82.68]) by arlo.cworth.org (Postfix) with ESMTPS id A15346DE00DF for\r
+ <notmuch@notmuchmail.org>; Tue,  3 May 2016 09:32:48 -0700 (PDT)\r
+Received: by mail-wm0-f68.google.com with SMTP id n129so4752904wmn.1\r
+ for <notmuch@notmuchmail.org>; Tue, 03 May 2016 09:32:48 -0700 (PDT)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=nikula-org.20150623.gappssmtp.com; s=20150623;\r
+ h=from:to:subject:in-reply-to:references:user-agent:date:message-id\r
+ :mime-version; bh=w7NKSYTHtyjmaTWC6Th3leenh5MlW/bZPpTo96BIm1c=;\r
+ b=qb47JWfUQamVHyrfQ3h/6+gDqEGSvls+Y7di9irAwtRbgqCz9hxhPeZL9foWM27m51\r
+ h1zZ9uNN6Oltcu+8yriQFPPVbkkkvRVFj3z+cnFc8ER6BUiUEDflEfpn8p7cE6nClaNn\r
+ CawvmWeEWf/r8iD3P4/uSScgqy+t0GwgHTsY5I0a7AmfKtC+eko80O3Y63D8KHjqmgl8\r
+ d1PjYoth0gZwU6J63ZHvxFJCJ8Udhc9gxV9z7mV5OXS35mUsDFUU0YwvfyU9t/jznym0\r
+ /eY2UawpL7s0p4Z4f3wp9JLgVFaoVDAy+jSjbBO529G4McJn7p+bdXtbySfQ9EIgUwfi\r
+ ecfA==\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:from:to:subject:in-reply-to:references\r
+ :user-agent:date:message-id:mime-version;\r
+ bh=w7NKSYTHtyjmaTWC6Th3leenh5MlW/bZPpTo96BIm1c=;\r
+ b=PCNS2wgKW8UpU5Vz/cRDYocEu0tdTiE70itIyL/04hwgVtGinri4hpeSdY77mcSG3u\r
+ IKJlilbs7tH6R5evx3FcsIAqmznTaEGXCoQ69iUx0rqe+C0kOgrjL2msOgatW4EK+k9d\r
+ ZR6VSmJZ7CEtlY1EtnTeUQTA6BISWh6FzqsmdxK2rlsUSPKZUlUn9vDoirmKcQm6mzM1\r
+ b0vOgh+ctXkL+rB57kO45h/qQmnB652aOJEr5WqFi1Ov21U70dbbvAhRgwyN++UddO0A\r
+ k93f5tqS/HuP7yUf7RtqJBzNwH0/Cq8KAWPaA/Ltgb5AYWSa2NthsfxPe1xZVbciXcnM\r
+ WbNw==\r
+X-Gm-Message-State:\r
+ AOPr4FUfV9JsYRRHqty7NB61QxfrBMOcjemuuiV7GuKAhq1pjN4bL1OGd9JpPSe/X3YLxg==\r
+X-Received: by 10.28.47.206 with SMTP id v197mr4510161wmv.87.1462293167223;\r
+ Tue, 03 May 2016 09:32:47 -0700 (PDT)\r
+Received: from localhost (mobile-access-bcee7f-102.dhcp.inet.fi.\r
+ [188.238.127.102])\r
+ by smtp.gmail.com with ESMTPSA id jq1sm4769437wjc.28.2016.05.03.09.32.46\r
+ (version=TLSv1/SSLv3 cipher=OTHER);\r
+ Tue, 03 May 2016 09:32:46 -0700 (PDT)\r
+From: Jani Nikula <jani@nikula.org>\r
+To: Istvan Marko <notmuch@kismala.com>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] Use the Xapian::DB_RETRY_LOCK flag when available\r
+In-Reply-To: <m38tzr2n6v.fsf@zsu.kismala.com>\r
+References: <m38tzr2n6v.fsf@zsu.kismala.com>\r
+User-Agent: Notmuch/0.22+9~g73339ad (http://notmuchmail.org) Emacs/24.4.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Tue, 03 May 2016 19:31:24 +0300\r
+Message-ID: <87k2jbgl2b.fsf@nikula.org>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 03 May 2016 16:32:57 -0000\r
+\r
+On Tue, 03 May 2016, Istvan Marko <notmuch@kismala.com> wrote:\r
+> Xapian 1.3 has introduced the DB_RETRY_LOCK flag (Xapian bug\r
+> 275). Detect it in configure and use it if available. With this flag\r
+> commands that need the write lock will wait for their turn instead of\r
+> aborting when it's not immediately available.\r
+> ---\r
+>  configure       | 25 ++++++++++++++++++++++++-\r
+>  lib/database.cc |  5 +++++\r
+>  2 files changed, 29 insertions(+), 1 deletion(-)\r
+>\r
+> diff --git a/configure b/configure\r
+> index 6231d2b..0c1d5bc 100755\r
+> --- a/configure\r
+> +++ b/configure\r
+> @@ -371,6 +371,21 @@ if [ ${have_xapian} = "1" ]; then\r
+>      esac\r
+>  fi\r
+>  \r
+> +# DB_RETRY_LOCK is only supported on Xapian > 1.3.2\r
+> +have_xapian_db_retry_lock=0\r
+> +if [ ${have_xapian} = "1" ]; then\r
+> +    printf "Checking for Xapian lock retry support... "\r
+> +    case "${xapian_version}" in\r
+> +    0.*|1.[012].*|1.3.[0-2])\r
+> +        printf "No (only available with Xapian > 1.3.2).\n" ;;\r
+> +    [1-9]*.[0-9]*.[0-9]*)\r
+> +        have_xapian_db_retry_lock=1\r
+> +        printf "Yes.\n" ;;\r
+> +    *)\r
+> +        printf "Unknown version.\n" ;;\r
+> +    esac\r
+> +fi\r
+> +\r
+>  default_xapian_backend=""\r
+>  if [ ${have_xapian} = "1" ]; then\r
+>      printf "Testing default Xapian backend... "\r
+> @@ -998,6 +1013,9 @@ HAVE_D_TYPE = ${have_d_type}\r
+>  # Whether the Xapian version in use supports compaction\r
+>  HAVE_XAPIAN_COMPACT = ${have_xapian_compact}\r
+>  \r
+> +# Whether the Xapian version in use supports DB_RETRY_LOCK\r
+> +HAVE_XAPIAN_DB_RETRY_LOCK = ${have_xapian_db_retry_lock}\r
+> +\r
+>  # Whether the getpwuid_r function is standards-compliant\r
+>  # (if not, then notmuch will #define _POSIX_PTHREAD_SEMANTICS\r
+>  # to enable the standards-compliant version -- needed for Solaris)\r
+> @@ -1072,6 +1090,7 @@ CONFIGURE_CFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS)      \\\r
+>                 -DSTD_GETPWUID=\$(STD_GETPWUID)                       \\\r
+>                 -DSTD_ASCTIME=\$(STD_ASCTIME)                         \\\r
+>                 -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT)         \\\r
+> +               -DHAVE_XAPIAN_DB_RETRY_LOCK=\$(HAVE_XAPIAN_DB_RETRY_LOCK) \\\r
+>                 -DUTIL_BYTE_ORDER=\$(UTIL_BYTE_ORDER)\r
+>  \r
+>  CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS)    \\\r
+> @@ -1086,6 +1105,7 @@ CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS)    \\\r
+>                   -DSTD_GETPWUID=\$(STD_GETPWUID)                     \\\r
+>                   -DSTD_ASCTIME=\$(STD_ASCTIME)                       \\\r
+>                   -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT)       \\\r
+> +                     -DHAVE_XAPIAN_DB_RETRY_LOCK=\$(HAVE_XAPIAN_DB_RETRY_LOCK) \\\r
+>                   -DUTIL_BYTE_ORDER=\$(UTIL_BYTE_ORDER)\r
+>  \r
+>  CONFIGURE_LDFLAGS =  \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(ZLIB_LDFLAGS) \$(XAPIAN_LDFLAGS)\r
+> @@ -1097,7 +1117,10 @@ cat > sh.config <<EOF\r
+>  # script of notmuch.\r
+>  \r
+>  # Whether the Xapian version in use supports compaction\r
+> -NOTMUCH_HAVE_XAPIAN_COMPACT=${have_xapian_compact}\r
+> +NOTMUCH_HAVE_XAPIAN_  =${have_xapian_compact}\r
+\r
+Ooops?\r
+\r
+> +\r
+> +# Whether the Xapian version in use supports DB_RETRY_LOCK\r
+> +NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK=${have_xapian_db_retry_lock}\r
+\r
+You don't need to update sh.config unless you need this in a shell\r
+script, typically in the tests. Which brings us to the question, how can\r
+you test this?\r
+\r
+>  \r
+>  # Which backend will Xapian use by default?\r
+>  NOTMUCH_DEFAULT_XAPIAN_BACKEND=${default_xapian_backend}\r
+> diff --git a/lib/database.cc b/lib/database.cc\r
+> index c8c5e26..4b503a2 100644\r
+> --- a/lib/database.cc\r
+> +++ b/lib/database.cc\r
+> @@ -929,8 +929,13 @@ notmuch_database_open_verbose (const char *path,\r
+>      string last_mod;\r
+>  \r
+>      if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE) {\r
+> +        #if HAVE_XAPIAN_DB_RETRY_LOCK\r
+> +        notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path,\r
+> +                                                           Xapian::DB_CREATE_OR_OPEN|Xapian::DB_RETRY_LOCK);\r
+> +        #else\r
+>          notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path,\r
+>                                                             Xapian::DB_CREATE_OR_OPEN);\r
+> +        #endif\r
+\r
+Please no #ifdefs like this in the middle of the code.\r
+\r
+A better alternative is to define something like this above the\r
+function:\r
+\r
+#if HAVE_XAPIAN_DB_RETRY_LOCK\r
+#define DB_ACTION      (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK)\r
+#else\r
+#define DB_ACTION      Xapian::DB_CREATE_OR_OPEN\r
+#endif\r
+\r
+and use that in new Xapian::WritableDatabase().\r
+\r
+\r
+BR,\r
+Jani.\r
+\r
+\r
+>      } else {\r
+>          notmuch->xapian_db = new Xapian::Database (xapian_path);\r
+>      }\r
+> -- \r
+> 2.4.10\r
+>\r
+>\r
+> -- \r
+>      Istvan\r
+> _______________________________________________\r
+> notmuch mailing list\r
+> notmuch@notmuchmail.org\r
+> https://notmuchmail.org/mailman/listinfo/notmuch\r