From: Austin Clements Date: Tue, 29 Jul 2014 16:47:58 +0000 (+2000) Subject: [PATCH v2 00/14] Implement and use database "features" X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=9752a1846e5dee176ff318e503553e186c47fda5;p=notmuch-archives.git [PATCH v2 00/14] Implement and use database "features" --- diff --git a/55/223aace8b62c8544b5143eb840ded70a24237d b/55/223aace8b62c8544b5143eb840ded70a24237d new file mode 100644 index 000000000..32d3d2fd7 --- /dev/null +++ b/55/223aace8b62c8544b5143eb840ded70a24237d @@ -0,0 +1,267 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 15DB7431E62 + for ; Tue, 29 Jul 2014 09:48:36 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -2.3 +X-Spam-Level: +X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 8XxPaj8Gw+Dd for ; + Tue, 29 Jul 2014 09:48:27 -0700 (PDT) +Received: from dmz-mailsec-scanner-3.mit.edu (dmz-mailsec-scanner-3.mit.edu + [18.9.25.14]) + (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 33CB6431FC3 + for ; Tue, 29 Jul 2014 09:48:21 -0700 (PDT) +X-AuditID: 1209190e-f79946d000007db1-be-53d7d052b4c2 +Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) + (using TLS with cipher AES256-SHA (256/256 bits)) + (Client did not present a certificate) + by dmz-mailsec-scanner-3.mit.edu (Symantec Messaging Gateway) with SMTP + id B8.F5.32177.350D7D35; Tue, 29 Jul 2014 12:48:19 -0400 (EDT) +Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11]) + by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s6TGmHM2003473; + Tue, 29 Jul 2014 12:48:18 -0400 +Received: from drake.dyndns.org (31-33-71.wireless.csail.mit.edu + [128.31.33.71]) (authenticated bits=0) + (User authenticated as amdragon@ATHENA.MIT.EDU) + by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6TGmFNk030242 + (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=NOT); + Tue, 29 Jul 2014 12:48:17 -0400 +Received: from amthrax by drake.dyndns.org with local (Exim 4.77) + (envelope-from ) + id 1XCAZS-0007GZ-Ky; Tue, 29 Jul 2014 12:48:14 -0400 +From: Austin Clements +To: notmuch@notmuchmail.org +Subject: [PATCH v2 00/14] Implement and use database "features" +Date: Tue, 29 Jul 2014 12:47:58 -0400 +Message-Id: <1406652492-27803-1-git-send-email-amdragon@mit.edu> +X-Mailer: git-send-email 2.0.0 +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFtrIIsWRmVeSWpSXmKPExsUixCmqrBt84XqwQf8JdovVc3ksrt+cyezA + 5LFz1l12j2erbjEHMEVx2aSk5mSWpRbp2yVwZexvecZYcM66Yv2kk+wNjG2GXYycHBICJhIN + 87eyQNhiEhfurWfrYuTiEBKYzSQxacU9JghnI6NE6+XNUM4xJok5y6ZAOXMZJRpmtLGB9LMJ + aEhs27+cEcQWEZCW2Hl3NiuIzSwgK/Hj/B8wW1jAXuLzur9g9SwCqhINEz+wg9i8Ag4SG69+ + Y4W4Q06i4cYntgmMvAsYGVYxyqbkVunmJmbmFKcm6xYnJ+blpRbpGuvlZpbopaaUbmIEh4Yk + 3w7GrweVDjEKcDAq8fBumHstWIg1say4MvcQoyQHk5Ior96+68FCfEn5KZUZicUZ8UWlOanF + hxglOJiVRHj5zgLleFMSK6tSi/JhUtIcLErivG+trYKFBNITS1KzU1MLUotgsjIcHEoSvK/P + AzUKFqWmp1akZeaUIKSZODhBhvMADd8HUsNbXJCYW5yZDpE/xajL8aH3TBuTEEtefl6qlDjv + JpAiAZCijNI8uDmwmH7FKA70ljDvQZAqHmA6gJv0CmgJE9ASVhewJSWJCCmpBkY+o9IvuUzm + IU47p2ld6sxYpKvJcZi5XOF0VJ9wyKw51TWBmbs2PKqfvH8Ot9tUOxNhi8KAZrdlYVoTlS/c + 1mD9Nj/amHNB1kaTDLWwQ4d7/jSLm1+PrLU+ulbiydnwEwtkwqP3H5+29e2XJbmSkQclkz6z + zjffGD+xZbGS+84v/47ytm3cdESJpTgj0VCLuag4EQBdjj2VxAIAAA== +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 29 Jul 2014 16:48:36 -0000 + +This is version 2 of +id:1406433173-19169-1-git-send-email-amdragon@mit.edu. It fixes a bug +pointed out by Mark [1] where the upgrade test would fail if the +prerequisites were not available (rather than cleanly skipping the +test like before). It also makes feature names more user-friendly and +eliminates notmuch->needs_upgrade in favor of computing it on-the-fly +in notmuch_database_needs_upgrade. The diff from v1 is below. + +[1] id:871tt718xp.fsf@qmul.ac.uk + +diff --git a/lib/database-private.h b/lib/database-private.h +index 323b9fe..2ffab33 100644 +--- a/lib/database-private.h ++++ b/lib/database-private.h +@@ -41,7 +41,6 @@ struct _notmuch_database { + + char *path; + +- notmuch_bool_t needs_upgrade; + notmuch_database_mode_t mode; + int atomic_nesting; + Xapian::Database *xapian_db; +diff --git a/lib/database.cc b/lib/database.cc +index f105e27..b323691 100644 +--- a/lib/database.cc ++++ b/lib/database.cc +@@ -238,7 +238,6 @@ static prefix_t PROBABILISTIC_PREFIX[]= { + { "subject", "XSUBJECT"}, + }; + +- + const char * + _find_prefix (const char *name) + { +@@ -275,12 +274,12 @@ static const struct + /* Compatibility flags when this feature is declared. */ + const char *flags; + } feature_names[] = { +- {NOTMUCH_FEATURE_FILE_TERMS, "file terms", "rw"}, +- {NOTMUCH_FEATURE_DIRECTORY_DOCS, "directory documents", "rw"}, ++ {NOTMUCH_FEATURE_FILE_TERMS, "multiple paths per message", "rw"}, ++ {NOTMUCH_FEATURE_DIRECTORY_DOCS, "relative directory paths", "rw"}, + /* Header values are not required for reading a database because a + * reader can just refer to the message file. */ +- {NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES, "from/subject/ID values", "w"}, +- {NOTMUCH_FEATURE_BOOL_FOLDER, "boolean folder terms", "rw"}, ++ {NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES, "from/subject/message-ID in database", "w"}, ++ {NOTMUCH_FEATURE_BOOL_FOLDER, "exact folder:/path: search", "rw"}, + }; + + const char * +@@ -790,7 +789,6 @@ notmuch_database_open (const char *path, + if (notmuch->path[strlen (notmuch->path) - 1] == '/') + notmuch->path[strlen (notmuch->path) - 1] = '\0'; + +- notmuch->needs_upgrade = FALSE; + notmuch->mode = mode; + notmuch->atomic_nesting = 0; + try { +@@ -839,11 +837,6 @@ notmuch_database_open (const char *path, + goto DONE; + } + +- /* Do we want an upgrade? */ +- if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE && +- NOTMUCH_FEATURES_CURRENT & ~notmuch->features) +- notmuch->needs_upgrade = TRUE; +- + notmuch->last_doc_id = notmuch->xapian_db->get_lastdocid (); + last_thread_id = notmuch->xapian_db->get_metadata ("last_thread_id"); + if (last_thread_id.empty ()) { +@@ -1173,7 +1166,8 @@ notmuch_database_get_version (notmuch_database_t *notmuch) + notmuch_bool_t + notmuch_database_needs_upgrade (notmuch_database_t *notmuch) + { +- return notmuch->needs_upgrade; ++ return notmuch->mode == NOTMUCH_DATABASE_MODE_READ_WRITE && ++ (NOTMUCH_FEATURES_CURRENT & ~notmuch->features); + } + + static volatile sig_atomic_t do_progress_notify = 0; +diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh +index c88bbc7..c4c4ac8 100755 +--- a/test/T530-upgrade.sh ++++ b/test/T530-upgrade.sh +@@ -3,52 +3,6 @@ test_description="database upgrade" + + . ./test-lib.sh + +-test_begin_subtest "future database versions abort open" +-${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 9999 "" +-output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') +-rm -rf ${MAIL_DIR}/.notmuch +-test_expect_equal "$output" "\ +-Error: Notmuch database at FILENAME +- has a newer database format version (9999) than supported by this +- version of notmuch (3)." +- +-test_begin_subtest "unknown 'rw' feature aborts read/write open" +-${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\trw' +-output=$(notmuch new 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') +-rm -rf ${MAIL_DIR}/.notmuch +-test_expect_equal "$output" "\ +-Error: Notmuch database at FILENAME +- requires features (test feature) +- not supported by this version of notmuch." +- +-test_begin_subtest "unknown 'rw' feature aborts read-only open" +-${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\trw' +-output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') +-rm -rf ${MAIL_DIR}/.notmuch +-test_expect_equal "$output" "\ +-Error: Notmuch database at FILENAME +- requires features (test feature) +- not supported by this version of notmuch." +- +-test_begin_subtest "unknown 'w' feature aborts read/write open" +-${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\tw' +-output=$(notmuch new 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') +-rm -rf ${MAIL_DIR}/.notmuch +-test_expect_equal "$output" "\ +-Error: Notmuch database at FILENAME +- requires features (test feature) +- not supported by this version of notmuch." +- +-test_begin_subtest "unknown 'w' feature does not abort read-only open" +-${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\tw' +-output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') +-rm -rf ${MAIL_DIR}/.notmuch +-test_expect_equal "$output" "" +- +-# +-# Database v1 +-# +- + dbtarball=database-v1.tar.xz + + # XXX: Accomplish the same with test lib helpers +diff --git a/test/T550-db-features.sh b/test/T550-db-features.sh +new file mode 100755 +index 0000000..5569768 +--- /dev/null ++++ b/test/T550-db-features.sh +@@ -0,0 +1,48 @@ ++#!/usr/bin/env bash ++test_description="database version and feature compatibility" ++ ++. ./test-lib.sh ++ ++test_begin_subtest "future database versions abort open" ++${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 9999 "" ++output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') ++rm -rf ${MAIL_DIR}/.notmuch ++test_expect_equal "$output" "\ ++Error: Notmuch database at FILENAME ++ has a newer database format version (9999) than supported by this ++ version of notmuch (3)." ++ ++test_begin_subtest "unknown 'rw' feature aborts read/write open" ++${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\trw' ++output=$(notmuch new 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') ++rm -rf ${MAIL_DIR}/.notmuch ++test_expect_equal "$output" "\ ++Error: Notmuch database at FILENAME ++ requires features (test feature) ++ not supported by this version of notmuch." ++ ++test_begin_subtest "unknown 'rw' feature aborts read-only open" ++${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\trw' ++output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') ++rm -rf ${MAIL_DIR}/.notmuch ++test_expect_equal "$output" "\ ++Error: Notmuch database at FILENAME ++ requires features (test feature) ++ not supported by this version of notmuch." ++ ++test_begin_subtest "unknown 'w' feature aborts read/write open" ++${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\tw' ++output=$(notmuch new 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') ++rm -rf ${MAIL_DIR}/.notmuch ++test_expect_equal "$output" "\ ++Error: Notmuch database at FILENAME ++ requires features (test feature) ++ not supported by this version of notmuch." ++ ++test_begin_subtest "unknown 'w' feature does not abort read-only open" ++${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 3 $'test feature\tw' ++output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') ++rm -rf ${MAIL_DIR}/.notmuch ++test_expect_equal "$output" "" ++ ++test_done + +