From: Tomi Ollila <tomi.ollila@iki.fi>
To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org
Subject: Re: [PATCH 3/6] lib: API to retrieve database revision and UUID
In-Reply-To: <1433525318-23756-4-git-send-email-david@tethera.net>
References: <1432936375-astroid-4-0i1n6yczs2-1520@strange>
 <1433525318-23756-1-git-send-email-david@tethera.net>
 <1433525318-23756-4-git-send-email-david@tethera.net>
User-Agent: Notmuch/0.20.2+42~gbf3bfb8 (http://notmuchmail.org) Emacs/24.3.1
 (x86_64-unknown-linux-gnu)
Date: Thu, 06 Aug 2015 13:26:51 +0300
Message-ID: <m2lhdoemqc.fsf@guru.guru-group.fi>
MIME-Version: 1.0
Content-Type: text/plain
Cc: Austin Clements <aclements@csail.mit.edu>
Precedence: list

On Fri, Jun 05 2015, David Bremner <david@tethera.net> wrote:

> From: Austin Clements <aclements@csail.mit.edu>
>
> This exposes the committed database revision to library users along
> with a UUID that can be used to detect when revision numbers are no
> longer comparable (e.g., because the database has been replaced).
> ---
>  lib/database-private.h         |  1 +
>  lib/database.cc                | 11 +++++++++++
>  lib/notmuch.h                  | 18 ++++++++++++++++++
>  test/T570-revision-tracking.sh | 37 +++++++++++++++++++++++++++++++++++++
>  test/test-lib.sh               |  5 +++++
>  5 files changed, 72 insertions(+)
>  create mode 100755 test/T570-revision-tracking.sh
>
> diff --git a/lib/database-private.h b/lib/database-private.h
> index 5c5a2bb..4e93257 100644
> --- a/lib/database-private.h
> +++ b/lib/database-private.h
> @@ -170,6 +170,7 @@ struct _notmuch_database {
>       * under a higher revision number, which can be generated with
>       * notmuch_database_new_revision. */
>      unsigned long revision;
> +    const char *uuid;
>  
>      Xapian::QueryParser *query_parser;
>      Xapian::TermGenerator *term_gen;
> diff --git a/lib/database.cc b/lib/database.cc
> index a68a487..ba8b8d9 100644
> --- a/lib/database.cc
> +++ b/lib/database.cc
> @@ -978,6 +978,8 @@ notmuch_database_open_verbose (const char *path,
>  	    notmuch->revision = 0;
>  	else
>  	    notmuch->revision = Xapian::sortable_unserialise (last_mod);
> +	notmuch->uuid = talloc_strdup (
> +	    notmuch, notmuch->xapian_db->get_uuid ().c_str ());
>  
>  	notmuch->query_parser = new Xapian::QueryParser;
>  	notmuch->term_gen = new Xapian::TermGenerator;
> @@ -1651,6 +1653,15 @@ DONE:
>      return NOTMUCH_STATUS_SUCCESS;
>  }
>  
> +unsigned long
> +notmuch_database_get_revision (notmuch_database_t *notmuch,
> +				const char **uuid)
> +{
> +    if (uuid)
> +	*uuid = notmuch->uuid;
> +    return notmuch->revision;
> +}
> +
>  /* We allow the user to use arbitrarily long paths for directories. But
>   * we have a term-length limit. So if we exceed that, we'll use the
>   * SHA-1 of the path for the database term.
> diff --git a/lib/notmuch.h b/lib/notmuch.h
> index 20c4e01..b6be727 100644
> --- a/lib/notmuch.h
> +++ b/lib/notmuch.h
> @@ -461,6 +461,24 @@ notmuch_status_t
>  notmuch_database_end_atomic (notmuch_database_t *notmuch);
>  
>  /**
> + * Return the committed database revision and UUID.
> + *
> + * The database revision number increases monotonically with each
> + * commit to the database.  Hence, all messages and message changes
> + * committed to the database (that is, visible to readers) have a last
> + * modification revision <= the committed database revision.  Any
> + * messages committed in the future will be assigned a modification
> + * revision > the committed database revision.
> + *
> + * The UUID is a NUL-terminated opaque string that uniquely identifies
> + * this database.  Two revision numbers are only comparable if they
> + * have the same database UUID.
> + */
> +unsigned long
> +notmuch_database_get_revision (notmuch_database_t *notmuch,
> +				const char **uuid);
> +
> +/**
>   * Retrieve a directory object from the database for 'path'.
>   *
>   * Here, 'path' should be a path relative to the path of 'database'
> diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh
> new file mode 100755
> index 0000000..74a7c49
> --- /dev/null
> +++ b/test/T570-revision-tracking.sh
> @@ -0,0 +1,37 @@
> +#!/usr/bin/env bash
> +test_description="database revision tracking"
> +
> +. ./test-lib.sh
> +
> +add_email_corpus
> +
> +test_begin_subtest "notmuch_database_get_revision"
> +test_C ${MAIL_DIR} <<'EOF'
> +#include <stdio.h>
> +#include <string.h>
> +#include <notmuch.h>
> +int main (int argc, char** argv)
> +{
> +   notmuch_database_t *db;
> +   notmuch_status_t stat;
> +   unsigned long revision;
> +   const char *uuid;
> +
> +   unsigned long rev;
> +
> +   stat = notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_ONLY, &db);
> +   if (stat)
> +       fputs ("open failed\n", stderr);
> +   revision = notmuch_database_get_revision (db, &uuid);
> +   printf("%s\t%lu\n", uuid, revision);
> +}
> +EOF
> +notmuch_uuid_sanitize < OUTPUT > CLEAN
> +cat <<'EOF' >EXPECTED
> +== stdout ==
> +UUID	53
> +== stderr ==
> +EOF
> +test_expect_equal_file EXPECTED CLEAN
> +
> +test_done
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 23085e7..1ec6c5a 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -719,6 +719,11 @@ notmuch_date_sanitize ()
>      sed \
>  	-e 's/^Date: Fri, 05 Jan 2001 .*0000/Date: GENERATED_DATE/'
>  }
> +
> +notmuch_uuid_sanitize ()
> +{
> +    sed  's/^[a-f0-9][a-f0-9-]*/UUID/'
> +}

Pretty much good so far. Here the output in "%s\t%lu\n" is good but
I am not sure whether in --count output... more of that later.

notmuch_uuid_sanitize () could be more generic:

notmuch_uuid_sanitize ()
{
    sed 's/[0-9a-f]\{8\}-[0-9a-f]\{4\}-[0-9a-f]\{4\}-[0-9a-f]\{4\}-[0-9a-f]\{12\}/UUID/g'
}

Oh, and `. ./test-lib.sh || exit 1` :D


Tomi

>  # End of notmuch helper functions
>  
>  # Use test_set_prereq to tell that a particular prerequisite is available.
> -- 
> 2.1.4