--- /dev/null
+Return-Path: <aclements@csail.mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 18AF0431FB6\r
+ for <notmuch@notmuchmail.org>; Tue, 21 Oct 2014 18:33:09 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.3\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id N+dcz96voSKY for <notmuch@notmuchmail.org>;\r
+ Tue, 21 Oct 2014 18:33:01 -0700 (PDT)\r
+Received: from outgoing.csail.mit.edu (outgoing.csail.mit.edu [128.30.2.149])\r
+ by olra.theworths.org (Postfix) with ESMTP id 5A730431FAE\r
+ for <notmuch@notmuchmail.org>; Tue, 21 Oct 2014 18:33:01 -0700 (PDT)\r
+Received: from [104.131.20.129] (helo=awakeningjr)\r
+ by outgoing.csail.mit.edu with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16)\r
+ (Exim 4.72) (envelope-from <aclements@csail.mit.edu>)\r
+ id 1XgknM-0004kE-Kp; Tue, 21 Oct 2014 21:33:00 -0400\r
+Received: from amthrax by awakeningjr with local (Exim 4.84)\r
+ (envelope-from <aclements@csail.mit.edu>)\r
+ id 1XgknM-0000Ah-Ay; Tue, 21 Oct 2014 21:33:00 -0400\r
+Date: Tue, 21 Oct 2014 21:33:00 -0400\r
+From: Austin Clements <aclements@csail.mit.edu>\r
+To: Mark Walters <markwalters1009@gmail.com>\r
+Subject: Re: [PATCH v2 07/12] lib: Internal support for querying and creating\r
+ ghost messages\r
+Message-ID: <20141022013300.GD7970@csail.mit.edu>\r
+References: <1412637438-4821-1-git-send-email-aclements@csail.mit.edu>\r
+ <1412637438-4821-8-git-send-email-aclements@csail.mit.edu>\r
+ <8738ahja72.fsf@qmul.ac.uk>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <8738ahja72.fsf@qmul.ac.uk>\r
+User-Agent: Mutt/1.5.23 (2014-03-12)\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 22 Oct 2014 01:33:09 -0000\r
+\r
+Quoth Mark Walters on Oct 22 at 12:05 am:\r
+> \r
+> Hi \r
+> \r
+> I am slowly working my way through this series: only two trivial queries\r
+> so far.\r
+> \r
+> On Tue, 07 Oct 2014, Austin Clements <aclements@csail.mit.edu> wrote:\r
+> > From: Austin Clements <amdragon@mit.edu>\r
+> >\r
+> > This updates the message abstraction to support ghost messages: it\r
+> > adds a message flag that distinguishes regular messages from ghost\r
+> > messages, and an internal function for initializing a newly created\r
+> > (blank) message as a ghost message.\r
+> > ---\r
+> > lib/message.cc | 52 +++++++++++++++++++++++++++++++++++++++++++++++++--\r
+> > lib/notmuch-private.h | 4 ++++\r
+> > lib/notmuch.h | 9 ++++++++-\r
+> > 3 files changed, 62 insertions(+), 3 deletions(-)\r
+> >\r
+> > diff --git a/lib/message.cc b/lib/message.cc\r
+> > index 55d2ff6..a7a13cc 100644\r
+> > --- a/lib/message.cc\r
+> > +++ b/lib/message.cc\r
+> > @@ -39,6 +39,9 @@ struct visible _notmuch_message {\r
+> > notmuch_message_file_t *message_file;\r
+> > notmuch_message_list_t *replies;\r
+> > unsigned long flags;\r
+> > + /* For flags that are initialized on-demand, lazy_flags indicates\r
+> > + * if each flag has been initialized. */\r
+> > + unsigned long lazy_flags;\r
+> \r
+> I wonder if valid_flags might be better here as, as far as I can see,\r
+> the reason for these is so we can invalidate a flag more than an\r
+> optimisation (which is what I thought the lazy implied).\r
+\r
+I do think of this as an optimization. If we were to compute the\r
+value of this flag when a message was created (and keep it\r
+up-to-date), there would be no need for lazy_flags. But, unlike the\r
+other flags, computing this is expensive.\r
+\r
+> > \r
+> > Xapian::Document doc;\r
+> > Xapian::termcount termpos;\r
+> > @@ -99,6 +102,7 @@ _notmuch_message_create_for_document (const void *talloc_owner,\r
+> > \r
+> > message->frozen = 0;\r
+> > message->flags = 0;\r
+> > + message->lazy_flags = 0;\r
+> > \r
+> > /* Each of these will be lazily created as needed. */\r
+> > message->message_id = NULL;\r
+> > @@ -192,7 +196,7 @@ _notmuch_message_create (const void *talloc_owner,\r
+> > *\r
+> > * There is already a document with message ID 'message_id' in the\r
+> > * database. The returned message can be used to query/modify the\r
+> > - * document.\r
+> > + * document. The message may be a ghost message.\r
+> > *\r
+> > * NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND:\r
+> > *\r
+> > @@ -305,6 +309,7 @@ _notmuch_message_ensure_metadata (notmuch_message_t *message)\r
+> > const char *thread_prefix = _find_prefix ("thread"),\r
+> > *tag_prefix = _find_prefix ("tag"),\r
+> > *id_prefix = _find_prefix ("id"),\r
+> > + *type_prefix = _find_prefix ("type"),\r
+> > *filename_prefix = _find_prefix ("file-direntry"),\r
+> > *replyto_prefix = _find_prefix ("replyto");\r
+> > \r
+> > @@ -337,10 +342,25 @@ _notmuch_message_ensure_metadata (notmuch_message_t *message)\r
+> > message->message_id =\r
+> > _notmuch_message_get_term (message, i, end, id_prefix);\r
+> > \r
+> > + /* Get document type */\r
+> > + assert (strcmp (id_prefix, type_prefix) < 0);\r
+> > + if (! NOTMUCH_TEST_BIT (message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST)) {\r
+> > + i.skip_to (type_prefix);\r
+> > + /* "T" is the prefix "type" fields. See\r
+> > + * BOOLEAN_PREFIX_INTERNAL. */\r
+> > + if (*i == "Tmail")\r
+> > + NOTMUCH_CLEAR_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);\r
+> > + else if (*i == "Tghost")\r
+> > + NOTMUCH_SET_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);\r
+> > + else\r
+> > + INTERNAL_ERROR ("Message without type term");\r
+> > + NOTMUCH_SET_BIT (&message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST);\r
+> > + }\r
+> > +\r
+> > /* Get filename list. Here we get only the terms. We lazily\r
+> > * expand them to full file names when needed in\r
+> > * _notmuch_message_ensure_filename_list. */\r
+> > - assert (strcmp (id_prefix, filename_prefix) < 0);\r
+> > + assert (strcmp (type_prefix, filename_prefix) < 0);\r
+> > if (!message->filename_term_list && !message->filename_list)\r
+> > message->filename_term_list =\r
+> > _notmuch_database_get_terms_with_prefix (message, i, end,\r
+> > @@ -371,6 +391,11 @@ _notmuch_message_invalidate_metadata (notmuch_message_t *message,\r
+> > message->tag_list = NULL;\r
+> > }\r
+> > \r
+> > + if (strcmp ("type", prefix_name) == 0) {\r
+> > + NOTMUCH_CLEAR_BIT (&message->flags, NOTMUCH_MESSAGE_FLAG_GHOST);\r
+> > + NOTMUCH_CLEAR_BIT (&message->lazy_flags, NOTMUCH_MESSAGE_FLAG_GHOST);\r
+> > + }\r
+> > +\r
+> > if (strcmp ("file-direntry", prefix_name) == 0) {\r
+> > talloc_free (message->filename_term_list);\r
+> > talloc_free (message->filename_list);\r
+> > @@ -869,6 +894,10 @@ notmuch_bool_t\r
+> > notmuch_message_get_flag (notmuch_message_t *message,\r
+> > notmuch_message_flag_t flag)\r
+> > {\r
+> > + if (flag == NOTMUCH_MESSAGE_FLAG_GHOST &&\r
+> > + ! NOTMUCH_TEST_BIT (message->lazy_flags, flag))\r
+> > + _notmuch_message_ensure_metadata (message);\r
+> > +\r
+> > return NOTMUCH_TEST_BIT (message->flags, flag);\r
+> > }\r
+> > \r
+> > @@ -880,6 +909,7 @@ notmuch_message_set_flag (notmuch_message_t *message,\r
+> > NOTMUCH_SET_BIT (&message->flags, flag);\r
+> > else\r
+> > NOTMUCH_CLEAR_BIT (&message->flags, flag);\r
+> > + NOTMUCH_SET_BIT (&message->lazy_flags, flag);\r
+> > }\r
+> > \r
+> > time_t\r
+> > @@ -989,6 +1019,24 @@ _notmuch_message_delete (notmuch_message_t *message)\r
+> > return NOTMUCH_STATUS_SUCCESS;\r
+> > }\r
+> > \r
+> > +/* Transform a blank message into a ghost message. The caller must\r
+> > + * _notmuch_message_sync the message. */\r
+> > +notmuch_private_status_t\r
+> > +_notmuch_message_initialize_ghost (notmuch_message_t *message,\r
+> > + const char *thread_id)\r
+> > +{\r
+> > + notmuch_private_status_t status;\r
+> > +\r
+> > + status = _notmuch_message_add_term (message, "type", "ghost");\r
+> > + if (status)\r
+> > + return status;\r
+> > + status = _notmuch_message_add_term (message, "thread", thread_id);\r
+> > + if (status)\r
+> > + return status;\r
+> > +\r
+> > + return NOTMUCH_PRIVATE_STATUS_SUCCESS;\r
+> > +}\r
+> > +\r
+> > /* Ensure that 'message' is not holding any file object open. Future\r
+> > * calls to various functions will still automatically open the\r
+> > * message file as needed.\r
+> > diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h\r
+> > index 7250291..2f43c1d 100644\r
+> > --- a/lib/notmuch-private.h\r
+> > +++ b/lib/notmuch-private.h\r
+> > @@ -308,6 +308,10 @@ _notmuch_message_sync (notmuch_message_t *message);\r
+> > notmuch_status_t\r
+> > _notmuch_message_delete (notmuch_message_t *message);\r
+> > \r
+> > +notmuch_private_status_t\r
+> > +_notmuch_message_initialize_ghost (notmuch_message_t *message,\r
+> > + const char *thread_id);\r
+> > +\r
+> > void\r
+> > _notmuch_message_close (notmuch_message_t *message);\r
+> > \r
+> > diff --git a/lib/notmuch.h b/lib/notmuch.h\r
+> > index dae0416..92594b9 100644\r
+> > --- a/lib/notmuch.h\r
+> > +++ b/lib/notmuch.h\r
+> > @@ -1221,7 +1221,14 @@ notmuch_message_get_filenames (notmuch_message_t *message);\r
+> > */\r
+> > typedef enum _notmuch_message_flag {\r
+> > NOTMUCH_MESSAGE_FLAG_MATCH,\r
+> > - NOTMUCH_MESSAGE_FLAG_EXCLUDED\r
+> > + NOTMUCH_MESSAGE_FLAG_EXCLUDED,\r
+> > +\r
+> > + /* This message is a "ghost message", meaning it has no filenames\r
+> > + * or content, but we know it exists because it was referenced by\r
+> > + * some other message. A ghost message has only a message ID and\r
+> > + * thread ID.\r
+> > + */\r
+> \r
+> Can I check here: we are not allowing a ghost message to have any tags?\r
+\r
+Correct, at least for now.\r
+\r
+However, I think it would make *a lot* of sense to be able to pre-seed\r
+ghost messages with tags. nmbug could use this to avoid races with\r
+receiving messages. Distributed tag sync could use it similarly.\r
+Insert could use it to eliminate the nasty races between storing the\r
+message, indexing it, and tagging it. Restore could potentially use\r
+it. When sending messages, we could pre-seed a sent tag for when the\r
+sent message is re-received (though insert may obviate that). I'm\r
+sure there are other uses I haven't thought of.\r
+\r
+This requires some new APIs, since there's currently no way for a\r
+library user to create a ghost message or get at it to tag it. It\r
+also slightly complicates notmuch_database_get_all_tags since that\r
+probably shouldn't return tags that are only on ghost messages (I\r
+think if we just collect all the docids in the Tghost posting list and\r
+use that to filter out tag terms that there should be almost no\r
+performance impact of this). But these are both quite doable things.\r
+\r
+A more complicated question is what we want to do with deleted\r
+messages. Currently we remove them entirely from the database, but we\r
+*could* keep around their tags so if the message reappears (e.g.,\r
+there was a transient problem) we can bring back the tags. After\r
+thinking about this a great deal, I concluded we should just continue\r
+deleting them from the database (or, at most, strip the message back\r
+down to its thread ID). If anyone's curious, I can write up my\r
+thoughts on this, but it boils down to complicated the semantics of\r
+initial tagging and dump/restore.\r
+\r
+> Best wishes\r
+> \r
+> Mark\r
+> \r
+> > + NOTMUCH_MESSAGE_FLAG_GHOST,\r
+> > } notmuch_message_flag_t;\r
+> > \r
+> > /**\r
+> >\r
+> > _______________________________________________\r
+> > notmuch mailing list\r
+> > notmuch@notmuchmail.org\r
+> > http://notmuchmail.org/mailman/listinfo/notmuch\r
projects / notmuch-archives.git / commitdiff