--- /dev/null
+Return-Path: <david@tethera.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 696116DE0243\r
+ for <notmuch@notmuchmail.org>; Wed, 1 Jun 2016 16:30:12 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.012\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.012 tagged_above=-999 required=5\r
+ tests=[AWL=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01]\r
+ autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 8WuOJfqdlZiw for <notmuch@notmuchmail.org>;\r
+ Wed, 1 Jun 2016 16:30:04 -0700 (PDT)\r
+Received: from fethera.tethera.net (fethera.tethera.net [198.245.60.197])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 5909E6DE01D0\r
+ for <notmuch@notmuchmail.org>; Wed, 1 Jun 2016 16:30:04 -0700 (PDT)\r
+Received: from remotemail by fethera.tethera.net with local (Exim 4.84)\r
+ (envelope-from <david@tethera.net>)\r
+ id 1b8FaA-00023w-9i; Wed, 01 Jun 2016 19:29:50 -0400\r
+Received: (nullmailer pid 9631 invoked by uid 1000);\r
+ Wed, 01 Jun 2016 23:29:59 -0000\r
+From: David Bremner <david@tethera.net>\r
+To: Daniel Kahn Gillmor <dkg@fifthhorseman.net>, notmuch@notmuchmail.org\r
+Subject: Re: [RFC2 Patch 5/5] lib: iterator API for message properties\r
+In-Reply-To: <87eg8ht2sb.fsf@alice.fifthhorseman.net>\r
+References: <1463927339-5441-1-git-send-email-david@tethera.net>\r
+ <1464608999-14774-1-git-send-email-david@tethera.net>\r
+ <1464608999-14774-6-git-send-email-david@tethera.net>\r
+ <8760tthfuy.fsf@zancas.localnet> <87pos1u14p.fsf@alice.fifthhorseman.net>\r
+ <87eg8ht2sb.fsf@alice.fifthhorseman.net>\r
+User-Agent: Notmuch/0.22+28~gb9bf3f4 (http://notmuchmail.org) Emacs/24.5.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Wed, 01 Jun 2016 20:29:59 -0300\r
+Message-ID: <87lh2ofpxk.fsf@zancas.localnet>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 01 Jun 2016 23:30:12 -0000\r
+\r
+Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:\r
+\r
+> On Tue 2016-05-31 21:52:06 -0400, Daniel Kahn Gillmor wrote:\r
+>\r
+> do we actually need this abstraction? If we're aiming to build specific\r
+> new features (the two i'm thinking of are cryptographic-session-keys and\r
+> reference-adjustments), couldn't we implement those features explicitly\r
+> in xapian with their own special prefix, rather than treating them as a\r
+> generic "property"?\r
+\r
+Sure, it's certainly possible.\r
+\r
+I guess if you don't care about the possibility of iterating all pairs\r
+with given key prefix (which I admit makes more sense for the config\r
+API), then the code could be simplified to look more like the tag list\r
+handling code. C is pretty crap at generics, but I guess looking at\r
+tags.c, it's really about iterators for notmuch_string_list_t. So it\r
+could probably be generalized to serve here.\r
+\r
+For each such prefix, one would need to roughly duplicate patches 1/5\r
+and 3/5. It took me a little while to figure 1/5 out, but now that I\r
+know, it would be less trouble. I guess my thinking here was that I\r
+would provide a low level interface that people using the C API or\r
+bindings could use without hacking xapian.\r
+\r
+> We already have a bit of an uncomfortable fit with tags and special\r
+> flags (encrypted, signed, attachment, etc), where some are expected to\r
+> be set and cleared automagically and some are expected to be manipulated\r
+> directly by the user. Are we setting ourselves up for more of the same,\r
+> or is there a principled way that a user can know which properties it's\r
+> kosher for them to set and clear, and which ones they should leave\r
+> alone?\r
+\r
+XPROPERTY is an internal prefix, which means it isn't added to the query\r
+parser. As it happens, I didn't plan on CLI access to these terms\r
+either. Both of those choices are tradeoffs to say that these are\r
+internal metadata, suitable for manipulation by programs. Such programs\r
+could be scripts using python or ruby.\r
+\r
+> If we add new specific features, we could potentially augment the dump\r
+> format explicitly for them, without having the property abstraction.\r
+\r
+We could, but I think should change the dump format quite rarely, since\r
+we risk breaking people's scripts. So if we did it for one prefix, I'd\r
+like to do in an extensible way so that adding new prefixes is somewhat\r
+transparent. It also means some duplication of effort/code in notmuch\r
+dump/restore to dump/restore each new prefix.\r
+\r
+It's probably true that per-prefix dump format would be more compact,\r
+since the keys would be implicit, rather than repeated for every pair.\r
+\r
+> We already have some explicit features for each message (subject,\r
+> from, to, attachment, mimetype, thread id, etc), and most of them are\r
+> derived from the message itself, with the hope that it could be\r
+> re-derived given just the message body. Is there a distinction\r
+> between properties that can be derived from the message body and\r
+> properties that need to be additionally derived from some other data?\r
+\r
+As Tomi always says, naming is the hardest thing; properties is a bit\r
+generic. I'm not sure the distinction you make between the "message" and\r
+the "message body" here. I think most of our derived terms are from the\r
+message header. My intent here is that "properties" are used for things\r
+that cannot be derived from the message (header or body).\r
+\r
+TL;DR:\r
+\r
+ - per prefix requires new code in the library and dump/restore\r
+ for every prefix\r
+ + the dump format might be more compact if done in a per prefix way.\r
+ + this code would be simpler than the generic properties code,\r
+ mainly because it would not need key value pairs,\r
+ - the library and dump/restore are parts of notmuch that have the\r
+ potential to "break the world". Not too many people are\r
+ comfortable hacking on them.\r
+ - changing the dump format is something like an ABI change for\r
+ people whose scripts rely on dump / restore.\r
+\r
projects / notmuch-archives.git / commitdiff