--- /dev/null
+Return-Path: <dkg@fifthhorseman.net>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 817326DE01D0\r
+ for <notmuch@notmuchmail.org>; Thu, 2 Jun 2016 10:34:08 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.02\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.02 tagged_above=-999 required=5 tests=[AWL=-0.020]\r
+ autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id 8i_EcJL7dRda for <notmuch@notmuchmail.org>;\r
+ Thu, 2 Jun 2016 10:34:00 -0700 (PDT)\r
+Received: from che.mayfirst.org (che.mayfirst.org [162.247.75.118])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 6CEF96DE00DB\r
+ for <notmuch@notmuchmail.org>; Thu, 2 Jun 2016 10:34:00 -0700 (PDT)\r
+Received: from fifthhorseman.net (unknown [38.109.115.130])\r
+ by che.mayfirst.org (Postfix) with ESMTPSA id 5AEBDF98B;\r
+ Thu, 2 Jun 2016 13:33:58 -0400 (EDT)\r
+Received: by fifthhorseman.net (Postfix, from userid 1000)\r
+ id 3AE6020245; Thu, 2 Jun 2016 13:33:58 -0400 (EDT)\r
+From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>\r
+To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org\r
+Subject: Re: [RFC2 Patch 5/5] lib: iterator API for message properties\r
+In-Reply-To: <87lh2ofpxk.fsf@zancas.localnet>\r
+References: <1463927339-5441-1-git-send-email-david@tethera.net>\r
+ <1464608999-14774-1-git-send-email-david@tethera.net>\r
+ <1464608999-14774-6-git-send-email-david@tethera.net>\r
+ <8760tthfuy.fsf@zancas.localnet> <87pos1u14p.fsf@alice.fifthhorseman.net>\r
+ <87eg8ht2sb.fsf@alice.fifthhorseman.net> <87lh2ofpxk.fsf@zancas.localnet>\r
+User-Agent: Notmuch/0.22+16~g87b7bd4 (http://notmuchmail.org) Emacs/24.5.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Thu, 02 Jun 2016 13:33:54 -0400\r
+Message-ID: <87inxrqyv1.fsf@alice.fifthhorseman.net>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/signed; boundary="=-=-=";\r
+ micalg=pgp-sha512; protocol="application/pgp-signature"\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 02 Jun 2016 17:34:08 -0000\r
+\r
+--=-=-=\r
+Content-Type: text/plain\r
+Content-Transfer-Encoding: quoted-printable\r
+\r
+Hi Bremner--\r
+\r
+thanks for the response! I didn't mean my post to be a wet-blanket,\r
+just wanted to think through the tradeoffs...\r
+\r
+On Wed 2016-06-01 19:29:59 -0400, David Bremner wrote:\r
+> I guess if you don't care about the possibility of iterating all pairs\r
+> with given key prefix (which I admit makes more sense for the config\r
+> API), then the code could be simplified to look more like the tag list\r
+> handling code. C is pretty crap at generics, but I guess looking at\r
+> tags.c, it's really about iterators for notmuch_string_list_t. So it\r
+> could probably be generalized to serve here.\r
+>\r
+> For each such prefix, one would need to roughly duplicate patches 1/5\r
+> and 3/5. It took me a little while to figure 1/5 out, but now that I\r
+> know, it would be less trouble. I guess my thinking here was that I\r
+> would provide a low level interface that people using the C API or\r
+> bindings could use without hacking xapian.\r
+ [...]\r
+> XPROPERTY is an internal prefix, which means it isn't added to the query\r
+> parser. As it happens, I didn't plan on CLI access to these terms\r
+> either. Both of those choices are tradeoffs to say that these are\r
+> internal metadata, suitable for manipulation by programs. Such programs\r
+> could be scripts using python or ruby.\r
+\r
+I think this makes sense, and makes me more comfortable with the overall\r
+idea of this patch series. maybe it'd be useful to clearly document the\r
+intended scope?\r
+\r
+>> If we add new specific features, we could potentially augment the dump\r
+>> format explicitly for them, without having the property abstraction.\r
+>\r
+> We could, but I think should change the dump format quite rarely, since\r
+> we risk breaking people's scripts. So if we did it for one prefix, I'd\r
+> like to do in an extensible way so that adding new prefixes is somewhat\r
+> transparent. It also means some duplication of effort/code in notmuch\r
+> dump/restore to dump/restore each new prefix.\r
+>\r
+> It's probably true that per-prefix dump format would be more compact,\r
+> since the keys would be implicit, rather than repeated for every pair.\r
+\r
+true, though i'm not sure how much compactness is necessary. presumably\r
+people are compressing their dumpfiles, and regularly repeated strings\r
+are the easiest thing to compress.\r
+\r
+>> We already have some explicit features for each message (subject,\r
+>> from, to, attachment, mimetype, thread id, etc), and most of them are\r
+>> derived from the message itself, with the hope that it could be\r
+>> re-derived given just the message body. Is there a distinction\r
+>> between properties that can be derived from the message body and\r
+>> properties that need to be additionally derived from some other data?\r
+>\r
+> As Tomi always says, naming is the hardest thing; properties is a bit\r
+> generic. I'm not sure the distinction you make between the "message" and\r
+> the "message body" here. I think most of our derived terms are from the\r
+> message header. My intent here is that "properties" are used for things\r
+> that cannot be derived from the message (header or body).\r
+\r
+To be clear, i didn't mean to distinguish betweeen "message" and\r
+"message body" -- i don't think of the headers as being significantly\r
+different from the body (and indeed, if we can get memoryhole working,\r
+then some headers might be derived from or influenced by the body).\r
+\r
+maybe it's worth thinking through each of these per-message features,\r
+and where they come from -- are they from the message itself (header,\r
+body, etc), from the message's position(s) in the filesystem, or\r
+somewhere else entirely?\r
+\r
+From=20the message:\r
+\r
+ * message-id\r
+ * subject\r
+ * mimetype\r
+ * attachment\r
+ * references\r
+ * from\r
+ * to\r
+ * replyto\r
+\r
+From=20the filesystem itself:\r
+\r
+ * filenames\r
+ * folder\r
+\r
+From=20elsewhere:\r
+\r
+ * for messages which have multiple files, which file is actually indexed\r
+ * thread-id\r
+ * tag\r
+\r
+we're now talking about adding properties, which are in the "elsewhere"\r
+category, right?\r
+\r
+It's worth noticing that the stuff in "elsewhere" is the stuff that\r
+won't propagate across a dump/restore unless it's explicitly in the dump\r
+somehow. We currently fail to restore thread-id and which file is\r
+actually indexed across a dump/restore :/\r
+\r
+> - per prefix requires new code in the library and dump/restore\r
+> for every prefix\r
+> + the dump format might be more compact if done in a per prefix way.\r
+> + this code would be simpler than the generic properties code,\r
+> mainly because it would not need key value pairs,\r
+> - the library and dump/restore are parts of notmuch that have the\r
+> potential to "break the world". Not too many people are\r
+> comfortable hacking on them.\r
+> - changing the dump format is something like an ABI change for\r
+> people whose scripts rely on dump / restore.\r
+\r
+I think you've convinced me that it's good to go ahead with the\r
+properties, assuming it's scoped as defined above. I still think that\r
+we need a better story for upgrades to the dump format in general, but\r
+maybe this isn't the place to make that particular case.\r
+\r
+ --dkg\r
+\r
+--=-=-=\r
+Content-Type: application/pgp-signature; name="signature.asc"\r
+\r
+-----BEGIN PGP SIGNATURE-----\r
+Version: GnuPG v2\r
+\r
+iQJ8BAEBCgBmBQJXUG4CXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w\r
+ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFREIyRTc0RjU2RkNGMkI2NzI5N0I3MzUy\r
+NEVDRkY1QUZGNjgzNzBBAAoJECTs/1r/aDcKAYIP/0gctfr4FyAaVpazBvRPjyC0\r
+BgAhDO7Wv3V1G88m4spGUcH5yVuWFPhZ+bQedcPD0pExloo3ax21dxlaNiS1/qVE\r
+FtTMkxQbUXHVcDqoYeu4XNBKMng1KSbNJuQ2LHq4g/88ytEKXvcCz7qTbNd6tTQ+\r
+LGlb101PdRJXtbU3MjLn86/Ehomt+AqqxYYDFMaRkDUEgaQPSzWe+H+V5nSWKZJG\r
+xthvfzoElvAXM1sAKNPosBQI2s5k87vn43mXSrKfzNZ0OCTn+Wf4os17ARCEVKA5\r
+PuecT1eXsfwF/R0rj7LPuPLlU3HvSKkPaL32SQsDTRwUgOqF6Cu1FMnzsL7aPXwf\r
+I3wAzfsn4/x7XEfzD3Mot0LhFiS6Iahu7djEshuoxyfUtfHrOLpZy6qDWs5bLyp2\r
+WFJx0zWP2hHoA0HqoabU+38riCiyii6Dq8Fo6qps1UGtX+2IsLVzFQq9599B8845\r
+8J5EU6Upf1zDPcsRCrLoEP4ePtb2eAZzeCmR5ENSFlme+Z9xK66dwEAzxeW3j+3L\r
+4eES+Ft6FXlh7ZMhFv0sssYcXcCdAn9Umvtvt0Q98d7J1/KLEX1HIl4WQJTkTEy8\r
+Ni5x7Zf9hS/M/f4jAJH0M6i366RgI8hQUN9FL0/9Bfq4+pqV6ZjLTAwEiT1GQiUb\r
+y3aH14NDyvKyhlF8Z4Ty\r
+=uF1c\r
+-----END PGP SIGNATURE-----\r
+--=-=-=--\r
projects / notmuch-archives.git / commitdiff