Proposal: Features instead of versions

diff --git a/9c/7a2f6c70f63c4ba1d71db3ab2e3a7d39bada9c b/9c/7a2f6c70f63c4ba1d71db3ab2e3a7d39bada9c
new file mode 100644 (file)
index 0000000..8784ec4
--- /dev/null
+++ b/9c/7a2f6c70f63c4ba1d71db3ab2e3a7d39bada9c
@@ -0,0 +1,115 @@
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id AE1DE431FAF\r
+       for <notmuch@notmuchmail.org>; Thu, 10 Jul 2014 19:58:10 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id kObUhyMx1zbp for <notmuch@notmuchmail.org>;\r
+       Thu, 10 Jul 2014 19:58:04 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-2.mit.edu (dmz-mailsec-scanner-2.mit.edu\r
+       [18.9.25.13])\r
+       (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id 60D67431FAE\r
+       for <notmuch@notmuchmail.org>; Thu, 10 Jul 2014 19:58:04 -0700 (PDT)\r
+X-AuditID: 1209190d-f79c06d000002f07-c7-53bf52bb490b\r
+Received: from mailhub-auth-3.mit.edu ( [18.9.21.43])\r
+       (using TLS with cipher AES256-SHA (256/256 bits))\r
+       (Client did not present a certificate)\r
+       by dmz-mailsec-scanner-2.mit.edu (Symantec Messaging Gateway) with SMTP\r
+       id B0.50.12039.BB25FB35; Thu, 10 Jul 2014 22:58:03 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+       by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id s6B2w2eR030201; \r
+       Thu, 10 Jul 2014 22:58:02 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+       (authenticated bits=0)\r
+       (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+       by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s6B2w0HB014386\r
+       (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+       Thu, 10 Jul 2014 22:58:01 -0400\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+       (envelope-from <amdragon@mit.edu>)\r
+       id 1X5R28-00061m-0b; Thu, 10 Jul 2014 22:58:00 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: notmuch@notmuchmail.org\r
+Subject: Proposal: Features instead of versions\r
+User-Agent: Notmuch/0.18+42~gfea7a41 (http://notmuchmail.org) Emacs/23.4.1\r
+       (i486-pc-linux-gnu)\r
+Date: Thu, 10 Jul 2014 22:57:59 -0400\r
+Message-ID: <87sim8ob48.fsf@awakening.csail.mit.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFrrEIsWRmVeSWpSXmKPExsUixCmqrbs7aH+wwafbOhZN050trt+cyezA\r
+       5HHr/mt2j2erbjEHMEVx2aSk5mSWpRbp2yVwZcxa95SlYC9/xZoDGg2Ml3i6GDk5JARMJCa+\r
+       6WCGsMUkLtxbz9bFyMUhJDCbSWLtw13MEM5GRok5S18yQTinmSTm7DwEVbaEUWLN1Ltg/WwC\r
+       GhLb9i9nBLFFBKQldt6dzQpiMwsoS7xYM4Wli5GDQ1hAX2L3rQwQU1QgTuLtbRuQChYBVYk3\r
+       s/6BdfICXfT65g4oW1Di5MwnLBBTtCRu/HvJNIGRfxaS1CwkqQWMTKsYZVNyq3RzEzNzilOT\r
+       dYuTE/PyUot0jfRyM0v0UlNKNzGCg06Sdwfju4NKhxgFOBiVeHhPrNkXLMSaWFZcmXuIUZKD\r
+       SUmU19Zif7AQX1J+SmVGYnFGfFFpTmrxIUYJDmYlEd4Z3EA53pTEyqrUonyYlDQHi5I471tr\r
+       q2AhgfTEktTs1NSC1CKYrAwHh5IE75ZAoEbBotT01Iq0zJwShDQTByfIcB6g4ZkgNbzFBYm5\r
+       xZnpEPlTjIpS4rxhAUAJAZBERmkeXC8sKbxiFAd6RZh3Jkg7DzChwHW/AhrMBDTY2mIPyOCS\r
+       RISUVANjsdYTkTmM50W/rNrB1Vrdaxp3hmV5Y/DM0im9Rsna15fGqMep3uhIl39srcN7Y+ou\r
+       6873ZruTz6sv+PL6vO4H0/5JZ1IfPmu8cGeucP0B7k+i3q7R233/zPJ/cWOx6ZcpNxPDq92q\r
+       934s6V4WbXo5xURk66YHay2+xtcauuzgstn3892ioLBDSizFGYmGWsxFxYkA6mPP2+UCAAA=\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 11 Jul 2014 02:58:11 -0000\r
+\r
+I believe our current approach to database schema changes is\r
+inhibiting the evolution of Notmuch's features and would like to\r
+propose what I think is lighter-weight solution.\r
+\r
+Currently, our database schema is versioned and each database schema\r
+change must occur "atomically" in Notmuch's development history:\r
+before some commit, Notmuch uses version N, after that commit, it uses\r
+version N+1.  Hence, each new schema version can introduce only one\r
+change, the task of developing a schema change falls on a single\r
+person, and it must all happen and be perfect in a single commit\r
+series.  This makes introducing a new schema version hard.  We've seen\r
+only two schema changes in the history of Notmuch.\r
+\r
+I'd like to propose that we switch to a "feature set", recorded in a\r
+database field.  The recent introduction of boolean folder terms would\r
+be a "feature" (and the lack of that feature would imply probabilistic\r
+folder terms).  Likewise, the file terms added in version 1 would be a\r
+"feature".\r
+\r
+The upgrade process would be structured around the delta between the\r
+database's feature set and the desired feature set.  For many things,\r
+it would be easy to support databases both with and without a feature,\r
+which would enable "unstable" features that can be developed and\r
+tested over time, and different features could be under development in\r
+parallel.  We can also mark features as required or optional for\r
+opening the database in read mode and replace our current unknown\r
+version warning (which the user can't act on in any useful way) with\r
+either no warning or a straight failure.\r
+\r
+I've done some of this for my ghost messages support (restructuring\r
+upgrade, unstable features, and supporting both the current schema and\r
+the new schema), and it was simple and worked nicely.  This is also very\r
+similar to how ext4 works [1].\r
+\r
+Thoughts?\r
+\r
+\r
+[1] https://ext4.wiki.kernel.org/index.php/Ext4_Disk_Layout#The_Super_Block\r
+fields 0x5C through 0x64.\r