From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org
Subject: Re: [RFC2 Patch 5/5] lib: iterator API for message properties
In-Reply-To: <8737oufn6f.fsf@zancas.localnet>
References: <1463927339-5441-1-git-send-email-david@tethera.net>
 <1464608999-14774-1-git-send-email-david@tethera.net>
 <1464608999-14774-6-git-send-email-david@tethera.net>
 <8760tthfuy.fsf@zancas.localnet> <87pos1u14p.fsf@alice.fifthhorseman.net>
 <87eg8ht2sb.fsf@alice.fifthhorseman.net> <87lh2ofpxk.fsf@zancas.localnet>
 <87inxrqyv1.fsf@alice.fifthhorseman.net> <8737oufn6f.fsf@zancas.localnet>
User-Agent: Notmuch/0.22+16~g87b7bd4 (http://notmuchmail.org) Emacs/24.5.1
 (x86_64-pc-linux-gnu)
Date: Fri, 03 Jun 2016 10:38:28 -0400
Message-ID: <87y46mpcbf.fsf@alice.fifthhorseman.net>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
Precedence: list

--=-=-=
Content-Type: text/plain

On Fri 2016-06-03 08:54:00 -0400, David Bremner wrote:
> Sure, where do you think that kind of documentation is appropriate?
> There is the giant comment about the database schema in
> lib/database.cc. Actually I just noticed I already failed to update that
> for libconfig stuff.

That comment seems OK, but it won't be exposed to the people who are in
that middle range (python or ruby programmers but not C programmers).
Do we have a place for this kind of mid-level documenation?

> [ dkg wrote: ]
>>  * for messages which have multiple files, which file is actually indexed
>
> yes. Although rather than storing that, I think the right answer is more
> like "all of them".

I don't think we do this, do we?  Is this a bug?  is it tracked somewhere?

>> It's worth noticing that the stuff in "elsewhere" is the stuff that
>> won't propagate across a dump/restore unless it's explicitly in the dump
>> somehow.   We currently fail to restore thread-id and which file is
>> actually indexed across a dump/restore :/
>
> The thread-id is in some sense derived from the message itself. Not in a
> reproducable way, but still, the dump file is the minimal set of extra
> data needed to reconstruct an equivalent database (pax threading bugs).

This is exactly my point -- i don't care about reproducibility of the
exact numbering, but , the thread-id is *not* reproducible from the
message sets.  This is not only because of the ghost message leakage bug
documented in T590-thread-breakage.sh, but also because threads can be
joined by a message that is later removed (e.g., the "notmuch-join"
script in id:87egabu5ta.fsf@alice.fifthhorseman.net ).

>> I think you've convinced me that it's good to go ahead with the
>> properties, assuming it's scoped as defined above.  I still think that
>> we need a better story for upgrades to the dump format in general, but
>> maybe this isn't the place to make that particular case.
>
> I'm not sure what you have in mind, something more ambitious than the
> header added post 0.22?

Can you point me to the definition for that header?  i still don't
understand what the batch-tag:2 part means.  (sorry i haven't been
keeping up with the master branch lately!)

           --dkg

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQJ8BAEBCgBmBQJXUZZkXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFREIyRTc0RjU2RkNGMkI2NzI5N0I3MzUy
NEVDRkY1QUZGNjgzNzBBAAoJECTs/1r/aDcK9vEP/jDBCHYh5Y5s+3bNE//dkGTU
3/739tfdWDO58ALmW101VZj6WDEph1nxe/4Iw0P13CCmLbUmRyU28yYJF5XxyiQH
eYMk2I3QJxOl1++nhZIoT/ztSCsAJKNK1V4YtJAIiiAbF/ok7qqmwe5BFRiiowEh
xqAb1eVkLhHJce71NpjdDQ0ItLqH6w3rbV+VuxnvZbnla+w6hbX7TOJsPv3tdXA4
lMGuKcHMNUmXeP1FA4MdoqQmluybSL1qxC8W8XraKcduDuA3H22IzPEXdPu6Mr07
bCVS8Jiu3o8ITPtjGDDUfaWZfT/+BV02vChMrrXKHJwOb1NHAnlwRhpo3sFqHcNL
O2WZyBU0Ti8yb3rpgBkT6k1Hl+im9jfV3SN0HrqE6LrDiL2eCl3uJKSYSAxPv+KT
3UnQOLdgACGpfjZC+fqRLtXccYjd8QsLfUN+YJeP5/ZHlfruKYkp4hxeOwsCipPA
cARmXXH0SdFDT+WdYqlIvgM6gNAaZgb9pMyzPnSBTQmf0GT9IK8rnmkCbsYcnMYk
MfpUwq/YjrVpq4L7HOtrYrUXs8AngzVnE1nMkqXunBR1MOIQVkV9Ct+k9RUwwnkl
gkKhRpp0n8OdqqSDn7z3aTCniT0o9/wKLt8B+8UbVdqWag5TuH2su4knXji4B6Dw
cbBqu8U7jUV4RbSRDlqA
=QSUx
-----END PGP SIGNATURE-----
--=-=-=--