From: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org
Subject: Re: [RFC2 Patch 5/5] lib: iterator API for message properties
In-Reply-To: <87mvn1euiz.fsf@zancas.localnet>
References: <1463927339-5441-1-git-send-email-david@tethera.net>
 <1464608999-14774-1-git-send-email-david@tethera.net>
 <1464608999-14774-6-git-send-email-david@tethera.net>
 <8760tthfuy.fsf@zancas.localnet> <87pos1u14p.fsf@alice.fifthhorseman.net>
 <87eg8ht2sb.fsf@alice.fifthhorseman.net> <87lh2ofpxk.fsf@zancas.localnet>
 <87inxrqyv1.fsf@alice.fifthhorseman.net> <8737oufn6f.fsf@zancas.localnet>
 <87y46mpcbf.fsf@alice.fifthhorseman.net> <87mvn1euiz.fsf@zancas.localnet>
User-Agent: Notmuch/0.22+47~g4441900 (http://notmuchmail.org) Emacs/24.5.1
 (x86_64-pc-linux-gnu)
Date: Sat, 04 Jun 2016 12:23:23 -0400
Message-ID: <874m98gbyc.fsf@alice.fifthhorseman.net>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
Precedence: list

--=-=-=
Content-Type: text/plain

On Fri 2016-06-03 19:12:52 -0400, David Bremner wrote:
>>> [ dkg wrote: ]
>>>>  * for messages which have multiple files, which file is actually indexed
>>>
>>> yes. Although rather than storing that, I think the right answer is more
>>> like "all of them".
>>
>> I don't think we do this, do we?  Is this a bug?  is it tracked somewhere?
>
> IMHO it is a bug. It's implicit in
>
>    id:87k42vrqve.fsf@pip.fifthhorseman.net
>
> and the various requests for List-Id indexing, but it's probably worth
> starting a seperate thread to track it.  Especially since there are some
> unresolved design issues. Like what to return for searches.

I'm happy to use that original thread from 2012 as the tracking thread
if you think that's a reasonable starting point.  Peter Wang's "Malice
has nothing on incompetence" message in that thread is a good reminder
about other issues there.

>> the thread-id is *not* reproducible from the
>> message sets.  This is not only because of the ghost message leakage bug
>> documented in T590-thread-breakage.sh, but also because threads can be
>> joined by a message that is later removed (e.g., the "notmuch-join"
>> script in id:87egabu5ta.fsf@alice.fifthhorseman.net ).
>
> I see, I guess that's the intended behaviour given 604d1e0977c.
>
> I haven't thought about the pros and cons of dumping/restoring
> thread-ids. At least my database has about half as many threads as
> messages, so it's a bit of data, but perhaps that's not a bit problem.
> It's somewhat orthogonal to this series since those terms are already
> attached to messages.

i agree that thread restoration is orthogonal to the per-message
properties.  I should also be clear that i don't mean to suggest that we
should dump the literal thread-id.  That'd be terrible, because you
wouldn't be able to merge two databases.

I'm happy to move the thread-id discussion to a separate topic, i just
want to make sure that people are aware that our current documentation
for notmuch-dump (below) kind of overstates the case:

------
       These tags are the only data in the  notmuch  database  that  can't  be
       recreated  from  the messages themselves. The output of notmuch dump is
       therefore the only critical thing to backup (and much more friendly  to
       incremental backup than the native database files.)
------

OK, back to the discussion of per-message properties: i think we should
go ahead with this work, now that i understand the scoping/framing of it
better!

      --dkg

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQJ8BAEBCgBmBQJXUwB7XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFREIyRTc0RjU2RkNGMkI2NzI5N0I3MzUy
NEVDRkY1QUZGNjgzNzBBAAoJECTs/1r/aDcKx+EQAKfJcsVNv30mIslw5d3GFJkF
66yZwn3QsEfkyR7FRAbAwI+N5Hn1oJl90HXrVML83CHXHB4OrgbgYrFOvWwJRPJr
ZnCYpTxn+yiDzN6Nbz3psTKIs/KNxAK3P+khMAFYCEvlgLpkk1hor+g7AkPryQBv
p4QDZt97eJkqCncOE/Tz/LZGFUU2QOaBDXLktRcwpv4VD6mcpW04t76ISF7uneKw
4VoJrmWumFRWbge5V1Wl1EPiwg376fH3c3K26cg9nRT2BfAmPr/8eY69z+JWyCSz
qv2AdOSZxtbaJ9uUT1VLsKRWZbfaO/LZekZ+UKEIjQ6YXOt8MTSuFRYn3Vnt6cqO
M+rJ8IVkbFxXq8Tx5PSHzuUQlXzRYBcVpGP3wTkn0t6eji73fGocPsJrWL1dcHr0
6C1Y5ml7LtKgPhGAeZ1JFCVbKnZo9t3ZFxOG0xlWjJ9JNBVRE2a++cUDWbZQYqhv
uUqrHhyNSF/qaj5uQLPTMkVqKTh1oNkBC4QqJWcB8P5Xig/iRCrJfHOynilvH9qW
CCK5m1wROMB6e+HAqkoqYErybkOGLMBY96/fq3DFM6d7kltpbiRCM26kiXb9+pB5
Sk2/c65uIrCpI6aEAZdDxbHZg1oLEow1c1HaEXyhzULurkrEQuzwC3pjE8vBxS/d
dumg/V0aZEkwxm7ZUWUL
=bhXV
-----END PGP SIGNATURE-----
--=-=-=--