From: David Bremner <david@tethera.net>
To: Igor Almeida <igor.contato@gmail.com>, notmuch@notmuchmail.org
Subject: Re: [PATCH/RFC 0/3] Maildir custom flags and notmuch tags
In-Reply-To: <1448504191-30974-1-git-send-email-igor.contato@gmail.com>
References: <1448504191-30974-1-git-send-email-igor.contato@gmail.com>
User-Agent: Notmuch/0.21+26~g9404723 (http://notmuchmail.org) Emacs/24.5.1
 (x86_64-pc-linux-gnu)
Date: Sun, 10 Jan 2016 09:59:28 -0400
Message-ID: <87mvsd7cxr.fsf@zancas.localnet>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list

Igor Almeida <igor.contato@gmail.com> writes:

> Building on Bremner's patch at
> <1411805835-3563-1-git-send-email-david@tethera.net>, this updates
> 'notmuch new' to tag messages based on lower-case letters in the file
> name.
> I'm testing this with an IMAP server, offlineimap and notmuch,
> and seems to work well enough for my unidirectional use case.
> Eventually I'd like to implement the reverse path, i.e. tagging in
> notmuch and seeing IMAP keywords show up in the server, but that's for
> later.
>
> Comments are very welcome.
>

Sorry about the long silence. I _have_ been thinking about and working
on this. This message summarizes my current thinking/plan.

I'm not very enthusiastic about the about the several places this can
get out of sync

1) config info in .notmuch-config can be out of sync with the "real
config" in the database.  This part is only until the next run of
notmuch new, so it's not so bad.

2) When the interpretation of maildir flags changes, there doesn't seem
to be a sensible way of retagging messages.

Some others have objected to supporting a nonstandard extension used (I
think) only by dovecot. For me this is not so bad, as long as the code
required specifically for this is not too large / intrusive.

So here are the pieces I think we need. 

a) some code in notmuch-new to sync flags a-z to a fixed set of tags
  [1]. This soundspretty much the functionality you have now, although I
  haven't really reviewed that part of the series.

b) some code to update metadata directly from the CLI (notmuch
   config/notmuch tag), probably using something like [2] as a back
   end. This metadata maps the fixed tags like $maildir_flag_k to and
   from some friendly format.

c) The use of xapian field processors to translate e.g. a query
   "tag:my-sensible-thing" into "tag:$maildir_flag_k". Optionally do the
   reverse when returning lists of tags. FieldProcessors are only
   available in Xapian 1.3.x, but apparently 1.3 series is nearing a
   stable release, so it's a sane time to optionally depend on it.
   I guess as a short term hack, this could be done by shell wrappers
   (not something we'd ship, but it might make it usable for you).          

d) code to dump and restore the metadata as part of the normal dump
   files. This is at least in progress [2].

In terms of how these changes relate to other goals

b,d) have strong connects with allowing database level config. Among
     other reasons this will help make bindings users less second class
     citizens
      

  c) has overlap with allowing date fields that are not ranges (among
     other things). At least all the machinery to configure xapian 1.2
     versus 1.3, and the understanding how to use field processors, is
     reusable.

I'd understand perfectly if you prefer to rebase your simpler solution
on top of master, but if you want to keep working on this, then I guess
(a), (b), and (c) are all things you would work on.

Probably (b) needs the library api from [2], which now that I start
typing this epic long message, I could split off from the dump/restore
stuff. In particular managing the config metadata only needs about half
of the LOC; the iterator is only needed for dump, I think.

David

[1]: eventually it might make sense to split the tags that are somehow
automatic from those that represent user choice. We already have this
issue with tag:encrypted and friends.

[2]: id:1452394301-29499-1-git-send-email-david@tethera.net