From: Michal Sojka <sojkam1@fel.cvut.cz>
To: martin f krafft <madduck@madduck.net>
Subject: Re: [PATCH] Mailstore abstraction v4 - part 2 (maildir
	synchronization)
In-Reply-To: <20100412123853.GA26402@piper.oerlikon.madduck.net>
References: <1270739592-30280-1-git-send-email-sojkam1@fel.cvut.cz>
	<20100412081805.GA25971@lapse.rw.madduck.net>
	<87hbngc2og.fsf@steelpick.2x.cz>
	<20100412123853.GA26402@piper.oerlikon.madduck.net>
Date: Mon, 12 Apr 2010 15:23:43 +0200
Message-ID: <87aat8by8w.fsf@steelpick.2x.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: notmuch@notmuchmail.org
Precedence: list

On Mon, 12 Apr 2010, martin f krafft wrote:
> also sprach Michal Sojka <sojkam1@fel.cvut.cz> [2010.04.12.1347 +0200]:
> > > Wouldn't it be better to postpone synchronisation of the tags
> > > until after emacs is done with the message?
> > 
> > Theoretically, it would be possible, but if, for some reason, the
> > synchronization step would not happen, then the tags in the
> > database and in the mailstore will be inconsistent and next run of
> > notmuch new would reset the tags according to the state in
> > mailstore.
> 
> Well, sure. But then again, notmuch (nor IMAP or Maildir) isn't
> transactional anyway. There are many other ways in which the
> database and store can get out of sync. And you are about to add
> another redundancy.
> 
> > The current implementation takes tags in mailstore as
> > authoritative and ensures that tags in mailstore are always
> > updated before tags in the database.
> 
> So why store them in the database at all?

Because we want to use them in searches.

> > > I understand this might be hard to make work with mailstore
> > > abstraction. Wouldn't it make more sense to have emacs call
> > > 'notmuch cat', which returns the entire message, removes the
> > > unread tag, changes the filename, and updates the database?
> > 
> > I do not like the fact that cat would do two things - cat and tag.
> > And then, 'unread' tag is not the only one which can be changed.
> 
> I wouldn't want an unread tag in the first place, especially not
> with Maildir semantics. In this case, what should really happen is:

The bellow sounds reasonable. I only have a few notes.
> 
>   1. cat feeds a message to client

I'd not use cat here, I'd stay with show which transfers only text/*
parts. Recently I was surprised how fast notmuch is when client is
running locally and notmuch is invoked remotely over ssh (even over slow
connection).

>   2. client instructs notmuch to update tags
>      - some tags require changes in the database
>      - others require filename changes, which must be completed in
>        unison with a database update so the new filename is stored.
>   3. user asks to see attachment, which the client can fulfill using
>      either a cached copy from (1.) in a tempfile, or by simply
>      asking for the message again, via notmuch search.

The latter option sounds well for me, but my elisp skills are not
sufficient for implementing this. Maybe, dme will do it in his new
client. On the other side, I'm not sure whether it has sense search for
the message again (in step 3) if you are not using maildir mailstore.
One possibility would be, when using maildir mailstore, to cheat the
client and use message-id instead of filename. Then, the client would
use message-id in cat command and implementation for open_file() in
maildir mailstore will search for the real file name.

What do you think?

> > > The message returned by cat would be stored in a temporary file
> > > for use by the client (emacs). And if the message was needed
> > > again, you could just search for it again.
> > > 
> > > I dislike the idea of heuristically probing a Maildir for files.
> > 
> > Well, I do not plan to use wired heuristics. At the end there will
> > be readdir() to traverse the cur/ directory to find the file with
> > the same part before flags. Since the S flag will probably be the
> > most frequent change, I may add one single test for added S flag
> > before trying more expensive readdir().
> 
> What is the point of storing these tags in the Maildir anyway? 

My point is to synchronize the status of reading of my emails with IMAP
server (through offlineimap). I use notmuch as my primary emails client,
but my mobile phone checks for new emails on the IMAP server and from
time to time I need to use other IMAP clients (on other computers).

I have offlineimap running in background so that whenever I read a
message in notmuch, after some short time, this information is
propagated back to IMAP server and when I go home from work, my mobile
phone shows zero unread mails.

> If you want to make this information (e.g. new, seen, unread)
> available to MUAs accessing Maildir directly, keep in mind that the
> database and mailstore will very quickly grow inconsistent until the
> next notmuch-new run, e.g. as messages are moved, or flags ('F') are
> added in a way that the notmuch database is not updated.

It depends on how you define quickly and if you run notmuch new also
very quickly, there is almost no inconsistency :))

-Michal