From: Michal Sojka <sojkam1@fel.cvut.cz>
To: Serge Z <triumhiz@yandex.ru>, notmuch@notmuchmail.org
Subject: Re: [PATCH] test: Add test for searching of uncommonly encoded
	messages
In-Reply-To: <20120224042925.2870.87924@localhost>
References: <877gzd5axk.fsf@steelpick.2x.cz>
	<1330043595-22054-1-git-send-email-sojkam1@fel.cvut.cz>
	<20120224042925.2870.87924@localhost>
User-Agent: Notmuch/0.11.1+210~g5c2fc0a (http://notmuchmail.org) Emacs/23.3.1
	(x86_64-pc-linux-gnu)
Date: Fri, 24 Feb 2012 08:00:02 +0100
Message-ID: <874nug67il.fsf@steelpick.2x.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Fri, 24 Feb 2012, Serge Z wrote:
> 
> Quoting Michal Sojka (2012-02-24 04:33:15)
> >Emails that are encoded differently than as ASCII or UTF-8 are not
> >indexed properly by notmuch. It is not possible to search for non-ASCII
> >words within those messages.
> 
> Ok. But we can preprocess each incoming message right after 'getmail' to
> convert it from html to text and to utf8 encoding. One solution is to create a
> seperate script for this and make gmail pipe all messages to this script, and
> then to notmuch. But It would be better if maildir contains original messages
> only, so the question is: can we make nomuch indexing engine to index
> preprocessed message while maildir will contain original message - as it was
> obtained?

Hi,

I'm not big fan of adding "preprocessor". First, I thing that both
reasons you mention are actually bugs and it would be better to fix them
for everybody than requiring each user to configure some preprocessor.
Second, depending on what and how would your preprocessor do, the
initial mail indexing could be a way slower, which is also nothing that
people want.

Do you have any other use case for the preprocessor besides utf8 and
html->text conversions?

Cheers,
-Michal