Re: [PATCH] test: Add test for searching of uncommonly encoded messages

diff --git a/9c/b4f29f1ff99302d9713e321086a30cde97fba8 b/9c/b4f29f1ff99302d9713e321086a30cde97fba8
new file mode 100644 (file)
index 0000000..4ccadba
--- /dev/null
+++ b/9c/b4f29f1ff99302d9713e321086a30cde97fba8
@@ -0,0 +1,94 @@
+Return-Path: <sojka@os.inf.tu-dresden.de>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id C5AAA431FBC\r
+       for <notmuch@notmuchmail.org>; Thu, 23 Feb 2012 23:00:07 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.3\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.3 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id FySAFePq64fa for <notmuch@notmuchmail.org>;\r
+       Thu, 23 Feb 2012 23:00:06 -0800 (PST)\r
+Received: from max.feld.cvut.cz (max.feld.cvut.cz [147.32.192.36])\r
+       by olra.theworths.org (Postfix) with ESMTP id 487F1431FAE\r
+       for <notmuch@notmuchmail.org>; Thu, 23 Feb 2012 23:00:04 -0800 (PST)\r
+Received: from localhost (unknown [192.168.200.4])\r
+       by max.feld.cvut.cz (Postfix) with ESMTP id 134A73CFEC6;\r
+       Fri, 24 Feb 2012 08:00:04 +0100 (CET)\r
+X-Virus-Scanned: IMAP AMAVIS\r
+Received: from max.feld.cvut.cz ([192.168.200.1])\r
+       by localhost (styx.feld.cvut.cz [192.168.200.4]) (amavisd-new,\r
+       port 10044)\r
+       with ESMTP id td6IedLwPKbr; Fri, 24 Feb 2012 08:00:03 +0100 (CET)\r
+Received: from imap.feld.cvut.cz (imap.feld.cvut.cz [147.32.192.34])\r
+       by max.feld.cvut.cz (Postfix) with ESMTP id 150A83CFEC3;\r
+       Fri, 24 Feb 2012 08:00:03 +0100 (CET)\r
+Received: from steelpick.2x.cz (cable-86-56-3-85.cust.telecolumbus.net\r
+       [86.56.3.85]) (Authenticated sender: sojkam1)\r
+       by imap.feld.cvut.cz (Postfix) with ESMTPSA id F2826660969;\r
+       Fri, 24 Feb 2012 08:00:02 +0100 (CET)\r
+Received: from wsh by steelpick.2x.cz with local (Exim 4.77)\r
+       (envelope-from <sojka@os.inf.tu-dresden.de>)\r
+       id 1S0p8M-0006qe-9A; Fri, 24 Feb 2012 08:00:02 +0100\r
+From: Michal Sojka <sojkam1@fel.cvut.cz>\r
+To: Serge Z <triumhiz@yandex.ru>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] test: Add test for searching of uncommonly encoded\r
+       messages\r
+In-Reply-To: <20120224042925.2870.87924@localhost>\r
+References: <877gzd5axk.fsf@steelpick.2x.cz>\r
+       <1330043595-22054-1-git-send-email-sojkam1@fel.cvut.cz>\r
+       <20120224042925.2870.87924@localhost>\r
+User-Agent: Notmuch/0.11.1+210~g5c2fc0a (http://notmuchmail.org) Emacs/23.3.1\r
+       (x86_64-pc-linux-gnu)\r
+Date: Fri, 24 Feb 2012 08:00:02 +0100\r
+Message-ID: <874nug67il.fsf@steelpick.2x.cz>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 24 Feb 2012 07:00:08 -0000\r
+\r
+On Fri, 24 Feb 2012, Serge Z wrote:\r
+> \r
+> Quoting Michal Sojka (2012-02-24 04:33:15)\r
+> >Emails that are encoded differently than as ASCII or UTF-8 are not\r
+> >indexed properly by notmuch. It is not possible to search for non-ASCII\r
+> >words within those messages.\r
+> \r
+> Ok. But we can preprocess each incoming message right after 'getmail' to\r
+> convert it from html to text and to utf8 encoding. One solution is to create a\r
+> seperate script for this and make gmail pipe all messages to this script, and\r
+> then to notmuch. But It would be better if maildir contains original messages\r
+> only, so the question is: can we make nomuch indexing engine to index\r
+> preprocessed message while maildir will contain original message - as it was\r
+> obtained?\r
+\r
+Hi,\r
+\r
+I'm not big fan of adding "preprocessor". First, I thing that both\r
+reasons you mention are actually bugs and it would be better to fix them\r
+for everybody than requiring each user to configure some preprocessor.\r
+Second, depending on what and how would your preprocessor do, the\r
+initial mail indexing could be a way slower, which is also nothing that\r
+people want.\r
+\r
+Do you have any other use case for the preprocessor besides utf8 and\r
+html->text conversions?\r
+\r
+Cheers,\r
+-Michal\r