[PATCH 0/6] lib: replace the message header parser with gmime

diff --git a/f2/5f987b87f932de8f50196e8da41839ca476d8f b/f2/5f987b87f932de8f50196e8da41839ca476d8f
new file mode 100644 (file)
index 0000000..d93c14d
--- /dev/null
+++ b/f2/5f987b87f932de8f50196e8da41839ca476d8f
@@ -0,0 +1,119 @@
+Return-Path: <jani@nikula.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 4C803431FC4\r
+       for <notmuch@notmuchmail.org>; Wed, 16 Oct 2013 12:00:27 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+       tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id gctA98Lw-GyQ for <notmuch@notmuchmail.org>;\r
+       Wed, 16 Oct 2013 12:00:19 -0700 (PDT)\r
+Received: from mail-ea0-f178.google.com (mail-ea0-f178.google.com\r
+       [209.85.215.178]) (using TLSv1 with cipher RC4-SHA (128/128 bits))\r
+       (No client certificate requested)\r
+       by olra.theworths.org (Postfix) with ESMTPS id F29F7431FAE\r
+       for <notmuch@notmuchmail.org>; Wed, 16 Oct 2013 12:00:18 -0700 (PDT)\r
+Received: by mail-ea0-f178.google.com with SMTP id a15so584795eae.9\r
+       for <notmuch@notmuchmail.org>; Wed, 16 Oct 2013 12:00:17 -0700 (PDT)\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+       d=1e100.net; s=20130820;\r
+       h=x-gm-message-state:from:to:cc:subject:date:message-id;\r
+       bh=G9U5RWyJjmaGecmuA6hU+oc4YCV4x1k5fLR30zG6cmI=;\r
+       b=mvkXKhFUt3exYy8FDX654uqJrzuDNsV57cMDHkfxgFzE7UjLsPKLa0w/M5XNyK+P0I\r
+       d+x4AInFp6iMZNoLnyp3gonr5gChs1qxlQWhKGjkSkwOrEfDncrvn80BR5eUBcgu6OXI\r
+       u8UKJ0dLgf0PJ5udXZ+To5YK+elAAQsytUkjxa+nOncD/KIM1skESXW/SOGujYSRK5mF\r
+       waj+ki2Yw3T0+bCYsxb3h9LyYdFefodzKd9gfzARzuoynw7yZybzFfKJqvv0nRn60aEX\r
+       66zZPFGrT1OZSab6W1unYHSE1o6W1D5hFjdwQ6LEPmp8x9z77e5FfMWBSig1egBhDyyO\r
+       3dRQ==\r
+X-Gm-Message-State:\r
+ ALoCoQkAtWjBN+yytMiLI7EJm996pBghwwWR24OayEuFj0e12Gvapp/9sLOGpSYea7VcM3G4ZIQi\r
+X-Received: by 10.15.44.8 with SMTP id y8mr6771790eev.38.1381950015296;\r
+       Wed, 16 Oct 2013 12:00:15 -0700 (PDT)\r
+Received: from localhost (dsl-hkibrasgw2-58c36f-91.dhcp.inet.fi.\r
+       [88.195.111.91]) by mx.google.com with ESMTPSA id\r
+       r48sm182994044eev.14.1969.12.31.16.00.00\r
+       (version=TLSv1.2 cipher=RC4-SHA bits=128/128);\r
+       Wed, 16 Oct 2013 12:00:14 -0700 (PDT)\r
+From: Jani Nikula <jani@nikula.org>\r
+To: notmuch@notmuchmail.org\r
+Subject: [PATCH 0/6] lib: replace the message header parser with gmime\r
+Date: Wed, 16 Oct 2013 22:00:07 +0300\r
+Message-Id: <cover.1381948853.git.jani@nikula.org>\r
+X-Mailer: git-send-email 1.8.4.rc3\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 16 Oct 2013 19:00:27 -0000\r
+\r
+Hi all, here's something to debate. ;)\r
+\r
+We have a homebrew message header parser in the lib, and we also parse\r
+messages, including headers, using gime during indexing. This means for\r
+messages that get indexed we parse the headers twice. (Duplicates and\r
+non-emails only get parsed using our own parser.)\r
+\r
+The two parsers handle some things differently, which may cause\r
+confusion (tab handling in header folding for example).\r
+\r
+In the interest of reducing somewhat complicated code to maintain, just\r
+nuke the homebrew parser in favor of gmime. I did not look into the\r
+history of why we have our own parser to begin with; it was more fun to\r
+just do some coding. ;)\r
+\r
+Patches 1-3 do prep work to fix some of the differences in the parsers\r
+in advance. Arguably they are not that bad regardless of the parser\r
+change.\r
+\r
+Patches 4-5 actually make the change. Having two patches is a somewhat\r
+artificial division, but perhaps makes it easier to review.\r
+\r
+Patch 6 is just a hack to make perf tests not ignore so many mails... we\r
+have quite a bit of non-emails in the corpus by gmime parser\r
+standards. And this illlustrates one of the differences in the parsers.\r
+\r
+\r
+BR,\r
+Jani.\r
+\r
+\r
+Austin Clements (1):\r
+  emacs: Sanitize authors and subjects in search and show\r
+\r
+Jani Nikula (5):\r
+  cli: sanitize tabs to spaces in notmuch search\r
+  cli: make the hacky from guessing more liberal\r
+  lib: replace the header parser with gmime\r
+  lib: parse messages only once\r
+  HACK: fix broken messages in the perf test corpus\r
+\r
+ emacs/notmuch-lib.el              |   6 +\r
+ emacs/notmuch-show.el             |   7 +-\r
+ emacs/notmuch.el                  |   6 +-\r
+ lib/database.cc                   |   6 +-\r
+ lib/index.cc                      |  70 +-------\r
+ lib/message-file.c                | 351 +++++++++++++-------------------------\r
+ lib/message.cc                    |   6 +\r
+ lib/notmuch-private.h             |  19 ++-\r
+ notmuch-reply.c                   |   4 +-\r
+ notmuch-search.c                  |   4 +-\r
+ performance-test/perf-test-lib.sh |   4 +\r
+ 11 files changed, 172 insertions(+), 311 deletions(-)\r
+\r
+-- \r
+1.8.4.rc3\r
+\r