Re: [notmuch] [PATCH] Reindex larger files that duplicate ids we have

diff --git a/54/066adb2dc20ea059dee61bc56171a1de785599 b/54/066adb2dc20ea059dee61bc56171a1de785599
new file mode 100644 (file)
index 0000000..70eab6f
--- /dev/null
+++ b/54/066adb2dc20ea059dee61bc56171a1de785599
@@ -0,0 +1,90 @@
+Return-Path: <cworth@cworth.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 62518431FBD;\r
+       Mon, 21 Dec 2009 10:33:27 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id l7qr1eBlIeIY; Mon, 21 Dec 2009 10:33:26 -0800 (PST)\r
+Received: from yoom.home.cworth.org (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id AB3FA431FAE;\r
+       Mon, 21 Dec 2009 10:33:26 -0800 (PST)\r
+Received: by yoom.home.cworth.org (Postfix, from userid 1000)\r
+       id 53070254306; Mon, 21 Dec 2009 10:33:26 -0800 (PST)\r
+From: Carl Worth <cworth@cworth.org>\r
+To: James Westby <jw+debian@jameswestby.net>, notmuch@notmuchmail.org\r
+In-Reply-To: <1261186149-24078-1-git-send-email-jw+debian@jameswestby.net>\r
+References: <871virzzjy.fsf@yoom.home.cworth.org>\r
+       <1261186149-24078-1-git-send-email-jw+debian@jameswestby.net>\r
+Date: Mon, 21 Dec 2009 10:33:21 -0800\r
+Message-ID: <87vdg0npn2.fsf@yoom.home.cworth.org>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/signed; boundary="=-=-=";\r
+       micalg=pgp-sha1; protocol="application/pgp-signature"\r
+Subject: Re: [notmuch] [PATCH] Reindex larger files that duplicate ids we\r
+       have\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.12\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Mon, 21 Dec 2009 18:33:27 -0000\r
+\r
+--=-=-=\r
+\r
+Hi James,\r
+\r
+I just got to a point in my outstanding rework where I thought it would\r
+make sense to pull this patch series in, (I'm adding support for storing\r
+multiple filenames in a single mail document).\r
+\r
+I took a closer look at this series, and I think it's still independent,\r
+so I'll finish up what I'm doing and then add this series on top later.\r
+\r
+But I can at least answer some of the questions you asked for now:\r
+\r
+>   Does the re-indexing replace the old terms?\r
+\r
+Before this patch, there's there's not yet any "re-indexing" in\r
+notmuch. So we'll basically need to think about what we want to do here.\r
+\r
+As this patch is written, (just calling into the existing _index_file\r
+function), the re-indexing only adds new terms, (and doesn't delete\r
+any). That's probably correct. We're using file size as an heuristic\r
+that the larger file is a superset of the smaller file, but it doesn't\r
+guarantee that the smaller file doesn't contain any unique terms. So I'd\r
+be extremely hesitant to drop any terms here.\r
+\r
+>                                               In the case\r
+>   where you had a collision with different text this could\r
+>   make a search return mails that don't contain that text.\r
+>   I don't think it's a big issue though, even if that is the\r
+>   case.\r
+\r
+That's correct. As mentioned in a previous thread, this is likely only a\r
+big issue in the face of deliberate message-ID spoofing or so. In that\r
+thread we talked about some ideas for mitigating that. But I don't think\r
+we need to solve that problem before applying this patch series.\r
+\r
+-Carl\r
+\r
+--=-=-=\r
+Content-Type: application/pgp-signature\r
+\r
+-----BEGIN PGP SIGNATURE-----\r
+Version: GnuPG v1.4.10 (GNU/Linux)\r
+\r
+iD8DBQFLL79x6JDdNq8qSWgRAs/9AKCZ+RWr9C2bkxVCrLeqau6L+2psigCfYcO9\r
+7WG+Tis8W9qhm0g7oESvOT8=\r
+=wtON\r
+-----END PGP SIGNATURE-----\r
+--=-=-=--\r