Re: Deduplication ?

diff --git a/3c/489e66737ebc35278a96cc39a5c30df03872ec b/3c/489e66737ebc35278a96cc39a5c30df03872ec
new file mode 100644 (file)
index 0000000..80892ca
--- /dev/null
+++ b/3c/489e66737ebc35278a96cc39a5c30df03872ec
@@ -0,0 +1,103 @@
+Return-Path: <tomi.ollila@iki.fi>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+       by olra.theworths.org (Postfix) with ESMTP id 6195C40D1CF\r
+       for <notmuch@notmuchmail.org>; Sat,  7 Jun 2014 06:38:01 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]\r
+       autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+       by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+       with ESMTP id YYNB-c-X9UwR for <notmuch@notmuchmail.org>;\r
+       Sat,  7 Jun 2014 06:37:50 -0700 (PDT)\r
+Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])\r
+       by olra.theworths.org (Postfix) with ESMTP id 9A9D040A924\r
+       for <notmuch@notmuchmail.org>; Sat,  7 Jun 2014 06:37:50 -0700 (PDT)\r
+Received: from guru.guru-group.fi (localhost [IPv6:::1])\r
+       by guru.guru-group.fi (Postfix) with ESMTP id 4F6681000B3;\r
+       Sat,  7 Jun 2014 16:37:41 +0300 (EEST)\r
+From: Tomi Ollila <tomi.ollila@iki.fi>\r
+To: Vladimir Marek <Vladimir.Marek@oracle.com>\r
+Subject: Re: Deduplication ?\r
+In-Reply-To: <20140606104018.GJ2154@virt.cz.oracle.com>\r
+References: <20140602123212.GA12639@virt.cz.oracle.com>\r
+       <87d2ers9mi.fsf@qmul.ac.uk> <m2ppirs8ea.fsf@guru.guru-group.fi>\r
+       <87ppirqtfa.fsf@qmul.ac.uk> <87y4xfz1fi.fsf@nikula.org>\r
+       <cunegz71aw9.fsf@gargravarr.hh.sledj.net>\r
+       <20140606104018.GJ2154@virt.cz.oracle.com>\r
+User-Agent: Notmuch/0.18+28~gcecaba1 (http://notmuchmail.org) Emacs/24.3.1\r
+       (x86_64-unknown-linux-gnu)\r
+X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL\r
+       $[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F\r
+       !)g;OY^,BjTbr)Np:%c_o'jj,Z\r
+Date: Sat, 07 Jun 2014 16:37:41 +0300\r
+Message-ID: <m2fvjgn8m2.fsf@guru.guru-group.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+       <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+       <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 07 Jun 2014 13:38:01 -0000\r
+\r
+On Fri, Jun 06 2014, Vladimir Marek <Vladimir.Marek@oracle.com> wrote:\r
+\r
+> Hi,\r
+>\r
+\r
+ // stuff deleted //\r
+\r
+>\r
+> I'm attaching my perl script if anyone is interested. It's in no way\r
+> complete solution. It is supposed to be used as\r
+>\r
+> notmuch search --output=files --duplicate=2 '*' > dups\r
+> ./dedup # It opens the file 'dups'\r
+>\r
+> The attached version does not remove anyting (the 'unlink' command is\r
+> commented out).\r
+>\r
+>\r
+> Interestingly this does not work (it seems to return all messages):\r
+> notmuch search --output=messages --duplicate=2 '*'\r
+>\r
+> Also I have found that if I run 'notmuch search' and 'notmuch new' at\r
+> the same time, the notmuch search crashes sometimes. That's why I don't\r
+> use\r
+>\r
+> notmuch search ... | ./dedup\r
+>\r
+> Use with care :)\r
+\r
+To me, any perl code that lacks use strict; use warning; looks like a BIG\r
+footgun ;/\r
+\r
+>\r
+> Thank you for your help\r
+> -- \r
+>      Vlad\r
+\r
+\r
+Tomi\r
+\r
+> #!/usr/bin/perl\r
+>\r
+> use Data::Dumper;\r
+> use List::Util;\r
+>\r
+>\r
+> @TO_IGNORE= (\r
+>\r