From: Dmitrijs Ledkovs Date: Thu, 4 Oct 2012 08:17:59 +0000 (+0100) Subject: Re: How to index arbitrary headers? X-Git-Url: http://git.tremily.us/gitweb.cgi?a=commitdiff_plain;h=9458fba49836291c7b33d9beb2e05af584086181;p=notmuch-archives.git Re: How to index arbitrary headers? --- diff --git a/75/32c05dfa93ce0dc5c48312acb7e086be97e1f5 b/75/32c05dfa93ce0dc5c48312acb7e086be97e1f5 new file mode 100644 index 000000000..f6b955a3c --- /dev/null +++ b/75/32c05dfa93ce0dc5c48312acb7e086be97e1f5 @@ -0,0 +1,114 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id D728F431FB6 + for ; Thu, 4 Oct 2012 01:18:02 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id lfUsalZBqIi9 for ; + Thu, 4 Oct 2012 01:18:01 -0700 (PDT) +Received: from mail-bk0-f53.google.com (mail-bk0-f53.google.com + [209.85.214.53]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) + (No client certificate requested) + by olra.theworths.org (Postfix) with ESMTPS id 01E4B431FAE + for ; Thu, 4 Oct 2012 01:18:00 -0700 (PDT) +Received: by mail-bk0-f53.google.com with SMTP id jg15so121917bkc.26 + for ; Thu, 04 Oct 2012 01:17:59 -0700 (PDT) +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=google.com; s=20120113; + h=mime-version:sender:x-originating-ip:in-reply-to:references:date + :x-google-sender-auth:message-id:subject:from:to:cc:content-type + :x-gm-message-state; + bh=HpT/K4LKWhKYQe/SGk/v7kRw45V/trWUFZRad66VnI8=; + b=IfXFjEdhGlRfpE0erhNPNU5wQqEvD+4jwHnCHg/dp1+kdBoiamv9uG9qxZTJmXCflp + xjl2TojaS7kUbMqxNlSRpFhZe2WQ8qg2ckXGVE+NJd/pgjnY5ioBwKxeXGA9PQ1oyam/ + NjM06iIX6tXYq5QVx3kEOAej32nkiKrX1X5FyBBvYx2CtWctCz8ipBm9HwmXwVzQSoSV + n6bOsAr27K7jwuqA86zkCnLUrVuRRcYaU1n+wT9l7CkhJ8ReuzFV6jphRTCiZiZClMa2 + 58T3E0lh4qu5VmslN0rINkzgjqcgq3pknFWsgafosMO/u5YBEquK9XFlG5si+Rd1u7qc + 4FVQ== +MIME-Version: 1.0 +Received: by 10.204.152.211 with SMTP id h19mr1303392bkw.45.1349338679258; + Thu, 04 Oct 2012 01:17:59 -0700 (PDT) +Sender: dmitrij.ledkov@surgut.co.uk +Received: by 10.205.45.4 with HTTP; Thu, 4 Oct 2012 01:17:59 -0700 (PDT) +X-Originating-IP: [86.26.160.15] +In-Reply-To: + +References: + + <20121001154309.GE26662@mit.edu> + +Date: Thu, 4 Oct 2012 09:17:59 +0100 +X-Google-Sender-Auth: jNoVKmyRDED0iYv3bXyIBs00dMk +Message-ID: + +Subject: Re: How to index arbitrary headers? +From: Dmitrijs Ledkovs +To: Petri Savolainen +Content-Type: text/plain; charset=UTF-8 +X-Gm-Message-State: + ALoCoQkEt0N1GzLzMv1SdyxlpT7azR88CY/UOlld5+cxOB/vVWlFywjit5lvcHipM2hyMz5btTSs +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 04 Oct 2012 08:18:03 -0000 + +On 3 October 2012 19:32, Petri Savolainen wrote: +> Hi, +> +> thanks for your response. I am evaluating notmuch / xapian for building an +> application for analyzing in various ways a fairly large number of emails +> accumulated over several years. I am afraid the number of headers that would +> ultimately need to be indexed is therefore quite a lot larger than what +> notmuch currently indexes. +> +> Petri +> +> 2012/10/1 Austin Clements +>> +>> Quoth Petri Savolainen on Oct 01 at 3:39 pm: +>> > Hello, +>> > I could not find information anywhere in notmuch docs about what is +>> > actually indexed - specifically, what email headers are indexed and +>> > searchable? If a header is not indexed, does searching for its value +>> > still +>> > result in a search hit? +>> > It would be nice if one could just provide the list of headers to be +>> > indexed in some configuration file or something. +>> > Thanks, +>> > Petri +>> +>> notmuch doesn't currently implement this, though it is an +>> oft-requested feature. One (not insurmountable) difficulty is that +>> the database would have to be rebuilt if a user-configured list of +>> headers changed and there are technical limitations that prevent us +>> from simply indexing all headers. Out of curiosity, what headers are +>> you interested in indexing? +>> +>> The currently indexed headers are described in man +>> notmuch-search-terms. +> + +Use mapreduce instead: hadoop or discoproject or haddop with dumbo +should be faster. + +Regards, + +Dmitrijs.