From: Vladimir Marek Date: Tue, 19 Jul 2016 10:36:11 +0000 (+0200) Subject: Re: Avoiding the "huge INBOX of death" X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=7d66e15c94f9fa8392cdf29b0fb25a57fcb925cd;p=notmuch-archives.git Re: Avoiding the "huge INBOX of death" --- diff --git a/97/c3796b213d5d6f5d1f08100b4806162f666819 b/97/c3796b213d5d6f5d1f08100b4806162f666819 new file mode 100644 index 000000000..65803eff1 --- /dev/null +++ b/97/c3796b213d5d6f5d1f08100b4806162f666819 @@ -0,0 +1,136 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id 371C76DE015B + for ; Tue, 19 Jul 2016 03:52:02 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: -0.096 +X-Spam-Level: +X-Spam-Status: No, score=-0.096 tagged_above=-999 required=5 tests=[AWL=2.214, + RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, + UNPARSEABLE_RELAY=0.001] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id 0MieoV6E8xKD for ; + Tue, 19 Jul 2016 03:51:54 -0700 (PDT) +X-Greylist: delayed 935 seconds by postgrey-1.35 at arlo; + Tue, 19 Jul 2016 03:51:54 PDT +Received: from userp1050.oracle.com (userp1050.oracle.com [156.151.31.82]) + by arlo.cworth.org (Postfix) with ESMTPS id 08F8B6DE012F + for ; Tue, 19 Jul 2016 03:51:53 -0700 (PDT) +Received: from userp1040.oracle.com (userp1040.oracle.com [156.151.31.81]) + by userp1050.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id + u6JAaKk0000382 + (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) + for ; Tue, 19 Jul 2016 10:36:20 GMT +Received: from aserv0022.oracle.com (aserv0022.oracle.com [141.146.126.234]) + by userp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id + u6JAaGRf014460 + (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); + Tue, 19 Jul 2016 10:36:17 GMT +Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) + by aserv0022.oracle.com (8.13.8/8.13.8) with ESMTP id u6JAaG1C002107 + (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); + Tue, 19 Jul 2016 10:36:16 GMT +Received: from abhmp0018.oracle.com (abhmp0018.oracle.com [141.146.116.24]) + by userv0121.oracle.com (8.13.8/8.13.8) with ESMTP id u6JAaFRe004424; + Tue, 19 Jul 2016 10:36:15 GMT +Received: from virtual.cz.oracle.com (/10.163.86.11) + by default (Oracle Beehive Gateway v4.0) + with ESMTP ; Tue, 19 Jul 2016 03:36:14 -0700 +Date: Tue, 19 Jul 2016 12:36:11 +0200 +From: Vladimir Marek +To: "=?utf-8?Q?Rapha=C3=ABl?= Fournier-S'niehotta" + +Cc: notmuch@notmuchmail.org +Subject: Re: Avoiding the "huge INBOX of death" +Message-ID: <20160719103610.GB972@virtual.cz.oracle.com> +References: <20160719094205.qmf5sjnja6crt5t3@gotlib> +MIME-Version: 1.0 +Content-Type: text/plain; charset=utf-8 +Content-Disposition: inline +In-Reply-To: <20160719094205.qmf5sjnja6crt5t3@gotlib> +User-Agent: Mutt/1.5.22.1-rc1 (2013-10-16) +X-Source-IP: userp1040.oracle.com [156.151.31.81] +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.20 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 19 Jul 2016 10:52:02 -0000 + +I can't speak for everyone, but here's what I am using. + +I do have 'messages' directory which contains 'imap' and 'archive'. + +messages + \-imap + \-archive + +The imap directory is being synchronized via offlineimap to the server. + +In the offlineimap postsync hook I call a script which does some +bookkeeping on the newly received messages and it also marks old and +already read and not flagged emails by the 'old-tread' tag. + +N=notmuch +while : ; do + AGE=100d # 100 days + # Find all threads containing old messages + OLD_MESSAGES=$( $N search --output=threads not tag:archive and date:..$AGE and not tag:unread and not tag:flagged ) + [ -n "$OLD_MESSAGES" ] || break + # Find threads containing newer messages + NEW_MESSAGES=$( $N search --output=threads not tag:archive and not '(' date:..$AGE or tag:unread or tag:flagged ')' ) + # remove the threads containing newer messages + OLD_THREADS=$( /usr/xpg4/bin/fgrep -v -x -f <(echo "$NEW_MESSAGES") < <(echo "$OLD_MESSAGES") ) + [ -n "$OLD_THREADS" ] || break + $N tag +old-thread -- $OLD_THREADS + break +done + +# If old thread received newer message, unmark it too +$N tag -old-thread -- tag:old-thread and '(' not date:..$AGE or tag:unread or tag:flagged or tag:archive ')' + + +Once in a while I review the old-thread messages and assign them by hand +'archive' flag. The postsync hook is looking for this flag and moves all +messages to be archived into archive folder. You can list files to be +archived by + +$N search --output=files 'path:imap/**' tag:archive + +I won't list the commands to move files from messages/imap to +messages/archive since I have a bit more complex scenario. But it should +be straightforward. + +On next run offlineimap sees that the messages disappeared from 'imap' +folder and removes them from imap server. + +$ notmuch count '*' +734346 +$ notmuch count 'path:imap/**' +18419 + + +In my case I am server side filtering the emails into several folders so +that I have at least some order in emails when reading email via phone, +so 'path:imap/**' contains all the messages in whole imap tree. + +$ notmuch count 'path:imap/** and tag:inbox and tag:unread' +11 + +I'm trying to have my virtual INBOX empty most of the time. + +I'm interested what tricks others use! +Cheers +-- + Vlad