--- /dev/null
+Return-Path:\r
+ <return-jmxwjhzeuidrb5in85caytaxb2@temporary-address.scs.stanford.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 4EB406DE13DB\r
+ for <notmuch@notmuchmail.org>; Sun, 23 Aug 2015 13:06:30 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.261\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.261 tagged_above=-999 required=5 tests=[AWL=0.590,\r
+ RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.55, SPF_PASS=-0.001]\r
+ autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id AVNoyFJljYQs for <notmuch@notmuchmail.org>;\r
+ Sun, 23 Aug 2015 13:06:28 -0700 (PDT)\r
+Received: from market.scs.stanford.edu (market.scs.stanford.edu [171.66.3.10])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 5C0D76DE13DA\r
+ for <notmuch@notmuchmail.org>; Sun, 23 Aug 2015 13:06:28 -0700 (PDT)\r
+Received: from market.scs.stanford.edu (localhost.scs.stanford.edu\r
+ [127.0.0.1]) by market.scs.stanford.edu (8.14.7/8.14.7) with ESMTP id\r
+ t7NK6LXZ024280; Sun, 23 Aug 2015 13:06:21 -0700 (PDT)\r
+Received: (from dm@localhost)\r
+ by market.scs.stanford.edu (8.14.7/8.14.7/Submit) id t7NK6KAv003545;\r
+ Sun, 23 Aug 2015 13:06:20 -0700 (PDT)\r
+X-Authentication-Warning: market.scs.stanford.edu: dm set sender to\r
+ return-jmxwjhzeuidrb5in85caytaxb2@ta.scs.stanford.edu using -f\r
+From: David Mazieres <dm-list-email-notmuch@scs.stanford.edu>\r
+To: Amadeusz =?utf-8?B?xbtvxYJub3dza2k=?= <aidecoe@aidecoe.name>,\r
+ notmuch@notmuchmail.org\r
+Subject: Re: muchsync files renames\r
+In-Reply-To: <871teu8kdd.fsf@freja.aidecoe.name>\r
+References: <878u93ujdo.fsf@freja.aidecoe.name>\r
+ <876146o920.fsf@ta.scs.stanford.edu> <871teu8kdd.fsf@freja.aidecoe.name>\r
+Reply-To: David Mazieres expires 2015-11-21 PST\r
+ <mazieres-f72cdtyxmh2n26c2fbep99wgpw@temporary-address.scs.stanford.edu>\r
+Date: Sun, 23 Aug 2015 13:06:20 -0700\r
+Message-ID: <87oahxojlv.fsf@ta.scs.stanford.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=utf-8\r
+Content-Transfer-Encoding: quoted-printable\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.18\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 23 Aug 2015 20:06:30 -0000\r
+\r
+Amadeusz =C5=BBo=C5=82nowski <aidecoe@aidecoe.name> writes:\r
+\r
+> Hi David,\r
+>\r
+> Fist of all thank you for such elaborate answer.\r
+>\r
+> I have missed the paragraph about maildir.synchronize_flags somehow. I\r
+> have it enabled. So this must be source of a problem (?).\r
+\r
+I've only ever tested with mailder.synchronize_flags =3D true, because I'm\r
+worried that if I disable it I will have a hard time re-enabling it. I\r
+do think it is a source of inefficiency, but muchsync should still be\r
+usable.\r
+\r
+> Here follows steps I followed:\r
+>\r
+> 1. I initialized server locally with muchsync -vv. My mail is stored in\r
+> ~/Mail on the server.\r
+> 2. I run muchsync --init ~/mail SERVER. (Directory names do not need to\r
+> be the same, do they?)\r
+\r
+Confirmed that directory names do not need to be the same.\r
+\r
+> 3. I run muchsync SERVER.\r
+> 4. When it lasted much longer then initialization I canceled it by\r
+> single SIGINT (^c).\r
+\r
+Interesting. I wish I knew why this was taking much longer than running\r
+it on the server, and whether the delay was caused by client activity or\r
+server activity.\r
+\r
+I don't suppose you'd be willing to make a copy of your mail database to\r
+repeat the experiment without any risk of messing up your real maildir?\r
+Basically what would be interesting to see is (assuming .notmuch-copy on\r
+server is configured for this disposable copy):\r
+\r
+ # Log everything for later analysis\r
+ script\r
+ # Use new config file location for disposable copy\r
+ export NOTMUCH_CONFIG=3D$HOME/.notmuch-copy\r
+ # Set up a new initial database\r
+ muchsync -vvvv --init ~/test-copy SERVER -vvvv --config=3D.notmuch-copy\r
+\r
+ # Now initial copy is made, see if client is slow\r
+ # Is notmuch new itself slow?\r
+ notmuch new\r
+ # Is resynchronizing muchsync with notmuch slow?\r
+ muchsync -vvvv\r
+\r
+ # Now see if something weird happened on server to make notmuch new slow\r
+ ssh SERVER notmuch --config=3D.notmuch-copy new\r
+ # Now see if something weird happened on server to make muchsync slow\r
+ ssh SERVER muchsync -vvvv --config=3D.notmuch-copy\r
+\r
+ # Now finally try resynchronizing to see if this is slow\r
+ muchsync -vvvv SERVER -vvvv --config=3D.notmuch-copy\r
+ # Save script file\r
+ exit\r
+\r
+Does something along these lines make sense? If you can figure out\r
+which of these is slow (other than --init, which always will be), and\r
+what the verbose chatter is, that would help me narrow down and identify\r
+any problem.\r
+\r
+> 5. I rerun muchsync SERVER and then it notified me that notmuch\r
+> identified files names changes - more than 1000.\r
+\r
+Were the link changes on the client (sent) or the server (received)\r
+side?\r
+\r
+> 6. I waited a bit and then I canceled it by SIGINT.\r
+> 7. I run muchsync --noup SERVER. This took only seconds to finish.\r
+\r
+So this suggests the issue is on the client side. It sounds like a bug.\r
+I wonder if I we can just reproduce this using a public email corpus, so\r
+we don't have to worry about people's private email.\r
+\r
+> I suspected that muchsync at step 3 and 5 tried to push files renames\r
+> back to server. But now I am not sure what was going on. Have I\r
+> desynchronized file mail flags? It's hard to say if anything has broken\r
+> for me, but I am a bit worried anyway.\r
+>\r
+> If I just disable maildir.synchronize_flags and rerun muchsync, will\r
+> everything get synchronized properly?\r
+\r
+I don't think that will change things. maildir.synchronized_flags will\r
+make things slower, but it shouldn't affect the SUMMARY numbers you see\r
+at the end of muchsync, other than maybe files moving from .../new to\r
+.../cur. But presumably most of your mail files were already in cur\r
+directories.\r
+\r
+David\r
projects / notmuch-archives.git / commitdiff