--- /dev/null
+Return-Path:\r
+ <return-tscnjiupa5jk2z8akbff4tt9se@temporary-address.scs.stanford.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 344406DE1B1B\r
+ for <notmuch@notmuchmail.org>; Sat, 22 Aug 2015 22:42:12 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -2.208\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-2.208 tagged_above=-999 required=5 tests=[AWL=0.643,\r
+ RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.55, SPF_PASS=-0.001]\r
+ autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id cd5ew9ssLEEv for <notmuch@notmuchmail.org>;\r
+ Sat, 22 Aug 2015 22:42:09 -0700 (PDT)\r
+Received: from market.scs.stanford.edu (market.scs.stanford.edu [171.66.3.10])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 6C6FD6DE18EF\r
+ for <notmuch@notmuchmail.org>; Sat, 22 Aug 2015 22:42:09 -0700 (PDT)\r
+Received: from market.scs.stanford.edu (localhost.scs.stanford.edu\r
+ [127.0.0.1]) by market.scs.stanford.edu (8.14.7/8.14.7) with ESMTP id\r
+ t7N5g1Cj019244; Sat, 22 Aug 2015 22:42:01 -0700 (PDT)\r
+Received: (from dm@localhost)\r
+ by market.scs.stanford.edu (8.14.7/8.14.7/Submit) id t7N5g0Dj012017;\r
+ Sat, 22 Aug 2015 22:42:00 -0700 (PDT)\r
+X-Authentication-Warning: market.scs.stanford.edu: dm set sender to\r
+ return-tscnjiupa5jk2z8akbff4tt9se@ta.scs.stanford.edu using -f\r
+From: David Mazieres <dm-list-email-notmuch@scs.stanford.edu>\r
+To: Amadeusz =?utf-8?B?xbtvxYJub3dza2k=?= <aidecoe@aidecoe.name>,\r
+ notmuch@notmuchmail.org\r
+Subject: Re: muchsync files renames\r
+In-Reply-To: <878u93ujdo.fsf@freja.aidecoe.name>\r
+References: <878u93ujdo.fsf@freja.aidecoe.name>\r
+Date: Sat, 22 Aug 2015 22:41:59 -0700\r
+Message-ID: <876146o920.fsf@ta.scs.stanford.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=utf-8\r
+Content-Transfer-Encoding: quoted-printable\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.18\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 23 Aug 2015 05:42:12 -0000\r
+\r
+Amadeusz =C5=BBo=C5=82nowski <aidecoe@aidecoe.name> writes:\r
+\r
+> Hi,\r
+>\r
+> I am testing muchsync-2 and it looks to me that files names across\r
+> machines are different. Moreover when syncing again after\r
+> initialization it seems muchsync is working on something. I have\r
+> canceled this and rerun muchsync. notmuch reported lots of files\r
+> renames on server. What and why it happens?\r
+\r
+What muchsync specifically synchronizes for messages in the mapping:\r
+\r
+ (directory, SHA-1-hash, link-count)\r
+\r
+So if a directory contains two copies of a file on one machine, it will\r
+end up with two copies on the other machine. However, the file names\r
+themselves are not the same, but rather are created in accordance with\r
+the maildir spec. (Note SHA-1 wouldn't be my first choice of hash\r
+function, but notmuch already uses this for messages with long message\r
+IDs, so I figured I'd just be consistent with existing practice.)\r
+\r
+In terms of what muchsync is working on, you can run it with "-vvvv" on\r
+both sides to get an idea, as in "muchsync -vvvv server -vvvv". Better\r
+yet, you can just run it on one side with "muchsync -vvvv". You'll get\r
+a lot of output, so maybe run it inside the script command to save the\r
+output.maybe run it inside the script command to save the output. If\r
+you have enabled maildir.synchronize_flags, it could be that notmuch is\r
+initially renaming all of your files, in which case muchsync needs to\r
+re-hash them to make sure they haven't changed.\r
+\r
+How did you cancel muchsync? If you send it a single SIGINT or SIGTERM,\r
+it attempts to clean up after itself. However, upon multiple signals or\r
+other signals, it immediately exits. Muchsync is conservative about\r
+updating the database, to avoid missing tags or files that have been\r
+changed. It always updates the notmuch database first, then its own\r
+sqlite database with a version number. That means if you kill muchsync,\r
+some number of files may get picked up as changed again even though\r
+really they were just copied from a peer.\r
+\r
+To mitigate this problem, the muchsync client syncs the database every\r
+10 seconds, so that in theory you should only get 10 seconds of extra\r
+work from killing the client. However, the server does not sync\r
+periodically, on the assumption that it is more likely to read an EOF\r
+than get killed, although currently it doesn't appear to commit any\r
+pending transactions to the sqlite database upon EOF, which may be an\r
+oversight.\r
+\r
+So to summarize:\r
+\r
+ * File names are not the same across machine, only file contents and\r
+ directory structure.\r
+\r
+ * Give muchsync lots of "-v" options to see what it is doing.\r
+\r
+ * Try to avoid killing muchsync. Doing so is safe, but likely to\r
+ generate extra work in the form of phantom renames or tag changes\r
+ that get synchronized even though they don't need to be.\r
+\r
+ * Possibly the server should handle EOF more gracefully and commit any\r
+ pending transactions, or the client should periodically send a\r
+ commit command to the server.\r
+\r
+If you think something is wrong, I can help you figure it out, but I\r
+need to know what maildir.synchronize_flags is set to on each replica,\r
+what you mean by "canceled", and roughly what was happening when you\r
+canceled (uploading or downloading).\r
+\r
+David\r
projects / notmuch-archives.git / commitdiff