--- /dev/null
+Return-Path: <wking@tremily.us>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by arlo.cworth.org (Postfix) with ESMTP id 6E0206DE0AF8\r
+ for <notmuch@notmuchmail.org>; Sat, 13 Feb 2016 22:31:39 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at cworth.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: 0.007\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=0.007 tagged_above=-999 required=5 tests=[AWL=0.108, \r
+ DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1,\r
+ RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled\r
+Received: from arlo.cworth.org ([127.0.0.1])\r
+ by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id jHIQUFT-wLaf for <notmuch@notmuchmail.org>;\r
+ Sat, 13 Feb 2016 22:31:36 -0800 (PST)\r
+Received: from resqmta-po-10v.sys.comcast.net (resqmta-po-10v.sys.comcast.net\r
+ [96.114.154.169])\r
+ by arlo.cworth.org (Postfix) with ESMTPS id 392CB6DE0A9A\r
+ for <notmuch@notmuchmail.org>; Sat, 13 Feb 2016 22:31:35 -0800 (PST)\r
+Received: from resomta-po-19v.sys.comcast.net ([96.114.154.243])\r
+ by resqmta-po-10v.sys.comcast.net with comcast\r
+ id J6XV1s0025FMDhs016XaqT; Sun, 14 Feb 2016 06:31:34 +0000\r
+Received: from mail.tremily.us ([73.221.72.168])\r
+ by resomta-po-19v.sys.comcast.net with comcast\r
+ id J6XY1s00J3dr3C9016XZ8s; Sun, 14 Feb 2016 06:31:33 +0000\r
+Received: by mail.tremily.us (Postfix, from userid 1000)\r
+ id 7018C1BB1A75; Sat, 13 Feb 2016 22:31:32 -0800 (PST)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tremily.us; s=odin;\r
+ t=1455431492; bh=doMnITzmjEm/nfzTRvyZujrx5mV36yKmPCZQnXGrkss=;\r
+ h=Date:From:To:Cc:Subject:References:In-Reply-To;\r
+ b=kF6iJAdjcQZJNAK/OlXWMxyBDmBpvLuEO1iVr4S+ldRN5+2BJtiVoFFscy0NfeJAh\r
+ 7t1gdCKy2Llv2oUdvto6uohoafvUxg4C+3Tg4TOhmspysMMa5eBk+wERZSfbbllmMP\r
+ 5No8kbxW7dJg41qh6uBUo8rnUDeTZNvMbDv1AUTU=\r
+Date: Sat, 13 Feb 2016 22:31:32 -0800\r
+From: "W. Trevor King" <wking@tremily.us>\r
+To: David Bremner <david@tethera.net>\r
+Cc: notmuch@notmuchmail.org\r
+Subject: Re: problems with nmbug and empty prefix (UnicodeWarning and broken\r
+ pipe)\r
+Message-ID: <20160214063132.GD4265@odin.tremily.us>\r
+References: <87oabko293.fsf@zancas.localnet>\r
+ <20160213223357.GC4265@odin.tremily.us>\r
+ <87ziv4813v.fsf@zancas.localnet>\r
+MIME-Version: 1.0\r
+Content-Type: multipart/signed; micalg=pgp-sha1;\r
+ protocol="application/pgp-signature"; boundary="bMDD5T6Tp9Yu3dvb"\r
+Content-Disposition: inline\r
+In-Reply-To: <87ziv4813v.fsf@zancas.localnet>\r
+OpenPGP: id=39A2F3FA2AB17E5D8764F388FC29BDCDF15F5BE8;\r
+ url=http://tremily.us/pubkey.txt\r
+User-Agent: Mutt/1.5.23 (2014-03-12)\r
+DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net;\r
+ s=q20140121; t=1455431494;\r
+ bh=MDLTWBSn1+hD9ci3RkDBCXFTpKUbbeQrMCqIoP35rPQ=;\r
+ h=Received:Received:Received:Date:From:To:Subject:Message-ID:\r
+ MIME-Version:Content-Type;\r
+ b=mNvKfayACyMkwVoXxWH+zVhLzHNKowXBuMQGYImobuSr/78R7Ry48Oy6gWY/a5kxW\r
+ DDKHc9ALCbmwA+i+t/i3wOzj6qNUiD9ntvk9F0FBn07lG/ujf56gnB7/ZInWSGVeai\r
+ Y09Be4xqDNUeRGu0JApmiTbXS0phTFbr2w7n5rOn3cyjJY5qsL/dC7opgJa22xWLXb\r
+ 5vYUY/EsfcztfIZLRC+7/DV+k87yzGY1RGEVhvPei6OmAVeXs/hGJuFAS43aT+76Zs\r
+ J6ARECHtk7ummoYNt6k36aZv4QeRd0uRnWHdBKaE4Qv5sCT+q2UNSy5dkMP4MmVDng\r
+ E6Cns7RLqNwIQ==\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.20\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <https://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch/>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <https://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sun, 14 Feb 2016 06:31:39 -0000\r
+\r
+\r
+--bMDD5T6Tp9Yu3dvb\r
+Content-Type: text/plain; charset=utf-8\r
+Content-Disposition: inline\r
+Content-Transfer-Encoding: quoted-printable\r
+\r
+On Sat, Feb 13, 2016 at 10:41:40PM -0400, David Bremner wrote:\r
+> Traceback (most recent call last):\r
+> File "/home/bremner/.config/scripts/nmbug.real", line 834, in <module>\r
+> args.func(**kwargs)\r
+> File "/home/bremner/.config/scripts/nmbug.real", line 324, in commit\r
+> status =3D get_status()\r
+> File "/home/bremner/.config/scripts/nmbug.real", line 581, in get_status\r
+> index =3D _index_tags()\r
+> File "/home/bremner/.config/scripts/nmbug.real", line 621, in _index_ta=\r
+gs\r
+> git.stdin.write(line)\r
+\r
+This traceback is pointing at what should be a stream write, so I\r
+don't see how urllib is involved there at all. I guess this traceback\r
+ends up in the =E2=80=9CBroken pipe=E2=80=9D message from your original pos=\r
+t?\r
+\r
+Dropping some debugging prints into the:\r
+\r
+ for line in notmuch.stdout:\r
+\r
+block will likely get us close enough to figure out which line in the\r
+=E2=80=98notmuch dump =E2=80=A6=E2=80=99 output causing the problem.\r
+\r
+> > We only call =E2=80=98notmuch dump =E2=80=A6=E2=80=99 from _index_tags,=\r
+ where dump's stdout is\r
+> > tweaked and fed into =E2=80=98git update-index =E2=80=A6=E2=80=99. You=\r
+r urllib UnicodeWarning\r
+> > suggests the issue lies in:\r
+> >\r
+> > tags =3D [\r
+> > _unquote(tag[len(prefix):])\r
+> > for tag in tags_string.split()\r
+> > if tag.startswith(prefix)]\r
+>=20\r
+> Looking at the source for urllib, that line is actually in quote,\r
+> which is called only from _hex_quote\r
+\r
+And we call _hex_quote from _index_tags_for_message, which is right\r
+before the git.stdin.write line from your traceback. So its certainly\r
+possible that we're feeding _hex_quote something it can't handle in\r
+Python 2. If I could reproduce this locally, I'd probably drop a\r
+debugging print in there as well:\r
+\r
+ for tag in tags:\r
+ _LOG.debug('building a quoted path for {!r} / {!r}'.format(id, tag))\r
+ path =3D 'tags/{id}/{tag}'.format(\r
+ id=3D_hex_quote(string=3Did), tag=3D_hex_quote(string=3Dtag))\r
+ yield '{mode} {hash}\t{path}\n'.format(mode=3Dmode, hash=3Dhash, path=\r
+=3Dpath)\r
+\r
+> Unfortunately despite my best efforts with filterwarnings, I\r
+> couldn't figure out how to get a stack trace for that\r
+> UnicodeWarning.\r
+\r
+I haven't spent much time with filterwarnings. My guess is that:\r
+\r
+ $ python -W error ./nmbug --log-level debug commit\r
+\r
+will turn it into a raised exception [1]. But you may have tried\r
+that, and it may not have worked for some reason :p.\r
+\r
+If dropping debugging prints into the relevant code sections doesn't\r
+turn up the problem, =E2=80=98strace -o /tmp/trace -f nmbug --log-level deb=\r
+ug\r
+commit=E2=80=99 will likely capture enough of the data moving between\r
+processes for us to figure out what nmbug is choking on.\r
+\r
+Another alternative would be to check your list of censored tags for\r
+anything that looks like it might contain Unicode-issue-triggering\r
+characters. What is your locale? Do you have any tags with non-ASCII\r
+characters? You should be able to isolate this problem by iterating\r
+through all your tags:\r
+\r
+ $ for TAG in <censored>\r
+ > do\r
+ > echo "${TAG}"\r
+ > NMBPREFIX=3D"${TAG%?}" nmbug commit\r
+ > done\r
+\r
+and see which one acts up.\r
+\r
+Cheers,\r
+Trevor\r
+\r
+[1]: https://docs.python.org/2/library/warnings.html#warning-filter\r
+\r
+--=20\r
+This email may be signed or encrypted with GnuPG (http://www.gnupg.org).\r
+For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy\r
+\r
+--bMDD5T6Tp9Yu3dvb\r
+Content-Type: application/pgp-signature; name="signature.asc"\r
+Content-Description: OpenPGP digital signature\r
+\r
+-----BEGIN PGP SIGNATURE-----\r
+Version: GnuPG v2\r
+\r
+iQIcBAEBAgAGBQJWwB9BAAoJEAPqygegUbGsJ5wQAKiq2gt5YYHb02ZW4gT3nfv6\r
+2QdO1FlUy9XhPhzwPKdK8NBeIZIhzI0Pu95/kXRauQxPgveZlgaDeBdEPYS9G1hf\r
+yuwwfXKclqKF2dNnieJ7EkdZ2KqPkgOpVWjoffONzGGhNBMUIzFR1pwtoJwxHSwS\r
+XE1Xhp91FBOQ2EbxTCHI1mgwQI49YVhZffIKkg2hrmIqieg/VgrU0fpEGVRCf6zr\r
+yfk1sbUKRxG/fC+B5lLJHhbu+A3a2InJtlgk+IyXXv8UQcFIBA34sxYqWI6Jxujl\r
+Jp5LM49DeiMmaL12PnDeCYJh6MfZdnVqq/LkeAzhVmVSNfRZ9DJUIr4zKlzY3lRU\r
+Bxge7yaEH/ZboQMOM1y06UqHcIBY+05OZNjCep+I/4dlQlYmDPCBR7MY5WdACVl5\r
+Je5F3Fr5ZZwyNT5K/6IhgWCT1d0IWIh8m0GY6VfsFasynzZLSEyOleU7SXFiQAjb\r
+vpk5jWsUIOGlWEwXnv+h7kdwRvY9Q+w8h8wON1RAFI+hEvPV2XFq2xqjvXqebsRd\r
+UdsFRxyQn7AWU3BuKGyo6p5RF1k3W+miCqnDR5h5IZSFQHTDjRUKcfP7rz0ep6CM\r
+so4Y23NTuQbRK20POiE9Ig/3MeCwWYfqs2uMr8/dZ5jg0UCk4oXU/XLwnx50kPxx\r
+injrOxzmqVKm4C9R1DtY\r
+=1g4t\r
+-----END PGP SIGNATURE-----\r
+\r
+--bMDD5T6Tp9Yu3dvb--\r
projects / notmuch-archives.git / commitdiff