1 Return-Path: <thomas@schwinge.name>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 1DC20431FB6
\r
6 for <notmuch@notmuchmail.org>; Wed, 2 Feb 2011 08:12:36 -0800 (PST)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=0 tagged_above=-999 required=5
\r
12 tests=[RCVD_IN_DNSWL_NONE=-0.0001] autolearn=disabled
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id ZfyjmOcJ06Vf for <notmuch@notmuchmail.org>;
\r
16 Wed, 2 Feb 2011 08:12:35 -0800 (PST)
\r
17 Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de
\r
19 by olra.theworths.org (Postfix) with ESMTP id D06AE431FB5
\r
20 for <notmuch@notmuchmail.org>; Wed, 2 Feb 2011 08:12:34 -0800 (PST)
\r
21 Received: from [87.180.46.33] (helo=stokes.schwinge.homeip.net)
\r
22 by smtprelay02.ispgateway.de with esmtpa (Exim 4.68)
\r
23 (envelope-from <thomas@schwinge.name>) id 1PkfJo-00079S-3A
\r
24 for notmuch@notmuchmail.org; Wed, 02 Feb 2011 17:12:32 +0100
\r
25 Received: (qmail 2062 invoked from network); 2 Feb 2011 16:12:21 -0000
\r
26 Received: from kepler.schwinge.homeip.net (192.168.111.7)
\r
27 by stokes.schwinge.homeip.net with QMQP; 2 Feb 2011 16:12:21 -0000
\r
28 Received: (nullmailer pid 8428 invoked by uid 1000);
\r
29 Wed, 02 Feb 2011 16:12:21 -0000
\r
30 From: Thomas Schwinge <thomas@schwinge.name>
\r
31 To: notmuch@notmuchmail.org
\r
32 Subject: Re: [BUG] Emacs UI dropping every 25th line, roughly
\r
33 In-Reply-To: <87tygqm7g4.fsf@kepler.schwinge.homeip.net>
\r
34 References: <87tygqm7g4.fsf@kepler.schwinge.homeip.net>
\r
35 User-Agent: Notmuch/0.5-33-g665f77b (http://notmuchmail.org) Emacs/23.2.1
\r
37 Date: Wed, 02 Feb 2011 17:12:16 +0100
\r
38 Message-ID: <87zkqeiffj.fsf@kepler.schwinge.homeip.net>
\r
40 Content-Type: multipart/signed; boundary="=-=-=";
\r
41 micalg=pgp-sha1; protocol="application/pgp-signature"
\r
42 X-Df-Sender: thomas@schwinge.name
\r
43 X-BeenThere: notmuch@notmuchmail.org
\r
44 X-Mailman-Version: 2.1.13
\r
46 List-Id: "Use and development of the notmuch mail system."
\r
47 <notmuch.notmuchmail.org>
\r
48 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
49 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
50 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
51 List-Post: <mailto:notmuch@notmuchmail.org>
\r
52 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
53 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
54 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
55 X-List-Received-Date: Wed, 02 Feb 2011 16:12:36 -0000
\r
58 Content-Type: text/plain; charset=utf-8
\r
59 Content-Transfer-Encoding: quoted-printable
\r
63 On Sun, 30 Jan 2011 22:02:03 +0100, I wrote:
\r
64 > That is, roughly every 25th line is dropped in the Emacs notmuch-search
\r
65 > buffer! (Via setting notmuch-command to a shell script, I intercepted
\r
66 > the notmuch / emacs pipeline with tee, and found that what comes out of
\r
67 > ``notmuch | tee'' is still sane, so I'm fairly sure it's an Emacs /
\r
68 > notmuch elisp code issue.)
\r
70 > And, 25 times the medium length of a ``notmuch search [...]'' output line
\r
71 > is... 4 KiB, the standard page size. Is this ``just'' a problem in the
\r
72 > notmuch elisp code, or is Emacs doing something awful with (wild
\r
73 > speculation...) short reads on buffer page boundaries (or whatever else)?
\r
75 I began to analyze this. Here is a dump of my steps.
\r
77 I used strace on the (already running) Emacs process. strace would log
\r
78 to several files: one for the main (Emacs) process, and a new one for
\r
79 each forked process. I can see that a ``notmuch search [...]'' process
\r
80 is fork()ed / exec()uted. It does a lot of read()ing from the DB, and
\r
81 write()s out the results record by record (which in our case means line
\r
82 by line). Up to the point where I examined, there have been no short
\r
85 [some write()s for the first two dozen search results]
\r
86 16:26:54.143464 write(1, "thread:00000000000006c7 2009-12-16 [2/2] Th=
\r
87 omas Schwinge; [subject] ([flags])\n", 135) =3D 135
\r
89 16:26:54.266866 write(1, "thread:0000000000000ac4 2009-12-16 [1/1] js=
\r
90 m28@sourceware.org; [subject] ([flags])\n", 181) =3D 181
\r
92 At this / in the middle of this point, the 4 KiB size is hit. After
\r
93 accumulating some more (notice the 3.6 seconds delay), Emacs read()s the
\r
94 first chunk (line breaks inserted for clarity):
\r
96 16:26:57.928798 read(8, "[first two dozen results]"
\r
97 "thread:00000000000006c7 2009-12-16 [2/2] Tho=
\r
98 mas Schwinge; [subject] ([flags])\n"
\r
101 The last result line is obviously incomplete, only the `t' so far.
\r
103 There is more to be read, so Emacs quickly goes on with the next chunk:
\r
105 16:26:57.966247 read(8, "hread:0000000000000ac4 2009-12-16 [1/1] jsm2=
\r
106 8@sourceware.org; [subject] ([flags])\n"
\r
107 "[more results]", 4096) =3D 4095
\r
109 That's the remainder of the partial line, plus further results. This is
\r
110 all fine. It's now Emacs' job to re-assemble the two partial lines
\r
111 (thread ac4). I note that indeed this search result is missing in the
\r
112 notmuch-search buffer.
\r
114 (By the way, I wonder why read() returns 4095 bytes instead of 4096?)
\r
116 I'll next move on to debugging what actually appears at the elisp layer.
\r
117 I somehow don't think the Emacs core is faulty; probably rather our
\r
118 asynchronous results presentation layer handles partial lines
\r
119 incorrectly? More to come later.
\r
126 Content-Type: application/pgp-signature
\r
128 -----BEGIN PGP SIGNATURE-----
\r
129 Version: GnuPG v1.4.10 (GNU/Linux)
\r
131 iQEcBAEBAgAGBQJNSYJgAAoJEGe3hdm9kOiiqm4H/ReVsJ9ND/UH+IMKkvMvyzsJ
\r
132 DJGg8hjrzfmFJmIKyC8I0+JHtHBGQtExgEy4GfkTIKkt1/7n/fVm0JVOG0p3yJY8
\r
133 mdagH7Cf3nI34zPn5vfZjtD0rn8EaezKbus67oO+3NGbjwgXEwg6K6RUdQSOhOq/
\r
134 V8giOpUwfjf48U2xVsg6+09jQnRf51LylbC2NoKEQyc/shDmsHV2rbrf/Ml6Plhp
\r
135 /O8AvhWNHoYx4LpUDvPY4U0bjEp5/rPPPIOJY/VcgDXRztpMABoGJsnQnu684k06
\r
136 kzWi8yn7BHzjL9ZjSclVm8/J/v168nW1TtOJ5nxKtfaSYP4l7J0AX1ENEiZV8Xs=
\r
138 -----END PGP SIGNATURE-----
\r