1 Return-Path: <amdragon@mit.edu>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 1CFD4431FAF
\r
6 for <notmuch@notmuchmail.org>; Mon, 3 Jun 2013 22:33:44 -0700 (PDT)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5
\r
12 tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id OJ1LkLe6edWA for <notmuch@notmuchmail.org>;
\r
16 Mon, 3 Jun 2013 22:33:36 -0700 (PDT)
\r
17 Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu
\r
19 by olra.theworths.org (Postfix) with ESMTP id D3775431FAE
\r
20 for <notmuch@notmuchmail.org>; Mon, 3 Jun 2013 22:33:35 -0700 (PDT)
\r
21 X-AuditID: 1209190f-b7fb76d0000008e6-e9-51ad7c2f95f3
\r
22 Received: from mailhub-auth-2.mit.edu ( [18.7.62.36])
\r
23 by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP
\r
24 id F0.DE.02278.F2C7DA15; Tue, 4 Jun 2013 01:33:35 -0400 (EDT)
\r
25 Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])
\r
26 by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id r545XXtn026451;
\r
27 Tue, 4 Jun 2013 01:33:33 -0400
\r
28 Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])
\r
29 (authenticated bits=0)
\r
30 (User authenticated as amdragon@ATHENA.MIT.EDU)
\r
31 by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r545XU4w004212
\r
32 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);
\r
33 Tue, 4 Jun 2013 01:33:31 -0400
\r
34 Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)
\r
35 (envelope-from <amdragon@mit.edu>)
\r
36 id 1UjjsA-0005St-F2; Tue, 04 Jun 2013 01:33:30 -0400
\r
37 From: Austin Clements <amdragon@MIT.EDU>
\r
38 To: Jameson Graef Rollins <jrollins@finestructure.net>,
\r
39 notmuch@notmuchmail.org
\r
40 Subject: Re: [PATCH v3 0/6] Make Emacs search use sexp format
\r
41 In-Reply-To: <87r4gk8qa5.fsf@servo.finestructure.net>
\r
42 References: <1370047208-12785-1-git-send-email-amdragon@mit.edu>
\r
43 <87sj12yqyu.fsf@maritornes.cs.unb.ca>
\r
44 <87r4gk8qa5.fsf@servo.finestructure.net>
\r
45 User-Agent: Notmuch/0.15.2+173~g7cdb0c1 (http://notmuchmail.org) Emacs/23.4.1
\r
47 Date: Tue, 04 Jun 2013 01:33:30 -0400
\r
48 Message-ID: <87bo7mtp79.fsf@awakening.csail.mit.edu>
\r
50 Content-Type: text/plain; charset=us-ascii
\r
51 X-Brightmail-Tracker:
\r
52 H4sIAAAAAAAAA+NgFuplleLIzCtJLcpLzFFi42IRYrdT0dWvWRtocHWppMWN1m5Giz37vCyu
\r
53 35zJbPFm5TxWBxaPu6e5PA5/Xcji8WzVLWaPLYfeMwewRHHZpKTmZJalFunbJXBlzDp2k7Fg
\r
54 v3bFs/6ljA2MG5W6GDk5JARMJG4/mMoOYYtJXLi3nq2LkYtDSGAfo8SUP5uYIZwNjBKXT6xh
\r
55 hHBOMUnMePAWqmwJo8Sj28fYQPrZBDQktu1fzghiiwhESMy+up4ZxGYWsJT4c/8UWI2wgK3E
\r
56 /9mvmEBsTgFTiSXbb0FN7WeU2LDqOFiRqECixN1l78FsFgFViTPPj7CA2LxAx1693sIIYQtK
\r
57 nJz5hAVigZbEjX8vmSYwCs5CkpqFJLWAkWkVo2xKbpVubmJmTnFqsm5xcmJeXmqRrolebmaJ
\r
58 XmpK6SZGUEhzSvLvYPx2UOkQowAHoxIPr8KWNYFCrIllxZW5hxglOZiURHm3lawNFOJLyk+p
\r
59 zEgszogvKs1JLT7EKMHBrCTC2+sClONNSaysSi3Kh0lJc7AoifNeTbnpLySQnliSmp2aWpBa
\r
60 BJOV4eBQkuDVrwZqFCxKTU+tSMvMKUFIM3FwggznARouAVLDW1yQmFucmQ6RP8Woy7H5/OR3
\r
61 jEIsefl5qVLivBkgRQIgRRmleXBzYKnoFaM40FvCvJEgVTzANAY36RXQEiagJZNfrwJZUpKI
\r
62 kJJqYJQPKi2dtVc6+1z4zjy760J//z7LmyNQ16rau6Nkv2pgpMru8F+s4gu1pG0SQlmPGn/w
\r
63 WijF8j5HZc9OxxMv9x1ZUqKizm11+Zp7CfOWy8YrmS6/LOUs7niy09w2VMT4tFVh2Lm/Ty02
\r
64 FVirfE1YefDUkfrwuMvuAQvkl2qs0LdY9m3JNke91UosxRmJhlrMRcWJAIRUfDIgAwAA
\r
65 Cc: tomi.ollila@iki.fi
\r
66 X-BeenThere: notmuch@notmuchmail.org
\r
67 X-Mailman-Version: 2.1.13
\r
69 List-Id: "Use and development of the notmuch mail system."
\r
70 <notmuch.notmuchmail.org>
\r
71 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
72 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
73 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
74 List-Post: <mailto:notmuch@notmuchmail.org>
\r
75 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
76 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
77 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
78 X-List-Received-Date: Tue, 04 Jun 2013 05:33:44 -0000
\r
80 On Sun, 02 Jun 2013, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
\r
81 > On Sat, Jun 01 2013, David Bremner <david@tethera.net> wrote:
\r
82 >> Austin Clements <amdragon@MIT.EDU> writes:
\r
84 >>> This is v3 of id:1369934016-22308-1-git-send-email-amdragon@mit.edu.
\r
85 >>> This tweaks the shell invocation as suggested by Tomi and fixes two
\r
86 >>> comment typos pointed out by Mark. It also adds a NEWS patch. I'm
\r
87 >>> going to go ahead and mark this ready because of Tomi's and Mark's
\r
90 >> The first 5 I pushed. The NEWS patch has a conflict.
\r
92 > I'm very happy to see the long-coming sexp handling working here. Good
\r
93 > work, folks, particularly to Austin for getting the awesome asynchronous
\r
94 > processing stuff working. Searches are now definitely noticeably
\r
97 > I am, however, seeing a couple of issues that we might want to address.
\r
99 > * Killing a search buffer that is still in the process of being filled
\r
100 > causes errors to be thrown. I'm seeing both of the following
\r
103 > [Sun Jun 2 08:26:40 2013]
\r
104 > notmuch exited with status killed
\r
105 > command: notmuch search --format\=sexp --format-version\=1 --sort\=newest-first to\:jrollins
\r
106 > exit signal: killed
\r
108 > [Sun Jun 2 08:32:26 2013]
\r
109 > notmuch exited with status hangup
\r
110 > command: notmuch search --format\=sexp --format-version\=1 --sort\=newest-first to\:jrollins
\r
111 > exit signal: hangup
\r
113 > This is somewhat understandable, as the notmuch binary exits with an
\r
114 > error if it hasn't finished dumping the output, but given how common
\r
115 > this particular scenario is I think we should try to avoid throwing
\r
116 > errors in this circumstance. I wonder if we shouldn't just modify the
\r
117 > binary to not return non-zero if it was manually killed while
\r
118 > processing the output, or at least special-case the particular error
\r
119 > caused by manually killing the search.
\r
121 Your assessment is correct, of course. The right place to fix this is
\r
122 in Emacs, not the CLI (the CLI *can't* do anything about this, since it
\r
123 gets killed by a signal). Probably we should do something different in
\r
124 the sentinel if the search process's buffer is no longer live. Clearly
\r
125 we should suppress the status error for the signal, but I think we still
\r
126 should report anything that appeared in err-file because it may be
\r
127 relevant to why the user killed the buffer (e.g., maybe a notmuch
\r
128 wrapper was blocked on something).
\r
130 > * The next thing I'm seeing is this:
\r
132 > Opening input file: no such file or directory, /home/jrollins/tmp/nmerr5390CAY
\r
134 > I'm not exactly sure what causes this error, but it looks to me like
\r
135 > the temporary error file was removed before we were finished with it.
\r
137 This one's pretty awesome (and I think is a bug in Emacs). At a high
\r
138 level, the sentinel is getting run twice and since the first call
\r
139 deletes the error file, the second call fails. At a low level, what
\r
140 causes this is fascinating.
\r
142 1) You kill the search buffer. This invokes kill_buffer_processes,
\r
143 which sends a SIGHUP to notmuch, but doesn't do anything else.
\r
144 Meanwhile, the notmuch search process has printed some more output,
\r
145 but Emacs hasn't consumed it yet (this is critical).
\r
147 2) Emacs gets a SIGCHLD from the dying notmuch process, which invokes
\r
148 handle_child_signal, which sets the new process status, but can't do
\r
149 anything else because it's a signal handler.
\r
151 3) Emacs returns to its idle loop, which calls status_notify, which sees
\r
152 that the notmuch process has a new status. This is where things get
\r
155 3.1) Emacs guarantees that it will run process filters on any unconsumed
\r
156 output before running the process sentinel, so status_notify calls
\r
157 read_process_output, which consumes the final output and calls
\r
158 notmuch-search-process-filter.
\r
160 3.1.1) notmuch-search-process-filter contains code to check if the
\r
161 search buffer is still alive and, since it's not, it calls
\r
164 3.1.1.1) delete-process correctly sees that the process is already dead
\r
165 and doesn't try to send another signal, *but* it still modifies
\r
166 the status to "killed". To deal with the new status, it calls
\r
167 status_notify. Dun dun dun. We've seen this function before.
\r
169 3.1.1.1.1) The *recursive* status_notify invocation sees that the
\r
170 process has a new status and doesn't have any more output to
\r
171 consume, so it invokes our sentinel and returns.
\r
173 3.2) The outer status_notify call (which we're still in) is now done
\r
174 flushing pending process output, so it *also* invokes our sentinel.
\r
176 It might be that the answer is to just remove the delete-process call
\r
177 from the filter. It seems completely redundant (and racy) with Emacs'
\r
178 automatic SIGHUP'ing.
\r
180 > * Finally, something happened that caused *12,000* of the following lines
\r
181 > to be sent to the *Notmuch errors* buffer:
\r
183 > A Xapian exception occurred performing query: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation
\r
185 > Again, this was related to killing a search buffer that was still
\r
186 > being filled. I'm pretty sure the database was not modified during
\r
189 I have no insight on this one. My best guess is that this has nothing
\r
190 to do with this change except that this change makes these warnings
\r
191 visible rather than burying them somewhere down in the search results
\r
194 > Let me know if I can help provide any more info.
\r
projects / notmuch-archives.git / blob