--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 1CFD4431FAF\r
+ for <notmuch@notmuchmail.org>; Mon, 3 Jun 2013 22:33:44 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id OJ1LkLe6edWA for <notmuch@notmuchmail.org>;\r
+ Mon, 3 Jun 2013 22:33:36 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-4.mit.edu (dmz-mailsec-scanner-4.mit.edu\r
+ [18.9.25.15])\r
+ by olra.theworths.org (Postfix) with ESMTP id D3775431FAE\r
+ for <notmuch@notmuchmail.org>; Mon, 3 Jun 2013 22:33:35 -0700 (PDT)\r
+X-AuditID: 1209190f-b7fb76d0000008e6-e9-51ad7c2f95f3\r
+Received: from mailhub-auth-2.mit.edu ( [18.7.62.36])\r
+ by dmz-mailsec-scanner-4.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id F0.DE.02278.F2C7DA15; Tue, 4 Jun 2013 01:33:35 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+ by mailhub-auth-2.mit.edu (8.13.8/8.9.2) with ESMTP id r545XXtn026451; \r
+ Tue, 4 Jun 2013 01:33:33 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id r545XU4w004212\r
+ (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+ Tue, 4 Jun 2013 01:33:31 -0400\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1UjjsA-0005St-F2; Tue, 04 Jun 2013 01:33:30 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Jameson Graef Rollins <jrollins@finestructure.net>,\r
+ notmuch@notmuchmail.org\r
+Subject: Re: [PATCH v3 0/6] Make Emacs search use sexp format\r
+In-Reply-To: <87r4gk8qa5.fsf@servo.finestructure.net>\r
+References: <1370047208-12785-1-git-send-email-amdragon@mit.edu>\r
+ <87sj12yqyu.fsf@maritornes.cs.unb.ca>\r
+ <87r4gk8qa5.fsf@servo.finestructure.net>\r
+User-Agent: Notmuch/0.15.2+173~g7cdb0c1 (http://notmuchmail.org) Emacs/23.4.1\r
+ (i486-pc-linux-gnu)\r
+Date: Tue, 04 Jun 2013 01:33:30 -0400\r
+Message-ID: <87bo7mtp79.fsf@awakening.csail.mit.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFuplleLIzCtJLcpLzFFi42IRYrdT0dWvWRtocHWppMWN1m5Giz37vCyu\r
+ 35zJbPFm5TxWBxaPu6e5PA5/Xcji8WzVLWaPLYfeMwewRHHZpKTmZJalFunbJXBlzDp2k7Fg\r
+ v3bFs/6ljA2MG5W6GDk5JARMJG4/mMoOYYtJXLi3nq2LkYtDSGAfo8SUP5uYIZwNjBKXT6xh\r
+ hHBOMUnMePAWqmwJo8Sj28fYQPrZBDQktu1fzghiiwhESMy+up4ZxGYWsJT4c/8UWI2wgK3E\r
+ /9mvmEBsTgFTiSXbb0FN7WeU2LDqOFiRqECixN1l78FsFgFViTPPj7CA2LxAx1693sIIYQtK\r
+ nJz5hAVigZbEjX8vmSYwCs5CkpqFJLWAkWkVo2xKbpVubmJmTnFqsm5xcmJeXmqRrolebmaJ\r
+ XmpK6SZGUEhzSvLvYPx2UOkQowAHoxIPr8KWNYFCrIllxZW5hxglOZiURHm3lawNFOJLyk+p\r
+ zEgszogvKs1JLT7EKMHBrCTC2+sClONNSaysSi3Kh0lJc7AoifNeTbnpLySQnliSmp2aWpBa\r
+ BJOV4eBQkuDVrwZqFCxKTU+tSMvMKUFIM3FwggznARouAVLDW1yQmFucmQ6RP8Woy7H5/OR3\r
+ jEIsefl5qVLivBkgRQIgRRmleXBzYKnoFaM40FvCvJEgVTzANAY36RXQEiagJZNfrwJZUpKI\r
+ kJJqYJQPKi2dtVc6+1z4zjy760J//z7LmyNQ16rau6Nkv2pgpMru8F+s4gu1pG0SQlmPGn/w\r
+ WijF8j5HZc9OxxMv9x1ZUqKizm11+Zp7CfOWy8YrmS6/LOUs7niy09w2VMT4tFVh2Lm/Ty02\r
+ FVirfE1YefDUkfrwuMvuAQvkl2qs0LdY9m3JNke91UosxRmJhlrMRcWJAIRUfDIgAwAA\r
+Cc: tomi.ollila@iki.fi\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Tue, 04 Jun 2013 05:33:44 -0000\r
+\r
+On Sun, 02 Jun 2013, Jameson Graef Rollins <jrollins@finestructure.net> wrote:\r
+> On Sat, Jun 01 2013, David Bremner <david@tethera.net> wrote:\r
+>> Austin Clements <amdragon@MIT.EDU> writes:\r
+>>\r
+>>> This is v3 of id:1369934016-22308-1-git-send-email-amdragon@mit.edu.\r
+>>> This tweaks the shell invocation as suggested by Tomi and fixes two\r
+>>> comment typos pointed out by Mark. It also adds a NEWS patch. I'm\r
+>>> going to go ahead and mark this ready because of Tomi's and Mark's\r
+>>> reviews of v2.\r
+>>\r
+>> The first 5 I pushed. The NEWS patch has a conflict.\r
+>\r
+> I'm very happy to see the long-coming sexp handling working here. Good\r
+> work, folks, particularly to Austin for getting the awesome asynchronous\r
+> processing stuff working. Searches are now definitely noticeably\r
+> faster.\r
+>\r
+> I am, however, seeing a couple of issues that we might want to address.\r
+>\r
+> * Killing a search buffer that is still in the process of being filled\r
+> causes errors to be thrown. I'm seeing both of the following\r
+> intermittently:\r
+>\r
+> [Sun Jun 2 08:26:40 2013]\r
+> notmuch exited with status killed\r
+> command: notmuch search --format\=sexp --format-version\=1 --sort\=newest-first to\:jrollins\r
+> exit signal: killed\r
+>\r
+> [Sun Jun 2 08:32:26 2013]\r
+> notmuch exited with status hangup\r
+> command: notmuch search --format\=sexp --format-version\=1 --sort\=newest-first to\:jrollins\r
+> exit signal: hangup\r
+>\r
+> This is somewhat understandable, as the notmuch binary exits with an\r
+> error if it hasn't finished dumping the output, but given how common\r
+> this particular scenario is I think we should try to avoid throwing\r
+> errors in this circumstance. I wonder if we shouldn't just modify the\r
+> binary to not return non-zero if it was manually killed while\r
+> processing the output, or at least special-case the particular error\r
+> caused by manually killing the search.\r
+\r
+Your assessment is correct, of course. The right place to fix this is\r
+in Emacs, not the CLI (the CLI *can't* do anything about this, since it\r
+gets killed by a signal). Probably we should do something different in\r
+the sentinel if the search process's buffer is no longer live. Clearly\r
+we should suppress the status error for the signal, but I think we still\r
+should report anything that appeared in err-file because it may be\r
+relevant to why the user killed the buffer (e.g., maybe a notmuch\r
+wrapper was blocked on something).\r
+\r
+> * The next thing I'm seeing is this:\r
+>\r
+> Opening input file: no such file or directory, /home/jrollins/tmp/nmerr5390CAY\r
+>\r
+> I'm not exactly sure what causes this error, but it looks to me like\r
+> the temporary error file was removed before we were finished with it.\r
+\r
+This one's pretty awesome (and I think is a bug in Emacs). At a high\r
+level, the sentinel is getting run twice and since the first call\r
+deletes the error file, the second call fails. At a low level, what\r
+causes this is fascinating.\r
+\r
+1) You kill the search buffer. This invokes kill_buffer_processes,\r
+ which sends a SIGHUP to notmuch, but doesn't do anything else.\r
+ Meanwhile, the notmuch search process has printed some more output,\r
+ but Emacs hasn't consumed it yet (this is critical).\r
+\r
+2) Emacs gets a SIGCHLD from the dying notmuch process, which invokes\r
+ handle_child_signal, which sets the new process status, but can't do\r
+ anything else because it's a signal handler.\r
+\r
+3) Emacs returns to its idle loop, which calls status_notify, which sees\r
+ that the notmuch process has a new status. This is where things get\r
+ interesting.\r
+\r
+3.1) Emacs guarantees that it will run process filters on any unconsumed\r
+ output before running the process sentinel, so status_notify calls\r
+ read_process_output, which consumes the final output and calls\r
+ notmuch-search-process-filter.\r
+\r
+3.1.1) notmuch-search-process-filter contains code to check if the\r
+ search buffer is still alive and, since it's not, it calls\r
+ delete-process.\r
+\r
+3.1.1.1) delete-process correctly sees that the process is already dead\r
+ and doesn't try to send another signal, *but* it still modifies\r
+ the status to "killed". To deal with the new status, it calls\r
+ status_notify. Dun dun dun. We've seen this function before.\r
+\r
+3.1.1.1.1) The *recursive* status_notify invocation sees that the\r
+ process has a new status and doesn't have any more output to\r
+ consume, so it invokes our sentinel and returns.\r
+\r
+3.2) The outer status_notify call (which we're still in) is now done\r
+ flushing pending process output, so it *also* invokes our sentinel.\r
+\r
+It might be that the answer is to just remove the delete-process call\r
+from the filter. It seems completely redundant (and racy) with Emacs'\r
+automatic SIGHUP'ing.\r
+\r
+> * Finally, something happened that caused *12,000* of the following lines\r
+> to be sent to the *Notmuch errors* buffer:\r
+>\r
+> A Xapian exception occurred performing query: The revision being read has been discarded - you should call Xapian::Database::reopen() and retry the operation\r
+>\r
+> Again, this was related to killing a search buffer that was still\r
+> being filled. I'm pretty sure the database was not modified during\r
+> this process.\r
+\r
+I have no insight on this one. My best guess is that this has nothing\r
+to do with this change except that this change makes these warnings\r
+visible rather than burying them somewhere down in the search results\r
+buffer.\r
+\r
+> Let me know if I can help provide any more info.\r
+>\r
+> jamie.\r
projects / notmuch-archives.git / commitdiff