1 Return-Path: <thomas@schwinge.name>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id 2F9DC431FB6
\r
6 for <notmuch@notmuchmail.org>; Fri, 28 Jan 2011 01:45:46 -0800 (PST)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=0 tagged_above=-999 required=5
\r
12 tests=[RCVD_IN_DNSWL_NONE=-0.0001] autolearn=unavailable
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id UOvd+mYWrwex for <notmuch@notmuchmail.org>;
\r
16 Fri, 28 Jan 2011 01:45:46 -0800 (PST)
\r
17 Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de
\r
19 by olra.theworths.org (Postfix) with ESMTP id C2183431FB5
\r
20 for <notmuch@notmuchmail.org>; Fri, 28 Jan 2011 01:45:45 -0800 (PST)
\r
21 Received: from [87.180.40.233] (helo=stokes.schwinge.homeip.net)
\r
22 by smtprelay02.ispgateway.de with esmtpa (Exim 4.68)
\r
23 (envelope-from <thomas@schwinge.name>) id 1Piktj-0007za-0x
\r
24 for notmuch@notmuchmail.org; Fri, 28 Jan 2011 10:45:43 +0100
\r
25 Received: (qmail 18234 invoked from network); 28 Jan 2011 09:45:25 -0000
\r
26 Received: from kepler.schwinge.homeip.net (192.168.111.7)
\r
27 by stokes.schwinge.homeip.net with QMQP; 28 Jan 2011 09:45:25 -0000
\r
28 Received: (nullmailer pid 18660 invoked by uid 1000);
\r
29 Fri, 28 Jan 2011 09:45:25 -0000
\r
30 From: Thomas Schwinge <thomas@schwinge.name>
\r
31 To: Carl Worth <cworth@cworth.org>, Austin Clements <amdragon@mit.edu>,
\r
32 Jameson Rollins <jrollins@finestructure.net>
\r
33 Subject: Re: notmuch's idea of concurrency / failing an invocation
\r
34 In-Reply-To: <87fwsdobpy.fsf@yoom.home.cworth.org>
\r
35 References: <87fwsetdin.fsf@kepler.schwinge.homeip.net>
\r
36 <8762taxk9y.fsf@algae.riseup.net>
\r
37 <87vd1a84qj.fsf@servo.finestructure.net>
\r
38 <AANLkTi=pPgcPXUN_XTXa0gAkbTR_OF4XxCP-d4-Be2pt@mail.gmail.com>
\r
39 <87fwsdobpy.fsf@yoom.home.cworth.org>
\r
40 User-Agent: Notmuch/0.5-33-g665f77b (http://notmuchmail.org) Emacs/23.2.1
\r
42 Date: Fri, 28 Jan 2011 10:45:19 +0100
\r
43 Message-ID: <877hdps6og.fsf@kepler.schwinge.homeip.net>
\r
45 Content-Type: multipart/signed; boundary="=-=-=";
\r
46 micalg=pgp-sha1; protocol="application/pgp-signature"
\r
47 X-Df-Sender: thomas@schwinge.name
\r
48 Cc: notmuch@notmuchmail.org
\r
49 X-BeenThere: notmuch@notmuchmail.org
\r
50 X-Mailman-Version: 2.1.13
\r
52 List-Id: "Use and development of the notmuch mail system."
\r
53 <notmuch.notmuchmail.org>
\r
54 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
55 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
56 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
57 List-Post: <mailto:notmuch@notmuchmail.org>
\r
58 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
59 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
60 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
61 X-List-Received-Date: Fri, 28 Jan 2011 09:45:46 -0000
\r
64 Content-Type: text/plain; charset=utf-8
\r
65 Content-Transfer-Encoding: quoted-printable
\r
69 On Fri, 28 Jan 2011 15:10:01 +1000, Carl Worth <cworth@cworth.org> wrote:
\r
70 > On Thu, 27 Jan 2011 17:20:21 -0500, Austin Clements <amdragon@mit.edu> wr=
\r
72 > > I'm looking into breaking notmuch new up into small transactions. It
\r
73 > > wouldn't be much a leap from there to simply close and reopen the datab=
\r
75 > > between transactions if another task wants to use it, which would relea=
\r
77 > > the lock and let the queued notmuch task have the database for a bit.
\r
79 > That sounds like something very useful to pursue. Please continue!
\r
81 Ack! And actually -- I just wondered about that: what happens if
\r
82 ``notmuch new'' has executed notmuch_database_add_message for a new
\r
83 message M, but then is killed before adding any tags and finishing up
\r
84 (and supposing that the DB isn't in an invalid state now). This process
\r
85 of adding M to the DB and applying any tags isn't one single transaction
\r
86 currently, right? (And this is exactly what you're working on
\r
87 chainging?) Am I right that what currently happens is that upon the next
\r
88 ``notmuch new'' run, notmuch will not reconsider M (given that it already
\r
89 is present in the DB), but continue with the next messages -- thus
\r
90 leaving M without any tags? This isn't a very likely scenario, but still
\r
93 > > It seems silly to have a daemon when all of notmuch's state is already =
\r
95 > > and queue on a lock is as good as a queue in a daemon, but without the
\r
96 > > accompanying architectural shenanigans.
\r
98 Ack, too. A daemon seems one abstraction layer too much. (But I'm not
\r
99 actively opposed either, if someone has a valid use for such a scheme.)
\r
101 > It would definitely be nice to avoid the complexity inherent in having a
\r
102 > daemon, but how do you imagine "queue on a lock" to work? We don't have
\r
103 > anything like that in place now.
\r
105 I suppose what he means is trying to get the lock, and if that fails wait
\r
106 a bit / wait until it is available again.
\r
108 Actually, as a next step, wouldn't it also be possible to add some
\r
109 heuristic to avoid ``notmuch new'' (being a low-priority task) blocking
\r
110 some interactive user (UI; high-priority task)? But we can pursue such
\r
111 schemes as soon as the basic infrastructure is in place.
\r
113 > Another advantage that can happen with queueing (wherever it occurs) is
\r
114 > to allow a client to be very responsive without waiting for an operation
\r
115 > to complete. Though that can of course be band if the operation isn't
\r
116 > reliably committed.
\r
118 (Obviously this can only work as long as we don't immediatelly need the
\r
119 operation's result; think ``notmuch show''.)
\r
121 So, if the DB has the functionality to internally queue and immediatelly
\r
122 acknowledge transactions, and only later (reliably) commit them, wouldn't
\r
123 that be fine indeed? For example, ``notmuch tag'' then wouldn't have to
\r
124 wait for the DB to be writable. (And note that I have no idea whether
\r
125 Xapian supports such things.) But on the other hand we would like to
\r
126 immediatelly display the requested change in the UI, right?
\r
128 What notmuch-show.el:notmuch-show-remove-tag currently does is *not*
\r
129 re-asking the DB for a message's current tags after having removed a
\r
130 specific one, but instead it interprets the tag removal command itself --
\r
131 which is easy enough in this case, and rather unlikely to ever yield
\r
132 different results, at least unless there's another process operating on
\r
133 the DB concurrently.
\r
135 Otherwise, the other way round, the client could maintain a list of to-do
\r
136 items, to which actions are added if the DB is currently busy, and this
\r
137 list is periodically worked on in order to get it empty. For example,
\r
138 tag changes that are in this list, but not yet committed in the DB could
\r
139 be displayed in another color in the UI. But doing so would shift the
\r
140 responsibility to the UI, which should be in the DB, in my humble
\r
141 opinion. (Actually, this issue feels similar to the one who should be
\r
142 doing the re-trying in case the DB is busy: the UI, or the notmuch
\r
143 process itself, which we're discussing in another thread.)
\r
146 As you can guess I'm not very much into DBs, and neither too much into
\r
147 concurrent systems, so if my ideas don't make sense, please feel free to
\r
148 refer me to literature.
\r
155 Content-Type: application/pgp-signature
\r
157 -----BEGIN PGP SIGNATURE-----
\r
158 Version: GnuPG v1.4.10 (GNU/Linux)
\r
160 iEYEARECAAYFAk1CkC8ACgkQFaWaPJ2HwAoguwCcCn2F8JpKLEehzryPb3iRFhD1
\r
161 2R0AnjOS0Swp28eBpFTyng1VAI6oHX7P
\r
163 -----END PGP SIGNATURE-----
\r
projects / notmuch-archives.git / blob