DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:sender:in-reply-to:references:date
	:x-google-sender-auth:message-id:subject:from:to:cc:content-type;
	b=cWbR38hGEJRQht2MSUVqidMBfm65IszE/5UWX5qYV8Fqk2y9UYjRSFNQsh97QIq9Yh
	bNuCemK6QbvoZDqjYtr/N95UYed71bQ+yFHdwsBz1H/o6o/29WBg/ioZUSeBZJA7xqju
	JNMLAfomJ+kN2tTmf40g9RGTZch2g4yENRokU=
MIME-Version: 1.0
Sender: amdragon@gmail.com
In-Reply-To: <1296228975-ner-3626@everglades>
References: <87fwsetdin.fsf@kepler.schwinge.homeip.net>
	<8762taxk9y.fsf@algae.riseup.net>
	<87vd1a84qj.fsf@servo.finestructure.net>
	<AANLkTi=pPgcPXUN_XTXa0gAkbTR_OF4XxCP-d4-Be2pt@mail.gmail.com>
	<87fwsdobpy.fsf@yoom.home.cworth.org>
	<877hdps6og.fsf@kepler.schwinge.homeip.net>
	<1296228975-ner-3626@everglades>
Date: Fri, 28 Jan 2011 11:57:34 -0500
Message-ID: <AANLkTim9FEEn6bpeAb1oHVScs4RGbK-UrFY+52ZKWv4E@mail.gmail.com>
Subject: Re: notmuch's idea of concurrency / failing an invocation
From: Austin Clements <amdragon@mit.edu>
To: Mike Kelly <pioto@pioto.org>
Content-Type: multipart/alternative; boundary=00163646b8ee726b67049aeaf657
Cc: notmuch@notmuchmail.org
Precedence: list

--00163646b8ee726b67049aeaf657
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Jan 28, 2011 at 10:36 AM, Mike Kelly <pioto@pioto.org> wrote:

> On Fri, 28 Jan 2011 10:45:19 +0100, Thomas Schwinge <thomas@schwinge.name>
> wrote:
> > > It would definitely be nice to avoid the complexity inherent in having
> a
> > > daemon, but how do you imagine "queue on a lock" to work? We don't have
> > > anything like that in place now.
> >
> > I suppose what he means is trying to get the lock, and if that fails wait
> > a bit / wait until it is available again.
> >
> > Actually, as a next step, wouldn't it also be possible to add some
> > heuristic to avoid ``notmuch new'' (being a low-priority task) blocking
> > some interactive user (UI; high-priority task)?  But we can pursue such
> > schemes as soon as the basic infrastructure is in place.
>
> Couldn't we pretty much get the desired behavior by using flock(2)?
> Basically, take out a LOCK_EX when we need to write, and a LOCK_SH when
> we only need to read. Using the blocking form, things should pretty much
> just queue up and take their turn, right?
>
> I'm not familiar with Xapian, but if it doesn't give us something we
> could use this sort of locking on, couldn't we just add some
> /path/to/mail/.notmuch.lock file that we open to hold a lock on?
>

Yes, exactly.  All of this.  Unfortunately, Xapian doesn't expose the
ability to block on the lock (see the fcntl call in backends/flint_lock.cc,
which is hard-coded to the non-blocking F_SETLK instead of F_SETLKW), so
we'd either need a new Xapian option, or we would just have to wrap our own
flock/fcntl lock around things as you suggest.

--00163646b8ee726b67049aeaf657
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div class=3D"gmail_quote">On Fri, Jan 28, 2011 at 10:36 AM, Mike Kelly <sp=
an dir=3D"ltr">&lt;<a href=3D"mailto:pioto@pioto.org">pioto@pioto.org</a>&g=
t;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class=3D"im">On Fri, 28 Jan 2011 10:45:19 +0100, Thomas Schwinge &lt;<=
a href=3D"mailto:thomas@schwinge.name">thomas@schwinge.name</a>&gt; wrote:<=
br>
&gt; &gt; It would definitely be nice to avoid the complexity inherent in h=
aving a<br>
&gt; &gt; daemon, but how do you imagine &quot;queue on a lock&quot; to wor=
k? We don&#39;t have<br>
&gt; &gt; anything like that in place now.<br>
&gt;<br>
&gt; I suppose what he means is trying to get the lock, and if that fails w=
ait<br>
&gt; a bit / wait until it is available again.<br>
&gt;<br>
&gt; Actually, as a next step, wouldn&#39;t it also be possible to add some=
<br>
&gt; heuristic to avoid ``notmuch new&#39;&#39; (being a low-priority task)=
 blocking<br>
&gt; some interactive user (UI; high-priority task)? =A0But we can pursue s=
uch<br>
&gt; schemes as soon as the basic infrastructure is in place.<br>
<br>
</div>Couldn&#39;t we pretty much get the desired behavior by using flock(2=
)?<br>
Basically, take out a LOCK_EX when we need to write, and a LOCK_SH when<br>
we only need to read. Using the blocking form, things should pretty much<br=
>
just queue up and take their turn, right?<br>
<br>
I&#39;m not familiar with Xapian, but if it doesn&#39;t give us something w=
e<br>
could use this sort of locking on, couldn&#39;t we just add some<br>
/path/to/mail/.notmuch.lock file that we open to hold a lock on?<br></block=
quote><div><br></div><div>Yes, exactly. =A0All of this. =A0Unfortunately, X=
apian doesn&#39;t expose the ability to block on the lock (see the fcntl ca=
ll in backends/flint_lock.cc, which is hard-coded to the non-blocking F_SET=
LK instead of F_SETLKW), so we&#39;d either need a new Xapian option, or we=
 would just have to wrap our own flock/fcntl lock around things as you sugg=
est.</div>
<div><br></div></div>

--00163646b8ee726b67049aeaf657--