Date: Mon, 11 Jul 2011 22:58:58 +0100
From: Patrick Totzke <patricktotzke@googlemail.com>
To: Jason Woofenden <jason@jasonwoof.com>
Subject: Re: Slowness (search opens every email file?)
Message-ID: <20110711215858.GA18884@brick.lan>
References: <20110711190721.GA5386@compy.jasonwoof.org>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="bg08WKrSYDhXBjb5"
Content-Disposition: inline
In-Reply-To: <20110711190721.GA5386@compy.jasonwoof.org>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: Notmuch Mail <notmuch@notmuchmail.org>
Precedence: list


--bg08WKrSYDhXBjb5
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi Jason,
On Mon, Jul 11, 2011 at 03:07:21PM -0400, Jason Woofenden wrote:
> notmuch search tag:foo is slow!
>=20
yes, i've just used the vim ui for the first time and i agreee, its sluggis=
h,
searching for * takes a while.

> (when my e-mail files are not already in the disk cache)
>=20
> I saw on my activity monitor applet that it was using mostly i/o,
> and started to wonder if it was opening every e-mail. I little work
> with strace and searching revealed that this command was opening
> many many e-mail files from my maildir(s). I spent a little while
> digging around in the notmuch source, and didn't see where it was
> opening the email files.

I cannot reproduce this. I'm no expert, but at least the output of=20
 strace vim -c ":NotMuch" 2>log
does not contain any path that matches that of my maildir.
Also, I wout be surprised if all individual mails would be read,
because for this, a search for all messages feels too fast.

> 2) Do you agree that it should instead get this information from
> the database?
agreed. And If the mail files get read on every search for you,
something is definately going wrong there.

A quick browse through notmuch.vim tells me that=20
1) it doesn't use notmuchs json output, i think it should, as iirc
this api is considered 'more stable' and is easier to parse than the=20
default output. More importantly,
2) the output of notmuch is copied into a list. This will of course
be slow if your query matches a lot of messages. Could this be done
by asyncronously writing to the buffer somehow?

best,
/p

--bg08WKrSYDhXBjb5
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAk4bciIACgkQlDQDZ9fWxaon3wCgvstp9RjrttNwVaoKWGu+4yvs
YTQAn0byopDOt2qtL68Xmvr1OxC20FpY
=glvO
-----END PGP SIGNATURE-----

--bg08WKrSYDhXBjb5--