From: Carl Worth <cworth@cworth.org>
To: notmuch@notmuchmail.org
Date: Thu, 04 Feb 2010 12:50:18 -0800
Message-ID: <87ljf8pvxx.fsf@yoom.home.cworth.org>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha1; protocol="application/pgp-signature"
Subject: [notmuch] A functional (but rudimentary) test suite for notmuch
Precedence: list

--=-=-=

I've just pushed out some improvements to make the little notmuch-test
script I had posted earlier into an actual test suite. I've included the
output of a run of the test suite below so that you can get an idea of
what it does so far.

What I'd like to see going forward is that new features get submitted
along with new test cases for the test suite. And it would be great if
everyone submitting patches could get into the habit of running:

	./test/notmuch-test

before submitting.

The test suite is still extremely rudimentary. Here are some things I'd
like to improve about it:

  * Make the test suite more modular. Right now it's just one long shell
    script---so it's not even easy to see where one test ends and the
    next begins.

  * Allow for executing a subset of the test suite, (eg. notmuch-test reply)

    I just wrote a bunch of "notmuch reply" tests and it was annoying to
    have to wait for all of the "notmuch new" tests to run each time.

  * Actually document what it takes to write a test, (what the test can
    assume about the environment in which its running, what functions
    are available to call, etc.)

I hope others find this useful.

-Carl

Testing "notmuch new" in several variations:
 No new messages...			PASS
 Single new message...			PASS
 Multiple new messages...		PASS
 No new messages (non-empty DB)... 	PASS
 New directories...			PASS
 Alternate inode order...		PASS
 Message moved in...			PASS
 Renamed message...			PASS
 Deleted message...			PASS
 Renamed directory...			PASS
 Deleted directory...			PASS
 New directory (at end of list)... 	PASS
 Deleted directory (end of list)... 	PASS
 New symlink to directory...		PASS
 New symlink to a file...		PASS
 New two-level directory...		PASS
 Deleted two-level directory... 	PASS

Testing "notmuch reply" in several variations:
 Basic reply...				PASS
 Multiple recipients...			PASS
 Reply with CC...			PASS
 Reply from alternate address...	PASS
 Support for Reply-To...		PASS
 Un-munging Reply-To...			PASS
Notmuch test suite complete.

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iD8DBQFLazMK6JDdNq8qSWgRAnrbAJ0ejxcS6VMQe0nYZ5N0V3xBdfEl4gCeOfaD
29pKupbyu860+cW9cSDFhD0=
=NvZi
-----END PGP SIGNATURE-----
--=-=-=--