Return-Path: <cworth@cworth.org>
X-Original-To: notmuch@notmuchmail.org
Delivered-To: notmuch@notmuchmail.org
Received: from localhost (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id A412D4196F2
	for <notmuch@notmuchmail.org>; Thu, 22 Apr 2010 14:14:39 -0700 (PDT)
X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
X-Spam-Flag: NO
X-Spam-Score: -2.89
X-Spam-Level: 
X-Spam-Status: No, score=-2.89 tagged_above=-999 required=5
	tests=[ALL_TRUSTED=-1, BAYES_00=-1.9, T_MIME_NO_TEXT=0.01]
	autolearn=ham
Received: from olra.theworths.org ([127.0.0.1])
	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id 8tkK1Lxf0kp4; Thu, 22 Apr 2010 14:14:38 -0700 (PDT)
Received: from yoom.home.cworth.org (localhost [127.0.0.1])
	by olra.theworths.org (Postfix) with ESMTP id 2837B431FC1;
	Thu, 22 Apr 2010 14:14:38 -0700 (PDT)
Received: by yoom.home.cworth.org (Postfix, from userid 1000)
	id A9ECB568DE4; Thu, 22 Apr 2010 14:14:37 -0700 (PDT)
From: Carl Worth <cworth@cworth.org>
To: Michal Sojka <sojkam1@fel.cvut.cz>, Gregor Hoffleit <gregor@hoffleit.de>,
	notmuch <notmuch@notmuchmail.org>
Subject: Improved diff-based failure reports from the test suite
In-Reply-To: <87vdbrerqs.fsf@steelpick.2x.cz>
References: <1267697893-sup-4538@sam.mediasupervision.de>
	<87r5mje55l.fsf@yoom.home.cworth.org>
	<1271176622-sup-4680@sam.mediasupervision.de>
	<87iq7tft7j.fsf@yoom.home.cworth.org>
	<87ochlt8r9.fsf@steelpick.2x.cz>
	<878w8ofpx9.fsf@yoom.home.cworth.org>
	<87vdbrerqs.fsf@steelpick.2x.cz>
Date: Thu, 22 Apr 2010 14:14:31 -0700
Message-ID: <87k4rzqjew.fsf@yoom.home.cworth.org>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
	micalg=pgp-sha1; protocol="application/pgp-signature"
X-BeenThere: notmuch@notmuchmail.org
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: "Use and development of the notmuch mail system."
	<notmuch.notmuchmail.org>
List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
List-Archive: <http://notmuchmail.org/pipermail/notmuch>
List-Post: <mailto:notmuch@notmuchmail.org>
List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
X-List-Received-Date: Thu, 22 Apr 2010 21:14:39 -0000

--=-=-=
Content-Transfer-Encoding: quoted-printable

On Fri, 16 Apr 2010 10:17:15 +0200, Michal Sojka <sojkam1@fel.cvut.cz> wrot=
e:
> It's definitely better than before. The current implementation of
> pass_if_equal has IMHO one drawback - if it compares multiline text and
> there is a difference, it is quite hard to see where.
>=20
> In my tests for maildir synchronization I use this approach:
>=20
>   notmuch search tag:inbox | filter_output > actual &&
>   diff -u - actual <<EOF
>   thread:XXX   2000-01-01 [1/1] Notmuch Test Suite; test message 3 (inbox)
>   EOF
>=20
> Thanks to the usee of diff, I immediately see only the differences.

I just added a very long multi-line test to the test suite.

So I took the opportunity to fix it to output the failures with diff. It
also saves both the complete expected and actual output (for any failed
tests) to files of the form test-XXX.expected and test-XXX.output.

Enjoy!

=2DCarl

--=-=-=
Content-Type: application/pgp-signature

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iD8DBQFL0Lw36JDdNq8qSWgRAmxyAJ9mJSYoUghzbQat3X2gLqv2oEdSPQCdGwMy
RYjDTxkknMu+MUcl73/EiNc=
=7zPR
-----END PGP SIGNATURE-----
--=-=-=--