From 17c9559c921f52de8a13763858e3a40f45be9993 Mon Sep 17 00:00:00 2001 From: Tomi Ollila Date: Thu, 24 Sep 2015 22:54:08 +0300 Subject: [PATCH] Re: [PATCH] test suite: don't consider skipped individual tests as failing --- fb/5a186e73635a7c24bb6a6f5a857d7350ffd0bf | 87 +++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 fb/5a186e73635a7c24bb6a6f5a857d7350ffd0bf diff --git a/fb/5a186e73635a7c24bb6a6f5a857d7350ffd0bf b/fb/5a186e73635a7c24bb6a6f5a857d7350ffd0bf new file mode 100644 index 000000000..fce27a8f6 --- /dev/null +++ b/fb/5a186e73635a7c24bb6a6f5a857d7350ffd0bf @@ -0,0 +1,87 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by arlo.cworth.org (Postfix) with ESMTP id 86BBF6DE0B7C + for ; Thu, 24 Sep 2015 12:54:08 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at cworth.org +X-Spam-Flag: NO +X-Spam-Score: 1.158 +X-Spam-Level: * +X-Spam-Status: No, score=1.158 tagged_above=-999 required=5 tests=[AWL=-0.238, + SPF_NEUTRAL=0.652, URIBL_SBL=0.644, URIBL_SBL_A=0.1] autolearn=disabled +Received: from arlo.cworth.org ([127.0.0.1]) + by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id dj4nBKrkqUws for ; + Thu, 24 Sep 2015 12:54:06 -0700 (PDT) +Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34]) + by arlo.cworth.org (Postfix) with ESMTP id EFEC26DE0B64 + for ; Thu, 24 Sep 2015 12:54:05 -0700 (PDT) +Received: from guru.guru-group.fi (localhost [IPv6:::1]) + by guru.guru-group.fi (Postfix) with ESMTP id E056910008D; + Thu, 24 Sep 2015 22:54:08 +0300 (EEST) +From: Tomi Ollila +To: David Bremner , notmuch@notmuchmail.org +Subject: Re: [PATCH] test suite: don't consider skipped individual tests as + failing +In-Reply-To: <1443093210-6511-1-git-send-email-david@tethera.net> +References: <1443093210-6511-1-git-send-email-david@tethera.net> +User-Agent: Notmuch/0.20.2+68~g0c35549 (http://notmuchmail.org) Emacs/24.3.1 + (x86_64-unknown-linux-gnu) +X-Face: HhBM'cA~ +MIME-Version: 1.0 +Content-Type: text/plain +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.18 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Thu, 24 Sep 2015 19:54:08 -0000 + +On Thu, Sep 24 2015, David Bremner wrote: + +> It isn't completely clear what we want to do here, but +> +> 1) We currently don't fail if we skip a whole test file (mainly because +> we neglect to count those skipped tests properly). This change at least +> makes the two kinds of skipping consistent. +> +> 2) Automated build environments may have good reasons for building with +> a minimal set of prereqs, and we don't want to discourage running our +> test suite by breaking builds. +> --- +> test/aggregate-results.sh | 2 +- +> 1 file changed, 1 insertion(+), 1 deletion(-) +> +> diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh +> index b016edb..b30da44 100755 +> --- a/test/aggregate-results.sh +> +++ b/test/aggregate-results.sh +> @@ -82,7 +82,7 @@ if [ "$skipped" != "0" ]; then +> echo "$skipped $tests skipped." +> fi +> + +Looks OK, but I think there should be (some good) comment here informing +any potential viewer that we're not checking $skipped... + +> -if [ $success -gt 0 -a $fixed -eq 0 -a $failed -eq 0 -a $skipped -eq 0 ] +> +if [ $success -gt 0 -a $fixed -eq 0 -a $failed -eq 0 ] +> then +> exit 0 +> else +> -- +> 2.5.1 + +Tomi -- 2.26.2