From: Pieter Praet <pieter@praet.org>
To: Mark Anderson <ma.skies@gmail.com>, notmuch@notmuchmail.org
Subject: Re: [PATCH] test:Folder tags shouldn't match after removal of file in
	given folder
In-Reply-To: <87sjqvw64n.fsf@gmail.com>
References: <87sjqvw64n.fsf@gmail.com>
User-Agent: Notmuch/0.5-243-g5a76944 (http://notmuchmail.org) Emacs/23.1.50.1
	(x86_64-pc-linux-gnu)
Date: Tue, 28 Jun 2011 08:43:49 +0200
Message-ID: <87iprqxxp6.fsf@praet.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: list

On Mon, 27 Jun 2011 11:12:24 -0600, Mark Anderson <ma.skies@gmail.com> wrote:
> 
> Test for bug.  Current stemming support for notmuch adds extra terms
> to the DB which aren't removed when the file renames are detected.
> 
> When folder tags are added to a message, Xapian terms for both XFOLDER
> and ZXFOLDER are generated.  When one of the filenames are
> renamed/removed, only the XFOLDER tags are removed, leaving it possible
> for a match on a folder: tag that was previously but is no longer a
> match in the maildir.
> ---
> 
> I found this bug last week.  Essentially when the folder:spam tag is
> added and puts the XFOLDERspam, it also inserts the ZXFOLDERspam.  Then
> if the mail is removed from the folder, it neglects to remove
> ZXFOLDERspam.
> 
> This was detected with my offlineimap usage with gmail, still haven't
> polished my personal folder/label transition.  As I was looking into it,
> I saw some unusual things flagged as spam, and investigated.

+1

This bug was reported by Sebastian back in January [1], and I
submitted a test [2], but yours is much more thorough, both
regarding the actual test as well as the commit message.

I'd forgotten all about this issue myself, so it's very nice
to see it brought up again.

Thanks!

> 
>  test/notmuch-test            |    1 +
>  test/search-folder-coherence |   48 ++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 49 insertions(+), 0 deletions(-)
>  create mode 100755 test/search-folder-coherence
> 
> diff --git a/test/notmuch-test b/test/notmuch-test
> index fe85c6a..79e6267 100755
> --- a/test/notmuch-test
> +++ b/test/notmuch-test
> @@ -41,6 +41,7 @@ TESTS="
>    maildir-sync
>    crypto
>    symbol-hiding
> +  search-folder-coherence
>  "
>  TESTS=${NOTMUCH_TESTS:=$TESTS}
>  
> diff --git a/test/search-folder-coherence b/test/search-folder-coherence
> new file mode 100755
> index 0000000..cf3ba40
> --- /dev/null
> +++ b/test/search-folder-coherence
> @@ -0,0 +1,48 @@
> +#!/usr/bin/env bash
> +test_description='folder tags removed and added through file renames remain consistent'
> +. ./test-lib.sh
> +
> +test_begin_subtest "No new messages"
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail."
> +
> +
> +test_begin_subtest "Single new message"
> +generate_message
> +file_x=$gen_msg_filename
> +id_x=$gen_msg_id
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "Added 1 new message to the database."
> +
> +test_begin_subtest "Add second folder for same message"
> +dir=$(dirname $file_x)
> +mkdir $dir/spam
> +cp $file_x $dir/spam
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail."
> +
> +
> +test_begin_subtest "Multiple files for same message"
> +cat <<EOF >EXPECTED
> +MAIL_DIR/msg-001
> +MAIL_DIR/spam/msg-001
> +EOF
> +notmuch search --output=files id:$id_x | sed -e "s,$MAIL_DIR,MAIL_DIR," >OUTPUT
> +test_expect_equal_file OUTPUT EXPECTED
> +
> +test_begin_subtest "Test matches folder:spam"
> +output=$(notmuch search folder:spam)
> +test_expect_equal "$output" "thread:0000000000000001   2001-01-05 [1/1] Notmuch Test Suite; Test message #1 (inbox unread)"
> +
> +sleep 1;
> +
> +test_begin_subtest "Remove folder:spam copy of email"
> +rm $dir/spam/$(basename $file_x)
> +output=$(NOTMUCH_NEW)
> +test_expect_equal "$output" "No new mail. Detected 1 file rename."
> +
> +test_begin_subtest "No mails match the folder:spam search"
> +output=$(notmuch search folder:spam)
> +test_expect_equal "$output" ""
> +
> +test_done
> -- 
> 1.7.4.1
> 
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch


Peace

-- 
Pieter

[1] id:"87vd1n4rd2.fsf@SSpaeth.de"
[2] id:"1305270308-30660-1-git-send-email-pieter@praet.org"