Austin Clements [Fri, 24 Oct 2014 12:49:15 +0000 (08:49 -0400)]
lib: Introduce macros for bit operations
These macros help clarify basic bit-twiddling code and are written to
be robust against C undefined behavior of shift operators.
Austin Clements [Thu, 23 Oct 2014 12:30:34 +0000 (08:30 -0400)]
lib: Update database schema doc for ghost messages
This describes the structure of ghost mail documents. Ghost messages
are not yet implemented.
Austin Clements [Thu, 23 Oct 2014 12:30:33 +0000 (08:30 -0400)]
lib: Add a ghost messages database feature
This will be implemented over the next several patches. The feature
is not yet "enabled" (this does not add it to
NOTMUCH_FEATURES_CURRENT).
David Bremner [Sat, 25 Oct 2014 16:55:25 +0000 (18:55 +0200)]
Merge tag '0.18.2'
notmuch 0.18.2 release
David Bremner [Sat, 25 Oct 2014 16:23:33 +0000 (18:23 +0200)]
debian: add changelog stanza for 0.18.2-1
David Bremner [Sat, 25 Oct 2014 16:23:06 +0000 (18:23 +0200)]
version: bump to 0.18.2
No non-documentation changes
David Bremner [Sat, 25 Oct 2014 10:10:05 +0000 (12:10 +0200)]
NEWS: give real entry for 0.18.2
Expand out what the actual test suite changes are, so people have a
better idea if the release is relevant to them.
David Bremner [Sat, 25 Oct 2014 09:38:18 +0000 (11:38 +0200)]
Merge tag '0.18.2_rc1'
notmuch 0.18.2~rc1 release
Conflicts:
NEWS
debian/changelog
David Bremner [Sat, 25 Oct 2014 09:20:57 +0000 (11:20 +0200)]
debian: build depend on dh-python
The build log asked me nicely, so I did.
David Bremner [Sat, 25 Oct 2014 08:48:31 +0000 (10:48 +0200)]
debian: add changelog stanza for 0.18.2~rc-1
David Bremner [Sat, 25 Oct 2014 08:42:00 +0000 (10:42 +0200)]
NEWS: add minimal news item for 0.18.2
The less said, the less typos to make.
David Bremner [Sat, 25 Oct 2014 08:38:11 +0000 (10:38 +0200)]
version: bump to 0.18.2~rc1
David Bremner [Sat, 25 Oct 2014 07:57:12 +0000 (09:57 +0200)]
test: kill '"filename": "signature.asc"' from json output
This starts to appear with emacs24.4, so we can't easily have it in
our expected output.
David Bremner [Sat, 25 Oct 2014 06:53:29 +0000 (08:53 +0200)]
test/emacs: force *Messages* buffer to be writable
In emacs 24.4 the messages buffer starts being read-only, which kills
these tests. This seems to be the point of the variable
inihibit-read-only, which has existed at least since emacs 21.
David Bremner [Wed, 10 Sep 2014 06:41:11 +0000 (08:41 +0200)]
test: simplify T360-symbol-hiding, use nm instead of objdump
After yet another variation in objdump output caused this test to fail
(on a Debian port, no less), I decided whatever putative benefit we
get from looking at the object files instead of the library isn't
worth the maintenence headache.
This version uses nm -P. nm -P should be portable, and fixed format.
It purposely doesn't use the -D argument, since that is non-POSIX and
nm on GNU/Linux seems do the right thing without it.
It still won't work out of the box on e.g. Mac OS/X. I think the right
thing to do there is to move some more configuration information into
sh.config.
(cherry picked from commit
c34d6bad0f9da300eac2181e2073aee130432932)
Austin Clements [Fri, 3 Oct 2014 16:58:03 +0000 (12:58 -0400)]
test: Port atomicity test to Python
Previously, this was implemented using a horrible GDB script (because
there is no such thing as a non-horrible GDB script). This GDB script
often broke with newer versions of GDB for mysterious reasons. Port
the test script to GDB's Python API, which makes the code much cleaner
and, hopefully, more stable.
(cherry picked from commit
cbbda62258360f035894cff9dfd66c60b0cc707f)
Conflicts:
test/T380-atomicity.sh
David Bremner [Fri, 24 Oct 2014 12:23:22 +0000 (14:23 +0200)]
README.rst: typo/wording fixes
It occurs that hardcoding the URL will not make sense in forks.
Mind you, the whole README won't make much sense in forks.
David Bremner [Fri, 24 Oct 2014 08:25:31 +0000 (10:25 +0200)]
doc: add README.rst especially for github
Apparently README.rst overrides README, so this will show up instead
of our generic README on github.
If the user is already on github, then clicking a link for more
information is not a hardship.
Ian Main [Wed, 1 Oct 2014 17:11:28 +0000 (10:11 -0700)]
VIM: Make starting in 'insert' mode for compose optional
This adds a variable to make starting in insert mode optional when
composing and replying to emails. I found it unusual to be started in
insert mode so I thought I'd make it optional as others may find this as
well.
Ian
Ian Main [Thu, 2 Oct 2014 23:47:15 +0000 (16:47 -0700)]
VIM: Use notmuch CLI for config
This patch switches from reading .notmuch-config directly to using
the CLI the same way that emacs does it. It actually uses less code
and is probably less error prone.
Ian
David Bremner [Sat, 18 Oct 2014 06:13:42 +0000 (08:13 +0200)]
test: check for gdb in insert tests
Commits
a6cee01 and
c48b12f otherwise break the build for people
without gdb.
David Bremner [Fri, 3 Oct 2014 21:19:00 +0000 (23:19 +0200)]
test/insert: check that indexing errors are accepted with --keep
This is overkill for the current code path, but should provide some
robustness for future changes in error handling.
Jani Nikula [Fri, 3 Oct 2014 21:18:59 +0000 (23:18 +0200)]
cli/insert: require succesful message indexing for success status
Add --keep option to keep any remaining stuff in index or file. We
could distinguish between failures to index and failures to apply tags
or maildir sync, but for simplicity just have one.
Jani Nikula [Fri, 3 Oct 2014 21:18:58 +0000 (23:18 +0200)]
cli/insert: add fail path to add_file_to_database
Handle failures gracefully in add_file_to_database, renamed simply
add_file while at it. Add keep option to not remove the message from
database if tagging or tag syncing to maildir flags fails. Expand the
function documentation to cover the changes.
David Bremner [Fri, 3 Oct 2014 21:18:57 +0000 (23:18 +0200)]
test/insert: add known broken tests for indexing failures
These tests are written with the assumption that we want all indexing
failures to be considered as failures by notmuch insert.
Austin Clements [Mon, 6 Oct 2014 23:17:09 +0000 (17:17 -0600)]
lib: Handle empty date value
In the interest of robustness, avoid undefined behavior of
sortable_unserialise if the date value is missing. This shouldn't
happen now, but ghost messages will have blank date values.
Austin Clements [Mon, 6 Oct 2014 23:17:08 +0000 (17:17 -0600)]
lib: Refactor _notmuch_database_link_message
This moves the code to retrieve and clear the metadata thread ID out
of _notmuch_database_link_message into its own function. This will
simplify future changes.
Austin Clements [Mon, 6 Oct 2014 23:17:07 +0000 (17:17 -0600)]
lib: Move message ID compression to _notmuch_message_create_for_message_id
Previously, this was performed by notmuch_database_add_message. This
happens to be the only caller currently (which is why this was safe),
but we're about to introduce more callers, and it makes more sense to
put responsibility for ID compression in the lower-level function
rather than requiring each caller to handle it.
David Bremner [Sun, 5 Oct 2014 05:53:28 +0000 (07:53 +0200)]
NEWS: mention the change in default build flags
It blows things up by a factor of six or so, so it's worth giving
people a heads up. It won't effect e.g. Debian, that already builds
with -g and then strips.
W. Trevor King [Tue, 7 Oct 2014 17:36:13 +0000 (10:36 -0700)]
NEWS: Document "nmbug: Translate to Python"
For more details, see the commit message for
7f2cb3be (nmbug:
Translate to Python, 2014-10-03). I realized while writing this that
the
7f2cb3be commit message has:
* 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.
But we dropped that exec in favor of the subprocess approach between
v3 and v4, I just forgot to update the commit message [1].
[1]: id:
e630b6763e9d0771718afee41ea15b29bb4a1de8.
1409935538.git.wking@tremily.us
http://article.gmane.org/gmane.mail.notmuch.general/19007
Jani Nikula [Sat, 4 Oct 2014 17:53:09 +0000 (20:53 +0300)]
hex-escape: remove unused variable default_buf_size
Found by clang:
CC -O2 util/hex-escape.o
util/hex-escape.c:28:21: warning: unused variable 'default_buf_size'
[-Wunused-const-variable]
static const size_t default_buf_size = 1024;
^
1 warning generated.
Austin Clements [Fri, 3 Oct 2014 16:58:03 +0000 (12:58 -0400)]
test: Port atomicity test to Python
Previously, this was implemented using a horrible GDB script (because
there is no such thing as a non-horrible GDB script). This GDB script
often broke with newer versions of GDB for mysterious reasons. Port
the test script to GDB's Python API, which makes the code much cleaner
and, hopefully, more stable.
David Bremner [Fri, 3 Oct 2014 20:22:10 +0000 (22:22 +0200)]
configure: add debug flags by default.
This makes development (in particular the test suite) easier. Those
concerned about the extra diskspace can override the default or use
strip.
David Bremner [Fri, 3 Oct 2014 20:22:09 +0000 (22:22 +0200)]
test: check for debug symbols in notmuch
In the future, tests may rely on debug symbols being present in
notmuch, so we plan to switch the default flags.
The main purpose of this test is to help explain the perhaps
mysterious failures of other tests which rely on symbols being
present.
W. Trevor King [Fri, 3 Oct 2014 18:20:57 +0000 (11:20 -0700)]
nmbug: Translate to Python
This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages. It should be compatible with Python 2.7 and later
(including the 3.x series).
Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:
* 'nmbug help' has been split between the general 'nmbug --help' and
the command-specific 'nmbug COMMAND --help'.
* Commands are no longer split into "most common", "other useful", and
"less common" sets. If we need something like this, I'd prefer
workflow examples highlighting common commands in the module
docstring (available with 'nmbug --help').
* 'nmbug commit' now only uses a single argument for the optional
commit-message text. I wanted to expose more of the underlying 'git
commit' UI, since I personally like to write my commit messages in
an editor with the notes added by 'git commit -v' to jog my memory.
Unfortunately, we're using 'git commit-tree' instead of 'git
commit', and commit-tree is too low-level for editor-launching. I'd
be interested in rewriting commit() to use 'git commit', but that
seemed like it was outside the scope of this rewrite. So I'm not
supporting all of Git's commit syntax in this patch, but I can at
least match 'git commit -m MESSAGE' in requiring command-line commit
messages to be a single argument.
* The default repository for 'nmbug push' and 'nmbug fetch' is now the
current branch's upstream (branch.<name>.remote) instead of
'origin'. When we have to, we extract this remote by hand, but
where possible we just call the Git command without a repository
argument, and leave it to Git to figure out the default.
* 'nmbug push' accepts multiple refspecs if you want to explicitly
specify what to push. Otherwise, the refspec(s) pushed depend on
push.default. The Perl version hardcoded 'master' as the pushed
refspec.
* 'nmbug pull' defaults to the current branch's upstream
(branch.<name>.remote and branch.<name>.merge) instead of hardcoding
'origin' and 'master'. It also supports multiple refspecs if for
some crazy reason you need an octopus merge (but mostly to avoid
breaking consistency with 'git pull').
* 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.
* 'nmbug status' now catches stderr, and doesn't print errors like:
No upstream configured for branch 'master'
The Perl implementation had just learned to avoid crashing on that
case, but wasn't yet catching the dying subprocess's stderr.
* 'nmbug archive' now accepts positional arguments for the tree-ish
and additional 'git archive' options. For example, you can run:
$ nmbug archive HEAD -- --format tar.gz
I wish I could have preserved the argument order from 'git archive'
(with the tree-ish at the end), but I'm not sure how to make
argparse accept arbitrary possitional arguments (some of which take
arguments). Flipping the order to put the tree-ish first seemed
easiest.
* 'nmbug merge' and 'pull' no longer checkout HEAD before running
their command, because blindly clobbering the index seems overly
risky.
* In order to avoid creating a dirty index, 'nmbug commit' now uses
the default index (instead of nmbug.index) for composing the commit.
That way the index matches the committed tree. To avoid leaving a
broken index after a failed commit, I've wrapped the whole thing in
a try/except block that resets the index to match the pre-commit
treeish on errors. That means that 'nmbug commit' will ignore
anything you've cached in the index via direct Git calls, and you'll
either end up with an index matching your notmuch tags and the new
HEAD (after a successful commit) or an index matching the original
HEAD (after a failed commit).
David Bremner [Sat, 4 Oct 2014 18:47:00 +0000 (20:47 +0200)]
doc: build notmuch-emacs info/html docs, link from index
Although this manual is far from complete, it may be helpful for
someone. In particular building it as part of the standard build
process makes it easier to find problems when editing the
notmuch-emacs-manual.
Austin Clements [Fri, 3 Oct 2014 14:27:26 +0000 (10:27 -0400)]
doc: Emacs manual improvements and expansions
Fix several typos, improve general wording and flow, and add some
information on notmuch-jump.
Austin Clements [Fri, 3 Oct 2014 14:22:59 +0000 (10:22 -0400)]
NEWS: News for notmuch-jump
Austin Clements [Thu, 2 Oct 2014 19:19:08 +0000 (15:19 -0400)]
lib: Simplify close and codify aborting atomic section
In Xapian, closing a database implicitly aborts any outstanding
transaction and commits changes. For historical reasons,
notmuch_database_close had grown to almost, but not quite duplicate
this behavior. Before closing the database, it would explicitly (and
unnecessarily) commit it. However, if there was an outstanding
transaction (ie atomic section), commit would throw a Xapian
exception, which notmuch_database_close would unnecessarily print to
stderr, even though notmuch_database_close would ultimately abort the
transaction anyway when it called close.
This patch simplifies notmuch_database_close to explicitly abort any
outstanding transaction and then just call Database::close. This
works for both read-only and read/write databases, takes care of
committing changes, unifies the exception handling path, and codifies
aborting outstanding transactions. This is currently the only way to
abort an atomic section (and may remain so, since it would be
difficult to roll back things we may have cached from rolled-back
modifications).
Jani Nikula [Mon, 22 Sep 2014 09:54:59 +0000 (11:54 +0200)]
cli/insert: rehash file writing functions
Make the function calls make more sense as independent building blocks
of the big picture, with clear inputs and outputs. Split up
write_message into two. Improve function documentation. Cleanup and
clarify the error paths.
Jani Nikula [Mon, 22 Sep 2014 09:54:58 +0000 (11:54 +0200)]
cli/insert: abstract temporary filename generation
This will clean up the usage. There's the slight functional change of
potentially ending up doing extra gethostname and getpid calls, but
this is neglible.
Jani Nikula [Mon, 22 Sep 2014 09:54:57 +0000 (11:54 +0200)]
cli/insert: use a single recursive mkdir function
Combine make_directory() and make_directory_and_parents() into a
single recursive mkdir_recursive() function. Clarify the code and
improve error handling. Improve error messages. Switch to using the
new function in maildir_create_folder(). Constify talloc context.
Jani Nikula [Mon, 22 Sep 2014 09:54:56 +0000 (11:54 +0200)]
cli/insert: clean up sync_dir
Clarify the code slightly, improve error messages. Apart from the
error message changes, no functional changes.
Jani Nikula [Mon, 22 Sep 2014 09:54:55 +0000 (11:54 +0200)]
cli/insert: rename file copy function
The copying has nothing to do with stdin, so call it copy_fd
instead. While at it, improve documentation and reverse the
parameters, as destination is traditionally the first parameter.
Jani Nikula [Mon, 22 Sep 2014 09:54:54 +0000 (11:54 +0200)]
cli/insert: move add_file_to_database to a better place
Move add_file_to_database around to keep the filesystem related
functions grouped together, improving readability. No functional
changes.
Jani Nikula [Mon, 22 Sep 2014 09:54:53 +0000 (11:54 +0200)]
cli/insert: rename check_folder_name to is_valid_folder_name
An "is something" predicate conveys the meaning better. While at it,
improve the function documentation and error message. Besides the
error message change, no functional changes.
Jani Nikula [Mon, 22 Sep 2014 09:54:52 +0000 (11:54 +0200)]
lib: actually return failures from notmuch_message_tags_to_maildir_flags
The function takes great care to preserve the first error status it
encounters, yet fails to return that status to the caller. Fix it.
Mark Walters [Thu, 4 Sep 2014 09:46:54 +0000 (10:46 +0100)]
emacs: jump: fix compile warning on emacs 23
notmuch-jump uses window-body-width which is not defined in emacs
23. To get around this it does
(unless (fboundp 'window-body-width)
;; Compatibility for Emacs pre-24
(defalias 'window-body-width 'window-width))
This makes sure window-body-width is defined and all should be
well. But it seems that the byte compiler does not realise that this
guarantees that window-body-width will be defined and so, when
compiling with emacs 23, it gives an error
In end of data:
notmuch-jump.el:172:1:Warning: the function `window-body-width' is not known to be defined.
Domo and I came to following on irc: wrap the (unless (fboundp ...))
inside eval-and-compile which ensures that both the test and the
defalias (if needed) happen at both compile and load time. This fixes
the warning.
Thibaut Horel [Mon, 11 Aug 2014 04:43:54 +0000 (00:43 -0400)]
python: Add binding for notmuch_query_add_tag_exclude
Implemented as the method `exclude_tag` of the class `Query`. This method takes
one argument, a string containing the name of the tag to exclude.
Austin Clements [Mon, 21 Apr 2014 18:37:40 +0000 (14:37 -0400)]
emacs: Fix coding system in `notmuch-show-view-raw-message'
This fixes the known-broken test of viewing 8bit messages added by the
previous commit.
Austin Clements [Mon, 21 Apr 2014 18:37:39 +0000 (14:37 -0400)]
test: New tests for Emacs charset handling
The test of viewing 8bit messages is known-broken. The rest pass, but
for very fragile reasons. The next several commits will fix the
known-broken test and make our charset handling robust.
Austin Clements [Mon, 21 Apr 2014 18:37:38 +0000 (14:37 -0400)]
emacs: Remove redundant NTH argument from `notmuch-get-bodypart-content'.
This can be derived from the PART argument (which is arguably
canonical), so there's no sense in giving the caller an extra foot
gun.
David Bremner [Sat, 20 Sep 2014 05:41:35 +0000 (07:41 +0200)]
test/emacs: globally force the html renderer to html2text
Previously we did this for a single test, but some other proposed
tests ( id:
1398105468-14317-3-git-send-email-amdragon@mit.edu ) show
similar breakage when switching renderers.
David Bremner [Sat, 20 Sep 2014 07:51:48 +0000 (09:51 +0200)]
debian: re-enable atomicity test on arm64
according to
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=757454
the kernel problem which caused this test to fail should be
fixed now
David Bremner [Tue, 16 Sep 2014 19:02:50 +0000 (21:02 +0200)]
debian: bump SONAME
Note that this is one of those cases where an ABI change is not
obvious from the symbols file. Several previously void functions now
have return codes.
David Bremner [Tue, 16 Sep 2014 18:50:57 +0000 (20:50 +0200)]
python: bump SONAME
This should have happened in commit
6754ad9f9, but oops.
This was not caught by our test suite because it uses an installed
notmuch library of it cannot find the just built one.
Peter Wang [Wed, 16 Apr 2014 12:59:21 +0000 (22:59 +1000)]
cli: refactor insert
Change insert_message into write_message and move its responsibilities
for indexing the message into the main function, to simplify the control
flow.
Peter Wang [Wed, 16 Apr 2014 12:59:20 +0000 (22:59 +1000)]
ruby: handle return status of database close
Throw an exception if notmuch_database_destroy fails.
Peter Wang [Wed, 16 Apr 2014 12:59:19 +0000 (22:59 +1000)]
go: add return status to database close method
Add return status to the Database.Close() method that calls
notmuch_database_destroy.
Peter Wang [Wed, 16 Apr 2014 12:59:18 +0000 (22:59 +1000)]
python: handle return status of database close and destroy
Throw an exception if notmuch_database_close or notmuch_database_destroy
fail.
Peter Wang [Wed, 16 Apr 2014 12:59:17 +0000 (22:59 +1000)]
lib: bump soname
Adding return values to notmuch_database_close and
notmuch_database_destroy may require bumping the soname.
David Bremner [Wed, 10 Sep 2014 06:41:11 +0000 (08:41 +0200)]
test: simplify T360-symbol-hiding, use nm instead of objdump
After yet another variation in objdump output caused this test to fail
(on a Debian port, no less), I decided whatever putative benefit we
get from looking at the object files instead of the library isn't
worth the maintenence headache.
This version uses nm -P. nm -P should be portable, and fixed format.
It purposely doesn't use the -D argument, since that is non-POSIX and
nm on GNU/Linux seems do the right thing without it.
It still won't work out of the box on e.g. Mac OS/X. I think the right
thing to do there is to move some more configuration information into
sh.config.
Gaute Hope [Wed, 10 Sep 2014 07:24:18 +0000 (09:24 +0200)]
notmuch_thread_get_authors: document match grouping with |
as stated in thread.cc:115
/* Construct an authors string from matched_authors_array and
* authors_array. The string contains matched authors first, then
* non-matched authors (with the two groups separated by '|'). Within
* each group, authors are listed in date order. */
this is, however, not reflected in the public API documentation in
notmuch.h:970. This patch a paragraph explaining how | separates the
group of authors of messages matching the query and those of messages
that do not, but are still contained in the thread.
Mark Walters [Tue, 2 Sep 2014 18:56:04 +0000 (19:56 +0100)]
emacs: jump: sort-order bugfix
default-value needs its argument to be quoted.
Slightly strangely default-value of 't or nil is 't or nil
respectively so the code
(default-value notmuch-search-oldest-first)
just gives the current value of notmuch-search-oldest-first rather
than intended default-value of this variable.
The symptom is that if you are in a search buffer and use notmuch jump
to run a saved search which does not have an explicitly set sort order
then the sort order of the saved-search is inherited from the current
search buffer rather than being the default search order.
Thanks to Jani for finding the bug.
Austin Clements [Wed, 3 Sep 2014 15:28:34 +0000 (11:28 -0400)]
cli: Be more helpful when .notmuch-config does not exist
Previously, if the user ran any subcommand that required a
configuration (e.g., notmuch new) but didn't have a configuration,
notmuch would give the rather un-friendly and un-actionable message
Error reading configuration file .notmuch-config: No such file or directory
Since this condition is expected for new users, this patch adds
specific handling for the file-not-found case to give a message that
is friendly and actionable.
Tomi Ollila [Mon, 1 Sep 2014 18:44:59 +0000 (21:44 +0300)]
doc: 'rm -f' potential doxygen temporary output file
Some (older) Doxygen versions do not create such a temporary file.
Austin Clements [Mon, 1 Sep 2014 22:49:07 +0000 (18:49 -0400)]
lib: Fix endless upgrade problem
48db8c8 introduced a disagreement between when
notmuch_database_needs_upgrade returned TRUE and when
notmuch_database_upgrade actually performed an upgrade. As a result,
if a database had a version less than 3, but no new features were
required, notmuch new would call notmuch_database_upgrade to perform
an upgrade, but notmuch_database_upgrade would return immediately
without updating the database version. Hence, the next notmuch new
would do the same, and so on.
Fix this by ensuring that the upgrade-required logic is identical
between the two.
Austin Clements [Mon, 25 Aug 2014 17:26:09 +0000 (13:26 -0400)]
lib: Update doc of notmuch_database_{needs_upgrade,upgrade}
Clients are no longer required to call these functions after opening a
database in read/write mode (which is good, because almost none of
them do!).
Austin Clements [Mon, 25 Aug 2014 17:26:08 +0000 (13:26 -0400)]
lib: Return an error from operations that require an upgrade
Previously, there was no protection against a caller invoking an
operation on an old database version that would effectively corrupt
the database by treating it like a newer version.
According to notmuch.h, any caller that opens the database in
read/write mode is supposed to check if the database needs upgrading
and perform an upgrade if it does. This would protect against this,
but nobody (even the CLI) actually does this.
However, with features, it's easy to protect against incompatible
operations on a fine-grained basis. This lightweight change allows
callers to safely operate on old database versions, while preventing
specific operations that would corrupt the database with an
informative error message.
Austin Clements [Mon, 25 Aug 2014 17:26:07 +0000 (13:26 -0400)]
lib: Support empty header values in database
Commit
567bcbc2 introduced support for storing various headers in
document values. However, doing so in a backwards-compatible way
meant that genuinely empty header values could not be distinguished
from the old behavior of not storing the headers at all, so these
required parsing the original message.
Now that we have database features, new databases can declare that all
messages have header values, so if we have this feature flag, we can
use the stored header value even if it's the empty string.
This requires slight cleanup to notmuch_message_get_header, since the
code previously couldn't distinguish between empty headers and headers
that are never stored in the database (previously this distinction
didn't matter).
Austin Clements [Mon, 25 Aug 2014 17:26:06 +0000 (13:26 -0400)]
lib: Report progress for combined upgrade operation
Previously, some parts of upgrade didn't report progress and for
others it was possible for the progress meter to restart at 0 part way
through the upgrade because each stage was reported separately.
Fix this by computing the total amount of work that needs to be done
up-front and updating completed work monotonically.
Austin Clements [Mon, 25 Aug 2014 17:26:05 +0000 (13:26 -0400)]
lib: Reorganize upgrade around document types
Rather than potentially making multiple passes over the same type of
data in the database, reorganize upgrade around each type of data that
may be upgraded. This eliminates code duplication, will make
multi-version upgrades faster, and will let us improve progress
reporting.
Austin Clements [Mon, 25 Aug 2014 17:26:04 +0000 (13:26 -0400)]
lib: Use database features to drive upgrade
Previously, we had database version information hard-coded in the
upgrade code. Slightly re-organize the upgrade process around the set
of new database features to be enabled by the upgrade.
Austin Clements [Mon, 25 Aug 2014 17:26:03 +0000 (13:26 -0400)]
lib: Simplify upgrade code using a transaction
Previously, the upgrade was organized as two passes -- an upgrade
pass, and a separate cleanup pass -- so the database was always in a
valid state. This change substantially simplifies this code by
performing the upgrade in a transaction and combining both passes in
to one. This 1) eliminates a lot of duplicate code between the
passes, 2) speeds up the upgrade process, 3) makes progress reporting
more accurate, 4) eliminates the potential for stale data if the
upgrade is interrupted during the cleanup pass, and 5) makes it easier
to reason about the safety of the upgrade code.
Austin Clements [Mon, 25 Aug 2014 17:26:02 +0000 (13:26 -0400)]
test: Tests for future version and unknown feature handling
Austin Clements [Mon, 25 Aug 2014 17:26:01 +0000 (13:26 -0400)]
test: Tool to build DB with specific version and features
This will let us test basic version and feature handling.
Austin Clements [Mon, 25 Aug 2014 17:26:00 +0000 (13:26 -0400)]
lib: Database version 3: Introduce fine-grained "features"
Previously, our database schema was versioned by a single number.
Each database schema change had to occur "atomically" in Notmuch's
development history: before some commit, Notmuch used version N, after
that commit, it used version N+1. Hence, each new schema version
could introduce only one change, the task of developing a schema
change fell on a single person, and it all had to happen and be
perfect in a single commit series. This made introducing a new schema
version hard. We've seen only two schema changes in the history of
Notmuch.
This commit introduces database schema version 3; hopefully the last
schema version we'll need for a while. With this version, we switch
from a single version number to "features": a set of named,
independent aspects of the database schema.
Features should make backwards compatibility easier. For many things,
it should be easy to support databases both with and without a
feature, which will allow us to make upgrades optional and will enable
"unstable" features that can be developed and tested over time.
Features also make forwards compatibility easier. The features
recorded in a database include "compatibility flags," which can
indicate to an older version of Notmuch when it must support a given
feature to open the database for read or for write. This lets us
replace the old vague "I don't recognize this version, so something
might go wrong, but I promise to try my best" warnings upon opening a
database with an unknown version with precise errors. If a database
is safe to open for read/write despite unknown features, an older
version will know that and issue no message at all. If the database
is not safe to open for read/write because of unknown features, an
older version will know that, too, and can tell the user exactly which
required features it lacks support for.
Austin Clements [Mon, 25 Aug 2014 17:25:59 +0000 (13:25 -0400)]
new: Don't report version after upgrade
The version number has always been pretty meaningless to the user and
it's about to become even more meaningless with the introduction of
"features". Hopefully, the database will remain on version 3 for some
time to come; however, the introduction of new features over time in
version 3 will necessitate upgrades within version 3. It would be
confusing if we always tell the user they've been "upgraded to version
3". If the user wants to know what's new, they should read the news.
Jani Nikula [Mon, 18 Aug 2014 16:33:21 +0000 (19:33 +0300)]
completion: fail silently if _init_completion is not found
The completion script depends on bash-completion 1.90 or later, with
_init_completion function. If that's not present, for some reason, the
completion currently fails with an ugly message, messing up user's
command line:
$ notmuch -bash: _init_completion: command not found
It's better to just not complete
Michal Sojka [Mon, 26 May 2014 14:05:57 +0000 (16:05 +0200)]
Make parsing of References and In-Reply-To header less error prone
According to RFC2822 References and In-Reply-To headers are supposed
to contain one or more Message-IDs, however older RFC822 allowed
almost any content. When both References and In-Reply-To headers ends
with something else that a Message-ID (see e.g. [1]), the thread
structure presented by notmuch is incorrect. The reason is that
notmuch treats this case as if the email contained no "replyto"
information (see _notmuch_database_link_message_to_parents).
This patch changes the parse_references() function to return the last
valid Message-ID encountered rather than NULL resulting from the last
hunk of text not being the Message-ID.
[1] https://lkml.org/lkml/headers/2014/5/19/864
Michal Sojka [Mon, 26 May 2014 14:05:56 +0000 (16:05 +0200)]
Add test for incorrect threading of messages
This happens when there is some garbage after the last Message-ID in
the References header. See for example
https://lkml.org/lkml/headers/2014/5/19/864.
Austin Clements [Fri, 8 Aug 2014 17:06:05 +0000 (13:06 -0400)]
emacs: Improved compatibility for window-body-width in Emacs < 24
Fix byte compiler warning "Warning: the function `window-body-width'
is not known to be defined." by moving our compatibility wrapper
before its use and simplify the definition to a defalias for the old
name of the function.
David Bremner [Sat, 9 Aug 2014 16:21:57 +0000 (13:21 -0300)]
Merge tag 'debian/0.18.1-2'
uploaded to Debian unstable
David Bremner [Sat, 9 Aug 2014 15:02:30 +0000 (12:02 -0300)]
debian: re-enable atomicity tests on armhf
Commit
a33ec9c seems to have fixed the problem on the armhf
porterbox (harris.debian.org).
David Bremner [Sat, 9 Aug 2014 12:42:04 +0000 (09:42 -0300)]
debian: build with emacs24 by default, disable gdb on arm64
From wookey@debian.org
id:
20140808012130.GT7605@stoneboat.aleph1.co.uk
Fixes for the port in progress of debian to arm64.
Tomi Ollila [Thu, 7 Aug 2014 15:19:50 +0000 (18:19 +0300)]
notmuch-config: talloc_strdup MAILDIR and NAME environment variables
When defined -- the pointer is soon given to talloc_free() which
expects it to be allocated by talloc.
Austin Clements [Fri, 1 Aug 2014 02:09:52 +0000 (22:09 -0400)]
util: Const version of strtok_len
Because of limitations in the C type system, we can't a strtok_len
that can work on both const string and non-const strings. The C
library solves this by taking a const char* and returning a char*
in functions like this (e.g., strchr), but that's not const-safe.
Solve it by introducing strtok_len_c, a version of strtok_len for
const strings.
Austin Clements [Fri, 1 Aug 2014 02:09:51 +0000 (22:09 -0400)]
test: Include generated dependencies for test sources
Previously the build system was generating automatic header
dependencies for test sources, but only smtp-dummy was in SRCS, so
only its dependencies were being included. Add all of the test
sources to SRCS so that the root Makefile.local includes their
dependencies.
Austin Clements [Tue, 5 Aug 2014 14:33:43 +0000 (10:33 -0400)]
new: Report and abort on upgrade failure
Previously the return status of notmuch_database_upgrade went
completely unchecked.
Austin Clements [Tue, 5 Aug 2014 01:29:16 +0000 (21:29 -0400)]
lib: Improve documentation of _notmuch_message_create_for_message_id
Clarify the state of the returned message when
_notmuch_message_create_for_message_id returns
NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND.
Austin Clements [Tue, 5 Aug 2014 01:45:21 +0000 (21:45 -0400)]
emacs: Expand default saved searches and add shortcut keys
This should help new users off to a better start with the addition of
more sensible saved searches and default shortcut keys. Most existing
users have probably customized this variable and won't be affected.
Austin Clements [Tue, 5 Aug 2014 01:45:20 +0000 (21:45 -0400)]
emacs: Introduce notmuch-jump: shortcut keys to saved searches
This introduces notmuch-jump, which is like a user-friendly,
user-configurable global prefix map for saved searches. This provides
a non-modal and much faster way to access saved searches than
notmuch-hello.
A user configures shortcut keys in notmuch-saved-searches, which are
immediately accessible from anywhere in Notmuch under the "j" key (for
"jump"). When the user hits "j", the minibuffer immediately shows a
helpful table of bindings reminiscent of a completions buffer.
This code is a combination of work from myself (originally,
"notmuch-go"), David Edmondson, and modifications from Mark Walters.
Austin Clements [Tue, 5 Aug 2014 01:57:48 +0000 (21:57 -0400)]
travis: Only notify IRC on success if the previous build failed
Without this, Travis is rather spammy. Travis will continue to notify
the IRC channel on each build failure, which seems desirable.
Austin Clements [Fri, 1 Aug 2014 02:59:13 +0000 (22:59 -0400)]
lib: Fix slight misinformation in the database schema doc
The database schema documentation made it sound like each mail
document had exactly one on-disk message file, which hasn't been true
for a long time.
Austin Clements [Fri, 1 Aug 2014 02:59:27 +0000 (22:59 -0400)]
lib: Invalidate message metadata in _notmuch_message_gen_terms
Previously, we invalidated stored message metadata in
_notmuch_message_add_term and _notmuch_message_remove_term, but not in
_notmuch_message_gen_terms. This doesn't currently result in any bugs
because of our limited uses of _notmuch_message_gen_terms, but it may
could cause trouble in the future.
Mark Oteiza [Mon, 21 Jul 2014 23:05:12 +0000 (19:05 -0400)]
config: read database.path from $MAILDIR if set
Try to read the config parameter database.path from $MAILDIR before
falling back to $HOME/mail
Mark Oteiza [Mon, 21 Jul 2014 03:06:28 +0000 (23:06 -0400)]
config: read user.name from $NAME if set
Try to read the config parameter user.name from $NAME before taking the
user name from /etc/passwd.
Austin Clements [Tue, 29 Jul 2014 16:48:00 +0000 (12:48 -0400)]
util: Make string-util.h C++-compatible
Austin Clements [Thu, 24 Jul 2014 21:19:31 +0000 (17:19 -0400)]
emacs: Clarify that notmuch-poll-script is deprecated
notmuch-poll-script has long since been deprecated in favor of
post-new hooks, but this wasn't obvious from the documentation.
Update the documentation to make this clear. Since
notmuch-poll-script could, to some extend, be used to control the path
of the notmuch binary and that use is now clearly discouraged, promote
notmuch-command to a real defcustom instead of just a variable.
Mark Walters [Wed, 16 Jul 2014 18:03:16 +0000 (19:03 +0100)]
emacs: show: make return value of notmuch-show-get-prop explicit
This makes the fact the notmuch-show-get-prop returns nil if the major
mode is neither show not tree explicit.