1 Return-Path: <bremner@tethera.net>
\r
2 X-Original-To: notmuch@notmuchmail.org
\r
3 Delivered-To: notmuch@notmuchmail.org
\r
4 Received: from localhost (localhost [127.0.0.1])
\r
5 by olra.theworths.org (Postfix) with ESMTP id ABFA3431FB6
\r
6 for <notmuch@notmuchmail.org>; Thu, 6 Dec 2012 17:27:10 -0800 (PST)
\r
7 X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
\r
11 X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
\r
13 Received: from olra.theworths.org ([127.0.0.1])
\r
14 by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
\r
15 with ESMTP id u-Cm-Q3sqx-V for <notmuch@notmuchmail.org>;
\r
16 Thu, 6 Dec 2012 17:27:09 -0800 (PST)
\r
17 Received: from tesseract.cs.unb.ca (tesseract.cs.unb.ca [131.202.240.238])
\r
18 (using TLSv1 with cipher AES256-SHA (256/256 bits))
\r
19 (No client certificate requested)
\r
20 by olra.theworths.org (Postfix) with ESMTPS id 3550A431FAE
\r
21 for <notmuch@notmuchmail.org>; Thu, 6 Dec 2012 17:27:09 -0800 (PST)
\r
22 Received: from fctnnbsc30w-142167090129.dhcp-dynamic.fibreop.nb.bellaliant.net
\r
23 ([142.167.90.129] helo=zancas.localnet)
\r
24 by tesseract.cs.unb.ca with esmtpsa
\r
25 (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72)
\r
26 (envelope-from <bremner@tethera.net>) id 1TgmiZ-0003Np-7E
\r
27 for notmuch@notmuchmail.org; Thu, 06 Dec 2012 21:27:08 -0400
\r
28 Received: from bremner by zancas.localnet with local (Exim 4.80)
\r
29 (envelope-from <bremner@tethera.net>) id 1TgmiT-0004kE-LV
\r
30 for notmuch@notmuchmail.org; Thu, 06 Dec 2012 21:27:01 -0400
\r
31 From: david@tethera.net
\r
32 To: notmuch@notmuchmail.org
\r
33 Subject: V3b of batch tagging/dump/restore patches
\r
34 Date: Thu, 6 Dec 2012 21:26:38 -0400
\r
35 Message-Id: <1354843607-17980-1-git-send-email-david@tethera.net>
\r
36 X-Mailer: git-send-email 1.7.10.4
\r
38 X-BeenThere: notmuch@notmuchmail.org
\r
39 X-Mailman-Version: 2.1.13
\r
41 List-Id: "Use and development of the notmuch mail system."
\r
42 <notmuch.notmuchmail.org>
\r
43 List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
\r
44 <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
\r
45 List-Archive: <http://notmuchmail.org/pipermail/notmuch>
\r
46 List-Post: <mailto:notmuch@notmuchmail.org>
\r
47 List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
\r
48 List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
\r
49 <mailto:notmuch-request@notmuchmail.org?subject=subscribe>
\r
50 X-List-Received-Date: Fri, 07 Dec 2012 01:27:10 -0000
\r
52 Here is a second piece of the tagging/dump/restore series.
\r
54 it obsoletes 8 of the patches in the series
\r
56 id:1353792017-31459-1-git-send-email-david@tethera.net
\r
58 [Patch v3b 1/9] notmuch-dump: add --format=(batch-tag|sup)
\r
59 [Patch v3b 3/9] util: add string-util.[ch]
\r
60 [Patch v3b 4/9] tag-util.[ch]: New files for common tagging routines
\r
61 [Patch v3b 5/9] notmuch-restore: add support for input format
\r
62 [Patch v3b 6/9] test: update dump-restore roundtripping test for
\r
63 [Patch v3b 7/9] test: second set of dump/restore --format=batch-tag
\r
64 [Patch v3b 8/9] notmuch-{dump,restore}.1: document new format
\r
65 [Patch v3b 9/9] tag-util: optimization of tag application
\r
67 It adds one new patch
\r
69 [Patch v3b 2/9] test: add sanity check for dump --format=batch-tag.
\r
71 I still have to work through some of the comments on the batch
\r
72 tagging; I still intend for that to follow fairly shortly, I just want
\r
73 to break the series at a logical point.
\r
75 Most of the changes are detailed in the following log (of changes
\r
76 before I squashed them)
\r
78 commit 5045d2f58beb4c3bc8e10f9419341e1c1b7748f2
\r
79 Author: David Bremner <bremner@debian.org>
\r
80 Date: Tue Dec 4 13:39:48 2012 -0400
\r
82 fixup for tag-util error messages
\r
84 diff --git a/tag-util.c b/tag-util.c
\r
85 index 2bb8355..ea05ee5 100644
\r
88 @@ -86,9 +86,8 @@ parse_tag_line (void *ctx, char *line,
\r
90 /* tok now points to the query string */
\r
91 if (hex_decode_inplace (tok) != HEX_SUCCESS) {
\r
92 - /* FIXME: line has been modified! */
\r
93 - fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
\r
95 + fprintf (stderr, "Hex decoding of %s failed\n",
\r
101 commit 5a1d697dc408c67424d586b6377976fdfb86d4ed
\r
102 Author: David Bremner <bremner@debian.org>
\r
103 Date: Tue Dec 4 13:40:09 2012 -0400
\r
105 fixup for restore error messages
\r
107 diff --git a/notmuch-restore.c b/notmuch-restore.c
\r
108 index 22fcd2d..e7584bb 100644
\r
109 --- a/notmuch-restore.c
\r
110 +++ b/notmuch-restore.c
\r
111 @@ -77,7 +77,7 @@ parse_sup_line (void *ctx, char *line,
\r
113 rerr = xregexec (®ex, line, 3, match, 0);
\r
114 if (rerr == REG_NOMATCH) {
\r
115 - fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
\r
116 + fprintf (stderr, "Warning: Ignoring invalid sup format line: %s\n",
\r
121 commit 7136d3b4e974a4ba8e247f328c71362efd0c9e11
\r
122 Author: David Bremner <bremner@debian.org>
\r
123 Date: Tue Dec 4 22:35:30 2012 -0400
\r
125 fixup for tag-util error handling and permit the null set of tag operations
\r
127 diff --git a/tag-util.c b/tag-util.c
\r
128 index ea05ee5..de7ecc8 100644
\r
131 @@ -21,6 +21,8 @@ parse_tag_line (void *ctx, char *line,
\r
134 size_t tok_len = 0;
\r
135 + char *line_for_error=talloc_strdup (ctx, line);
\r
138 chomp_newline (line);
\r
140 @@ -29,8 +31,10 @@ parse_tag_line (void *ctx, char *line,
\r
143 /* Skip empty and comment lines. */
\r
144 - if (*tok == '\0' || *tok == '#')
\r
146 + if (*tok == '\0' || *tok == '#') {
\r
151 tag_op_list_reset (tag_ops);
\r
153 @@ -51,8 +55,9 @@ parse_tag_line (void *ctx, char *line,
\r
155 /* If tag is terminated by NUL, there's no query string. */
\r
156 if (*(tok + tok_len) == '\0') {
\r
159 + fprintf (stderr, "no query string: %s\n", line_for_error);
\r
164 /* Terminate, and start next token after terminator. */
\r
165 @@ -63,37 +68,43 @@ parse_tag_line (void *ctx, char *line,
\r
167 /* Maybe refuse empty tags. */
\r
168 if (!(flags & TAG_FLAG_BE_GENEROUS) && *tag == '\0') {
\r
171 + fprintf (stderr, "Error: empty tag: %s\n", line_for_error);
\r
176 if (hex_decode_inplace (tag) != HEX_SUCCESS) {
\r
179 + fprintf (stderr, "Hex decoding of tag %s failed\n",
\r
185 - if (tag_op_list_append (ctx, tag_ops, tag, remove))
\r
187 + if (tag_op_list_append (ctx, tag_ops, tag, remove)) {
\r
193 - if (tok == NULL || tag_ops->count == 0) {
\r
194 - /* FIXME: line has been modified! */
\r
195 + if (tok == NULL) {
\r
196 fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
\r
204 /* tok now points to the query string */
\r
205 if (hex_decode_inplace (tok) != HEX_SUCCESS) {
\r
206 - fprintf (stderr, "Hex decoding of %s failed\n",
\r
207 + fprintf (stderr, "Hex decoding of query %s failed\n",
\r
214 *query_string = tok;
\r
218 + talloc_free (line_for_error);
\r
224 commit 5912c738d3683aae24ca5529839eb0513520d190
\r
225 Author: David Bremner <bremner@debian.org>
\r
226 Date: Thu Dec 6 07:49:15 2012 -0400
\r
228 fixup for id:87wqx1qrmq.fsf@nikula.org; use size_t for tag_op_list count
\r
230 diff --git a/tag-util.c b/tag-util.c
\r
231 index de7ecc8..1a0cf53 100644
\r
235 #include "string-util.h"
\r
236 #include "tag-util.h"
\r
237 #include "hex-escape.h"
\r
238 +#include <assert.h>
\r
240 struct _tag_operation_t {
\r
242 @@ -9,8 +10,8 @@ struct _tag_operation_t {
\r
244 struct _tag_op_list_t {
\r
245 tag_operation_t *ops;
\r
253 @@ -44,7 +45,7 @@ parse_tag_line (void *ctx, char *line,
\r
256 /* Optional explicit end of tags marker. */
\r
257 - if (strncmp (tok, "--", tok_len) == 0) {
\r
258 + if (tok_len == 2 && strncmp (tok, "--", tok_len) == 0) {
\r
259 tok = strtok_len (tok + tok_len, " ", &tok_len);
\r
262 @@ -126,17 +127,16 @@ makes_changes (notmuch_message_t *message,
\r
263 tag_op_list_t *list,
\r
264 tag_op_flag_t flags)
\r
269 notmuch_tags_t *tags;
\r
270 notmuch_bool_t changes = FALSE;
\r
273 /* First, do we delete an existing tag? */
\r
275 for (tags = notmuch_message_get_tags (message);
\r
276 ! changes && notmuch_tags_valid (tags);
\r
277 notmuch_tags_move_to_next (tags)) {
\r
279 const char *cur_tag = notmuch_tags_get (tags);
\r
280 int last_op = (flags & TAG_FLAG_REMOVE_ALL) ? -1 : 0;
\r
282 @@ -182,8 +182,7 @@ tag_op_list_apply (notmuch_message_t *message,
\r
283 tag_op_list_t *list,
\r
284 tag_op_flag_t flags)
\r
289 notmuch_status_t status = 0;
\r
290 tag_operation_t *tag_ops = list->ops;
\r
292 @@ -199,7 +198,7 @@ tag_op_list_apply (notmuch_message_t *message,
\r
293 if (flags & TAG_FLAG_REMOVE_ALL) {
\r
294 status = notmuch_message_remove_all_tags (message);
\r
296 - message_error (message, status, "removing all tags" );
\r
297 + message_error (message, status, "removing all tags");
\r
301 @@ -241,8 +240,8 @@ tag_op_list_apply (notmuch_message_t *message,
\r
305 -/* Array of tagging operations (add or remove), terminated with an
\r
306 - * empty element. Size will be increased as necessary. */
\r
307 +/* Array of tagging operations (add or remove. Size will be increased
\r
308 + * as necessary. */
\r
311 tag_op_list_create (void *ctx)
\r
312 @@ -299,6 +298,7 @@ tag_op_list_append (void *ctx,
\r
314 tag_op_list_isremove (const tag_op_list_t *list, size_t i)
\r
316 + assert (i < list->count);
\r
317 return list->ops[i].remove;
\r
320 @@ -329,5 +329,6 @@ tag_op_list_size (const tag_op_list_t *list)
\r
322 tag_op_list_tag (const tag_op_list_t *list, size_t i)
\r
324 + assert (i < list->count);
\r
325 return list->ops[i].tag;
\r
328 commit e1af69d57854d5c6e927b0870be97e9d2e2f28ea
\r
329 Author: David Bremner <bremner@debian.org>
\r
330 Date: Thu Dec 6 07:51:43 2012 -0400
\r
332 changes for id:87wqx1qrmq.fsf@nikula.org. tag_op_list_t.count -> size_t
\r
334 diff --git a/tag-util.c b/tag-util.c
\r
335 index 1a0cf53..ad13147 100644
\r
338 @@ -22,8 +22,8 @@ parse_tag_line (void *ctx, char *line,
\r
341 size_t tok_len = 0;
\r
342 - char *line_for_error=talloc_strdup (ctx, line);
\r
344 + char *line_for_error = talloc_strdup (ctx, line);
\r
347 chomp_newline (line);
\r
349 @@ -33,7 +33,7 @@ parse_tag_line (void *ctx, char *line,
\r
351 /* Skip empty and comment lines. */
\r
352 if (*tok == '\0' || *tok == '#') {
\r
358 @@ -68,7 +68,7 @@ parse_tag_line (void *ctx, char *line,
\r
361 /* Maybe refuse empty tags. */
\r
362 - if (!(flags & TAG_FLAG_BE_GENEROUS) && *tag == '\0') {
\r
363 + if (! (flags & TAG_FLAG_BE_GENEROUS) && *tag == '\0') {
\r
364 fprintf (stderr, "Error: empty tag: %s\n", line_for_error);
\r
367 @@ -76,7 +76,7 @@ parse_tag_line (void *ctx, char *line,
\r
369 if (hex_decode_inplace (tag) != HEX_SUCCESS) {
\r
370 fprintf (stderr, "Hex decoding of tag %s failed\n",
\r
376 @@ -103,7 +103,7 @@ parse_tag_line (void *ctx, char *line,
\r
379 *query_string = tok;
\r
382 talloc_free (line_for_error);
\r
386 commit 3f00fa4eba68876635df86ea54f60b68d172f580
\r
387 Author: David Bremner <bremner@debian.org>
\r
388 Date: Thu Dec 6 08:30:58 2012 -0400
\r
390 changes for id:87zk1wd1ko.fsf@nikula.org
\r
392 diff --git a/notmuch-restore.c b/notmuch-restore.c
\r
393 index e7584bb..41b742f 100644
\r
394 --- a/notmuch-restore.c
\r
395 +++ b/notmuch-restore.c
\r
396 @@ -48,11 +48,10 @@ tag_message (unused (void *ctx),
\r
398 /* In order to detect missing messages, this check/optimization is
\r
399 * intentionally done *after* first finding the message. */
\r
400 - if ( (flags & TAG_FLAG_REMOVE_ALL) || (tag_op_list_size (tag_ops)))
\r
401 + if ((flags & TAG_FLAG_REMOVE_ALL) || tag_op_list_size (tag_ops))
\r
402 tag_op_list_apply (message, tag_ops, flags);
\r
405 - notmuch_message_destroy (message);
\r
406 + notmuch_message_destroy (message);
\r
410 @@ -184,6 +183,12 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
\r
414 + tag_ops = tag_op_list_create (ctx);
\r
415 + if (tag_ops == NULL) {
\r
416 + fprintf (stderr, "Out of memory.\n");
\r
420 for (p = line; *p; p++) {
\r
422 input_format = DUMP_FORMAT_SUP;
\r
423 @@ -198,28 +203,28 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
\r
425 INTERNAL_ERROR ("compile time constant regex failed.");
\r
427 - tag_ops = tag_op_list_create (ctx);
\r
428 - if (tag_ops == NULL) {
\r
429 - fprintf (stderr, "Out of memory.\n");
\r
434 char *query_string;
\r
436 if (input_format == DUMP_FORMAT_SUP) {
\r
437 - ret = parse_sup_line (ctx, line, &query_string, tag_ops);
\r
438 + ret = parse_sup_line (ctx, line, &query_string, tag_ops);
\r
440 - ret = parse_tag_line (ctx, line, TAG_FLAG_BE_GENEROUS,
\r
441 - &query_string, tag_ops);
\r
442 + ret = parse_tag_line (ctx, line, TAG_FLAG_BE_GENEROUS,
\r
443 + &query_string, tag_ops);
\r
446 - if ( strncmp ("id:", query_string, 3) != 0) {
\r
447 + if (strncmp ("id:", query_string, 3) != 0) {
\r
448 fprintf (stderr, "Unsupported query: %s\n", query_string);
\r
451 - /* delete id: from front of string; tag_message expects a
\r
452 - * raw message-id */
\r
453 + /* delete id: from front of string; tag_message
\r
454 + * expects a raw message-id.
\r
456 + * XXX: Note that query string id:foo and bar will be
\r
457 + * interpreted as a message id "foo and bar". This
\r
458 + * should eventually be fixed to give a better error
\r
461 query_string = query_string + 3;
\r
464 @@ -233,8 +238,8 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
\r
466 } while ((line_len = getline (&line, &line_size, input)) != -1);
\r
469 - regfree (®ex);
\r
470 + if (input_format == DUMP_FORMAT_SUP)
\r
471 + regfree (®ex);
\r
476 commit 473fa928931080004706cff169c7cc9337601172
\r
477 Author: David Bremner <bremner@debian.org>
\r
478 Date: Thu Dec 6 13:33:42 2012 -0400
\r
480 fixup: notmuch-restore only auto-detect in auto mode
\r
482 diff --git a/notmuch-restore.c b/notmuch-restore.c
\r
483 index 41b742f..ceec2d3 100644
\r
484 --- a/notmuch-restore.c
\r
485 +++ b/notmuch-restore.c
\r
486 @@ -189,7 +189,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char *argv[])
\r
490 - for (p = line; *p; p++) {
\r
491 + for (p = line; (input_format == DUMP_FORMAT_AUTO) && *p; p++) {
\r
493 input_format = DUMP_FORMAT_SUP;
\r
496 commit ee7d25521e3f6f70b3f4fb79f586a11d39efec15
\r
497 Author: David Bremner <bremner@debian.org>
\r
498 Date: Thu Dec 6 19:39:37 2012 -0400
\r
500 changes for id:87wqx0d124.fsf@nikula.org; no deprecation for the moment
\r
502 diff --git a/man/man1/notmuch-dump.1 b/man/man1/notmuch-dump.1
\r
503 index 9f59905..770b00f 100644
\r
504 --- a/man/man1/notmuch-dump.1
\r
505 +++ b/man/man1/notmuch-dump.1
\r
506 @@ -64,15 +64,16 @@ and tags containing whitespace or non-\fBascii\fR(7) characters.
\r
507 Each line has the form
\r
510 -.RI "+<" "encoded-tag" "> " "" "+<" "encoded-tag" "> ... -- " "" " <" encoded-message-id >
\r
511 +.RI "+<" "encoded-tag" "> " "" "+<" "encoded-tag" "> ... -- " "" " id:<" encoded-message-id >
\r
513 where encoded means that every byte not matching the regex
\r
514 -.B [A-Za-z0-9+-_@=.:,]
\r
515 +.B [A-Za-z0-9@=.,_+-]
\r
518 where nn is the two digit hex encoding.
\r
519 The astute reader will notice this is a special case of the batch input
\r
520 -format for \fBnotmuch-tag\fR(1).
\r
521 +format for \fBnotmuch-tag\fR(1); note that the single message-id query is
\r
522 +mandatory for \fBnotmuch-restore\fR(1).
\r
526 diff --git a/man/man1/notmuch-restore.1 b/man/man1/notmuch-restore.1
\r
527 index 3860829..6bba628 100644
\r
528 --- a/man/man1/notmuch-restore.1
\r
529 +++ b/man/man1/notmuch-restore.1
\r
530 @@ -32,8 +32,8 @@ replacing each message's tags as they are read in from the dump file.
\r
532 .B \-\-format=(sup|batch-tag|auto)
\r
534 -Notmuch restore supports two plain text dump formats, with one message-id
\r
535 -per line, and a list of tags.
\r
536 +Notmuch restore supports two plain text dump formats, with each line
\r
537 +specifying a message-id and a set of tags.
\r
538 For details of the actual formats, see \fBnotmuch-dump\fR(1).
\r
542 commit 96c383be46cdb8ceaf7ed15590ef876799d6357e
\r
543 Author: David Bremner <bremner@debian.org>
\r
544 Date: Thu Dec 6 20:40:55 2012 -0400
\r
546 Changes for id:87txs4cy7v.fsf@nikula.org
\r
548 diff --git a/tag-util.c b/tag-util.c
\r
549 index ad13147..9ab07e9 100644
\r
552 @@ -140,9 +140,11 @@ makes_changes (notmuch_message_t *message,
\r
553 const char *cur_tag = notmuch_tags_get (tags);
\r
554 int last_op = (flags & TAG_FLAG_REMOVE_ALL) ? -1 : 0;
\r
556 - for (i = 0; i < list->count; i++) {
\r
557 + /* slight contortions to count down with an unsigned index */
\r
558 + for (i = list->count; i-- > 0; /*nothing*/) {
\r
559 if (strcmp (cur_tag, list->ops[i].tag) == 0) {
\r
560 last_op = list->ops[i].remove ? -1 : 1;
\r
565 @@ -157,6 +159,9 @@ makes_changes (notmuch_message_t *message,
\r
566 for (i = 0; i < list->count; i++) {
\r
567 notmuch_bool_t exists = FALSE;
\r
569 + if (list->ops[i].remove)
\r
572 for (tags = notmuch_message_get_tags (message);
\r
573 notmuch_tags_valid (tags);
\r
574 notmuch_tags_move_to_next (tags)) {
\r
575 @@ -168,9 +173,11 @@ makes_changes (notmuch_message_t *message,
\r
577 notmuch_tags_destroy (tags);
\r
579 - /* the following test is conservative, it's ok to think we
\r
580 - * make changes when we don't */
\r
581 - if ( ! exists && ! list->ops[i].remove )
\r
582 + /* the following test is conservative,
\r
583 + * in the sense it ignores cases like +foo ... -foo
\r
584 + * but this is OK from a correctness point of view
\r