--- /dev/null
+Return-Path: <m.walters@qmul.ac.uk>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 4D7E3431FAF\r
+ for <notmuch@notmuchmail.org>; Fri, 4 Jan 2013 05:55:56 -0800 (PST)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -1.098\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-1.098 tagged_above=-999 required=5\r
+ tests=[DKIM_ADSP_CUSTOM_MED=0.001, FREEMAIL_FROM=0.001,\r
+ NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_MED=-2.3] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id e1Yd0mtt9-SA for <notmuch@notmuchmail.org>;\r
+ Fri, 4 Jan 2013 05:55:55 -0800 (PST)\r
+Received: from mail2.qmul.ac.uk (mail2.qmul.ac.uk [138.37.6.6])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 91A16431FAE\r
+ for <notmuch@notmuchmail.org>; Fri, 4 Jan 2013 05:55:55 -0800 (PST)\r
+Received: from smtp.qmul.ac.uk ([138.37.6.40])\r
+ by mail2.qmul.ac.uk with esmtp (Exim 4.71)\r
+ (envelope-from <m.walters@qmul.ac.uk>)\r
+ id 1Tr7kY-0002Ze-12; Fri, 04 Jan 2013 13:55:54 +0000\r
+Received: from 93-97-24-31.zone5.bethere.co.uk ([93.97.24.31] helo=localhost)\r
+ by smtp.qmul.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.69)\r
+ (envelope-from <m.walters@qmul.ac.uk>)\r
+ id 1Tr7kX-0002Vc-HT; Fri, 04 Jan 2013 13:55:53 +0000\r
+From: Mark Walters <markwalters1009@gmail.com>\r
+To: notmuch@notmuchmail.org\r
+Subject: Xapian Quote tags\r
+User-Agent: Notmuch/0.14+236~g1d0044f (http://notmuchmail.org) Emacs/23.4.1\r
+ (x86_64-pc-linux-gnu)\r
+Date: Fri, 04 Jan 2013 13:55:52 +0000\r
+Message-ID: <878v89hx1z.fsf@qmul.ac.uk>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Sender-Host-Address: 93.97.24.31\r
+X-QM-SPAM-Info: Sender has good ham record. :)\r
+X-QM-Body-MD5: 2da5113837db2c5d4ff5b044120fdc9b (of first 20000 bytes)\r
+X-SpamAssassin-Score: -1.8\r
+X-SpamAssassin-SpamBar: -\r
+X-SpamAssassin-Report: The QM spam filters have analysed this message to\r
+ determine if it is\r
+ spam. We require at least 5.0 points to mark a message as spam.\r
+ This message scored -1.8 points.\r
+ Summary of the scoring: \r
+ * -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/,\r
+ * medium trust\r
+ * [138.37.6.40 listed in list.dnswl.org]\r
+ * 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail\r
+ provider * (markwalters1009[at]gmail.com)\r
+ * 0.5 AWL AWL: From: address is in the auto white-list\r
+X-QM-Scan-Virus: ClamAV says the message is clean\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Fri, 04 Jan 2013 13:55:56 -0000\r
+\r
+\r
+Hello\r
+\r
+I would like to suggest that we Xapian quote the tags for notmuch\r
+dump/restore. The general view on irc is that we probably want to do\r
+this in the long term and I think it would be nice if we can avoid\r
+changing the dump format a second time.\r
+\r
+One problem is that our current line based parsing cannot cope with\r
+Xapian quoted newlines (which stay newlines). So allow a line to start\r
+with % which means hex-decode the whole line before passing to the main\r
+parser. Thus the query tags etc still need to be Xapian encoded.\r
+\r
+I attach a patch to show roughly what I mean. This is not complete: the\r
+dump routine does not hex encode lines with newlines yet, tests man\r
+pages etc are not updated and it is significantly unpolished. Also there\r
+should be some consolidation between parse_boolean_term and the\r
+xapian_decode routine.\r
+\r
+Despite the above caveats it broadly seems to work.\r
+\r
+Best wishes\r
+\r
+Mark\r
+\r
+\r
+\r
+\r
+>From d518e2be27ff7243ddc156699c2bfc38dec78b43 Mon Sep 17 00:00:00 2001\r
+From: Mark Walters <markwalters1009@gmail.com>\r
+Date: Fri, 4 Jan 2013 13:37:42 +0000\r
+Subject: [PATCH] notmuch dump: xapian quote tags\r
+\r
+---\r
+ notmuch-dump.c | 6 +++---\r
+ tag-util.c | 53 ++++++++++++++++++++++++++++++++++++++++++++++-------\r
+ 2 files changed, 49 insertions(+), 10 deletions(-)\r
+\r
+diff --git a/notmuch-dump.c b/notmuch-dump.c\r
+index bf01a39..e94d870 100644\r
+--- a/notmuch-dump.c\r
++++ b/notmuch-dump.c\r
+@@ -120,9 +120,9 @@ notmuch_dump_command (unused (void *ctx), int argc, char *argv[])\r
+ if (output_format == DUMP_FORMAT_SUP) {\r
+ fputs (tag_str, output);\r
+ } else {\r
+- if (hex_encode (notmuch, tag_str,\r
+- &buffer, &buffer_size) != HEX_SUCCESS) {\r
+- fprintf (stderr, "Error: failed to hex-encode tag %s\n",\r
++ if (make_boolean_term (notmuch, NULL, tag_str,\r
++ &buffer, &buffer_size)) {\r
++ fprintf (stderr, "Error: failed to xapian-encode tag %s\n",\r
+ tag_str);\r
+ return 1;\r
+ }\r
+diff --git a/tag-util.c b/tag-util.c\r
+index ca12b3b..7384afa 100644\r
+--- a/tag-util.c\r
++++ b/tag-util.c\r
+@@ -31,6 +31,38 @@ line_error (tag_parse_status_t status,\r
+ return status;\r
+ }\r
+ \r
++static int\r
++xapian_decode_tag_inplace (char *str, char* tok, size_t *tok_len)\r
++{\r
++ char *pos = str;\r
++ char *out = str;\r
++\r
++ if (*pos == '"') {\r
++ int closed = 0;\r
++ /* Skip the opening quote, find the closing quote, and\r
++ * un-double doubled internal quotes. */\r
++ for (++pos; *pos; ) {\r
++ if (*pos == '"') {\r
++ ++pos;\r
++ if (*pos != '"') {\r
++ /* Found the closing quote. */\r
++ closed = 1;\r
++ break;\r
++ }\r
++ }\r
++ *out++ = *pos++;\r
++ }\r
++ if (! closed || *pos != ' ')\r
++ return HEX_SYNTAX_ERROR;\r
++ *tok_len = pos - tok;\r
++ } else {\r
++ out = tok + (*tok_len)++;\r
++ }\r
++ /* Terminate token */\r
++ *out = '\0';\r
++ return HEX_SUCCESS;\r
++}\r
++\r
+ tag_parse_status_t\r
+ parse_tag_line (void *ctx, char *line,\r
+ tag_op_flag_t flags,\r
+@@ -60,6 +92,15 @@ parse_tag_line (void *ctx, char *line,\r
+ goto DONE;\r
+ }\r
+ \r
++ if (*tok == '%') {\r
++ tok++;\r
++ if (hex_decode_inplace (tok) != HEX_SUCCESS) {\r
++ ret = line_error (TAG_PARSE_INVALID, line_for_error,\r
++ "hex decoding of line failed", "");\r
++ goto DONE;\r
++ }\r
++ }\r
++\r
+ tag_op_list_reset (tag_ops);\r
+ \r
+ /* Parse tags. */\r
+@@ -89,23 +130,21 @@ parse_tag_line (void *ctx, char *line,\r
+ goto DONE;\r
+ }\r
+ \r
+- /* Terminate, and start next token after terminator. */\r
+- *(tok + tok_len++) = '\0';\r
+-\r
+ remove = (*tok == '-');\r
+ tag = tok + 1;\r
+ \r
+- /* Maybe refuse empty tags. */\r
++ /* Maybe refuse empty tags. Note a quoted empty tag is allowed. */\r
+ if (! (flags & TAG_FLAG_BE_GENEROUS) && *tag == '\0') {\r
+ ret = line_error (TAG_PARSE_INVALID, line_for_error,\r
+ "empty tag");\r
+ goto DONE;\r
+ }\r
+ \r
+- /* Decode tag. */\r
+- if (hex_decode_inplace (tag) != HEX_SUCCESS) {\r
++ /* Find real (quoted) end, terminate, and start next token\r
++ * after terminator. */\r
++ if (xapian_decode_tag_inplace (tag, tok, &tok_len) != HEX_SUCCESS) {\r
+ ret = line_error (TAG_PARSE_INVALID, line_for_error,\r
+- "hex decoding of tag %s failed", tag);\r
++ "xapian decoding of tag %s failed", tag);\r
+ goto DONE;\r
+ }\r
+ \r
+-- \r
+1.7.9.1\r
+\r
projects / notmuch-archives.git / commitdiff