--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 15DBA431FBD\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 06:24:53 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id keNYugqGW-o2 for <notmuch@notmuchmail.org>;\r
+ Thu, 10 Apr 2014 06:24:44 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-7.mit.edu (dmz-mailsec-scanner-7.mit.edu\r
+ [18.7.68.36])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 9854D431FBC\r
+ for <notmuch@notmuchmail.org>; Thu, 10 Apr 2014 06:24:44 -0700 (PDT)\r
+X-AuditID: 12074424-f79e26d000000c70-8b-53469b9a40a1\r
+Received: from mailhub-auth-3.mit.edu ( [18.9.21.43])\r
+ (using TLS with cipher AES256-SHA (256/256 bits))\r
+ (Client did not present a certificate)\r
+ by dmz-mailsec-scanner-7.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id A8.8B.03184.A9B96435; Thu, 10 Apr 2014 09:24:42 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+ by mailhub-auth-3.mit.edu (8.13.8/8.9.2) with ESMTP id s3ADOfiS028048; \r
+ Thu, 10 Apr 2014 09:24:42 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s3ADOcn9015845\r
+ (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+ Thu, 10 Apr 2014 09:24:40 -0400\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1WYEy6-0003DR-Kb; Thu, 10 Apr 2014 09:24:38 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org\r
+Subject: Re: [Patch v7 4/6] restore: transparently support gzipped input\r
+In-Reply-To: <1396712636-10640-5-git-send-email-david@tethera.net>\r
+References: <1396712636-10640-1-git-send-email-david@tethera.net>\r
+ <1396712636-10640-5-git-send-email-david@tethera.net>\r
+User-Agent: Notmuch/0.17+167~g3863755 (http://notmuchmail.org) Emacs/23.4.1\r
+ (i486-pc-linux-gnu)\r
+Date: Thu, 10 Apr 2014 09:24:38 -0400\r
+Message-ID: <8761mhxqp5.fsf@awakening.csail.mit.edu>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFrrHIsWRmVeSWpSXmKPExsUixCmqrTtrtluwwd+tphY3WrsZLa7fnMns\r
+ wOTxbNUtZo8th94zBzBFcdmkpOZklqUW6dslcGXcmLCAqeCWScW9k8uZGxjvqXcxcnJICJhI\r
+ LN54lh3CFpO4cG89WxcjF4eQwGwmiUVNc1hBEkICGxkl+icUQiROM0lMWryIHcJZwiix5PwV\r
+ sCo2AQ2JbfuXM4LYIgJ2Ej0XzgMVcXAIC3hIzH7qBmJyCjhJfFigAmIKCZRLvLmrCVIsKhAv\r
+ 0doP0cgioCqxfvZMsHt4gW47tf81E4QtKHFy5hMWEJtZQEvixr+XTBMYBWYhSc1CklrAyLSK\r
+ UTYlt0o3NzEzpzg1Wbc4OTEvL7VI11wvN7NELzWldBMjKBjZXVR2MDYfUjrEKMDBqMTDe3CZ\r
+ a7AQa2JZcWXuIUZJDiYlUV61SW7BQnxJ+SmVGYnFGfFFpTmpxYcYJTiYlUR4N04FyvGmJFZW\r
+ pRblw6SkOViUxHnfWlsFCwmkJ5akZqemFqQWwWRlODiUJHiXzAJqFCxKTU+tSMvMKUFIM3Fw\r
+ ggznARq+CKSGt7ggMbc4Mx0if4pRUUqc1wwkIQCSyCjNg+uFJYtXjOJArwjz7gCp4gEmGrju\r
+ V0CDmYAGp9q5gAwuSURISTUw7t50hoOjQFGiXlmwuHZLsZSAr7SgQkuZ/VrPTfUr3/h0bXnY\r
+ ZtZYUPd52oolMVpX5Tas9+jnE4tVDLrJJf41rHWN4sk3t9yzFI66XBGImFv6dfuWnzff9V1Y\r
+ uFHyp1TBCUaLlrq86mB3zllrq6afOma1MNti3s6yu/uzN7zosAmb+Gif0NMKJZbijERDLeai\r
+ 4kQAVK0GH/ECAAA=\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Thu, 10 Apr 2014 13:24:53 -0000\r
+\r
+On Sat, 05 Apr 2014, David Bremner <david@tethera.net> wrote:\r
+> We rely completely on zlib to do the right thing in detecting gzipped\r
+> input. Since our dump format is chosen to be 7 bit ascii, this should\r
+> be fine.\r
+> ---\r
+> doc/man1/notmuch-restore.rst | 8 ++++\r
+> notmuch-restore.c | 93 +++++++++++++++++++++++++++++++++-----------\r
+> test/T240-dump-restore.sh | 14 +++++++\r
+> 3 files changed, 92 insertions(+), 23 deletions(-)\r
+>\r
+> diff --git a/doc/man1/notmuch-restore.rst b/doc/man1/notmuch-restore.rst\r
+> index d6cf19a..936b138 100644\r
+> --- a/doc/man1/notmuch-restore.rst\r
+> +++ b/doc/man1/notmuch-restore.rst\r
+> @@ -50,6 +50,14 @@ Supported options for **restore** include\r
+> format, this heuristic, based the fact that batch-tag format\r
+> contains no parentheses, should be accurate.\r
+> \r
+> +GZIPPED INPUT\r
+> +=============\r
+> +\r
+> +\ **notmuch restore** will detect if the input is compressed in\r
+> +**gzip(1)** format and automatically decompress it while reading. This\r
+> +detection does not depend on file naming and in particular works for\r
+> +standard input.\r
+> +\r
+> SEE ALSO\r
+> ========\r
+> \r
+> diff --git a/notmuch-restore.c b/notmuch-restore.c\r
+> index c54d513..7abee0a 100644\r
+> --- a/notmuch-restore.c\r
+> +++ b/notmuch-restore.c\r
+> @@ -22,6 +22,7 @@\r
+> #include "hex-escape.h"\r
+> #include "tag-util.h"\r
+> #include "string-util.h"\r
+> +#include "zlib-extra.h"\r
+> \r
+> static regex_t regex;\r
+> \r
+> @@ -128,10 +129,10 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[])\r
+> tag_op_list_t *tag_ops;\r
+> \r
+> char *input_file_name = NULL;\r
+> - FILE *input = stdin;\r
+> + const char *name_for_error = NULL;\r
+> + gzFile input = NULL;\r
+> char *line = NULL;\r
+> void *line_ctx = NULL;\r
+> - size_t line_size;\r
+> ssize_t line_len;\r
+> \r
+> int ret = 0;\r
+> @@ -157,39 +158,69 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[])\r
+> };\r
+> \r
+> opt_index = parse_arguments (argc, argv, options, 1);\r
+> - if (opt_index < 0)\r
+> - return EXIT_FAILURE;\r
+> + if (opt_index < 0) {\r
+> + ret = EXIT_FAILURE;\r
+> + goto DONE;\r
+> + }\r
+> +\r
+> + name_for_error = input_file_name ? input_file_name : "stdin";\r
+> \r
+> if (! accumulate)\r
+> flags |= TAG_FLAG_REMOVE_ALL;\r
+> \r
+> - if (input_file_name) {\r
+> - input = fopen (input_file_name, "r");\r
+> - if (input == NULL) {\r
+> - fprintf (stderr, "Error opening %s for reading: %s\n",\r
+> - input_file_name, strerror (errno));\r
+> - return EXIT_FAILURE;\r
+> + errno = 0;\r
+> + if (input_file_name)\r
+> + input = gzopen (input_file_name, "r");\r
+> + else {\r
+> + int infd = dup (STDIN_FILENO);\r
+> + if (infd < 0) {\r
+> + fprintf (stderr, "Error duping stdin: %s\n",\r
+> + strerror (errno));\r
+> + ret = EXIT_FAILURE;\r
+> + goto DONE;\r
+> }\r
+> + input = gzdopen (infd, "r");\r
+> + if (! input)\r
+> + close (infd);\r
+> + }\r
+> +\r
+> + if (input == NULL) {\r
+> + fprintf (stderr, "Error opening %s for (gzip) reading: %s\n",\r
+> + name_for_error, strerror (errno));\r
+> + ret = EXIT_FAILURE;\r
+> + goto DONE;\r
+> }\r
+> \r
+> if (opt_index < argc) {\r
+> fprintf (stderr, "Unused positional parameter: %s\n", argv[opt_index]);\r
+> - return EXIT_FAILURE;\r
+> + ret = EXIT_FAILURE;\r
+> + goto DONE;\r
+> }\r
+> \r
+> tag_ops = tag_op_list_create (config);\r
+> if (tag_ops == NULL) {\r
+> fprintf (stderr, "Out of memory.\n");\r
+> - return EXIT_FAILURE;\r
+> + ret = EXIT_FAILURE;\r
+> + goto DONE;\r
+> }\r
+> \r
+> do {\r
+> - line_len = getline (&line, &line_size, input);\r
+> + util_status_t status;\r
+> +\r
+> + status = gz_getline (line_ctx, &line, &line_len, input);\r
+> \r
+> /* empty input file not considered an error */\r
+> - if (line_len < 0)\r
+> - return EXIT_SUCCESS;\r
+> + if (status == UTIL_EOF) {\r
+> + ret = EXIT_SUCCESS;\r
+> + goto DONE;\r
+> + }\r
+> \r
+> + if (status) {\r
+> + fprintf (stderr, "Error reading (gzipped) input: %s\n",\r
+> + gz_error_string(status, input));\r
+> + ret = EXIT_FAILURE;\r
+> + goto DONE;\r
+> + }\r
+> } while ((line_len == 0) ||\r
+> (line[0] == '#') ||\r
+> /* the cast is safe because we checked about for line_len < 0 */\r
+> @@ -254,21 +285,37 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[])\r
+> if (ret)\r
+> break;\r
+> \r
+> - } while ((line_len = getline (&line, &line_size, input)) != -1);\r
+> + } while (! (ret = gz_getline (line_ctx, &line, &line_len, input)));\r
+> + \r
+> \r
+> - if (line_ctx != NULL)\r
+> - talloc_free (line_ctx);\r
+> + /* EOF is normal loop termination condition, UTIL_SUCCESS is\r
+> + * impossible here */\r
+> + if (ret == UTIL_EOF) {\r
+> + ret = UTIL_SUCCESS;\r
+> + } else {\r
+> + fprintf (stderr, "Error reading (gzipped) input: %s\n",\r
+> + gz_error_string (ret, input));\r
+\r
+ret = EXIT_FAILURE; here?\r
+\r
+> + }\r
+> +\r
+> + /* currently this should not be after DONE: since we don't \r
+> + * know if the xregcomp was reached\r
+> + */\r
+> \r
+> if (input_format == DUMP_FORMAT_SUP)\r
+> regfree (®ex);\r
+> \r
+> - if (line)\r
+> - free (line);\r
+> + DONE:\r
+> + if (line_ctx != NULL)\r
+> + talloc_free (line_ctx);\r
+> \r
+> - notmuch_database_destroy (notmuch);\r
+> + if (notmuch)\r
+> + notmuch_database_destroy (notmuch);\r
+> \r
+> - if (input != stdin)\r
+> - fclose (input);\r
+> + if (input && gzclose_r (input)) {\r
+> + fprintf (stderr, "Error closing %s: %s\n",\r
+> + name_for_error, gzerror (input, NULL));\r
+> + ret = EXIT_FAILURE;\r
+> + }\r
+> \r
+> return ret ? EXIT_FAILURE : EXIT_SUCCESS;\r
+> }\r
+> diff --git a/test/T240-dump-restore.sh b/test/T240-dump-restore.sh\r
+> index b6d8602..efe463e 100755\r
+> --- a/test/T240-dump-restore.sh\r
+> +++ b/test/T240-dump-restore.sh\r
+> @@ -80,6 +80,20 @@ notmuch dump --gzip --output=dump-gzip-outfile.gz\r
+> gunzip dump-gzip-outfile.gz\r
+> test_expect_equal_file dump.expected dump-gzip-outfile\r
+> \r
+> +test_begin_subtest "restoring gzipped stdin"\r
+> +notmuch dump --gzip --output=backup.gz\r
+> +notmuch tag +new_tag '*'\r
+> +notmuch restore < backup.gz\r
+> +notmuch dump --output=dump.actual\r
+> +test_expect_equal_file dump.expected dump.actual\r
+> +\r
+> +test_begin_subtest "restoring gzipped file"\r
+> +notmuch dump --gzip --output=backup.gz\r
+> +notmuch tag +new_tag '*'\r
+> +notmuch restore --input=backup.gz\r
+> +notmuch dump --output=dump.actual\r
+> +test_expect_equal_file dump.expected dump.actual\r
+> +\r
+> # Note, we assume all messages from cworth have a message-id\r
+> # containing cworth.org\r
+> \r
+> -- \r
+> 1.9.0\r
+>\r
+> _______________________________________________\r
+> notmuch mailing list\r
+> notmuch@notmuchmail.org\r
+> http://notmuchmail.org/mailman/listinfo/notmuch\r
projects / notmuch-archives.git / commitdiff