--- /dev/null
+Return-Path: <jani@nikula.org>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 1A5CB431FBC\r
+ for <notmuch@notmuchmail.org>; Sat, 29 Mar 2014 02:25:27 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id GA8I+embZj8Y for <notmuch@notmuchmail.org>;\r
+ Sat, 29 Mar 2014 02:25:20 -0700 (PDT)\r
+Received: from mail-ee0-f43.google.com (mail-ee0-f43.google.com\r
+ [74.125.83.43]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client\r
+ certificate requested) by olra.theworths.org (Postfix) with ESMTPS id\r
+ 5059F431FAF for <notmuch@notmuchmail.org>; Sat, 29 Mar 2014 02:25:20 -0700\r
+ (PDT)\r
+Received: by mail-ee0-f43.google.com with SMTP id e53so4876282eek.30\r
+ for <notmuch@notmuchmail.org>; Sat, 29 Mar 2014 02:25:17 -0700 (PDT)\r
+X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;\r
+ d=1e100.net; s=20130820;\r
+ h=x-gm-message-state:from:to:subject:in-reply-to:references\r
+ :user-agent:date:message-id:mime-version:content-type;\r
+ bh=+z21J9bCP/itUjQx1xsrXDUL6xEXLoT2amKlaIVsx3s=;\r
+ b=hUt4eKd8PbiF67AP/gCSMeBrOnDhHNU8wjny9dSRNExY7hVht5FxXp9HbzZhIcqZLt\r
+ 8AdyxLQ2yGaaH4cgETICZZ273hCcdpqRyh4ZlX1/Pg45JFko8YIe4sv0s0VbEgEbwtSM\r
+ bqOetCHlyrwWrB11exIcsmLNmWwmpkNKpzDse8SkSUt3RfrPmFtcgPTtNmw2uB4BJV83\r
+ HHaUsY4czTgc6zofVZQkvRvCkRjGfziwIxcCebeakiv6hN25Apcm6JfxhS8udCmPt/jU\r
+ CIPzE+EXwZZGd0Z3ZNNY6s6ThQhjZXZ0ltoQfZxdthEjIxqtuqjpTAUtyl7/ZpGiCmMu\r
+ Grdw==\r
+X-Gm-Message-State:\r
+ ALoCoQlaUGLVXMAghuKux+WfPe+LmNUMuwWHi4Bz+/Omkc2V3mThPE4wu54kOeTFPxllyeEhvvFg\r
+X-Received: by 10.14.215.9 with SMTP id d9mr885559eep.64.1396085116412;\r
+ Sat, 29 Mar 2014 02:25:16 -0700 (PDT)\r
+Received: from localhost (dsl-hkibrasgw2-58c36f-91.dhcp.inet.fi.\r
+ [88.195.111.91])\r
+ by mx.google.com with ESMTPSA id g3sm17442780eet.35.2014.03.29.02.25.14\r
+ for <multiple recipients>\r
+ (version=TLSv1.2 cipher=RC4-SHA bits=128/128);\r
+ Sat, 29 Mar 2014 02:25:15 -0700 (PDT)\r
+From: Jani Nikula <jani@nikula.org>\r
+To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org\r
+Subject: Re: [PATCH] RFC: impliment gzipped output for notmuch dump\r
+In-Reply-To: <1396056046-2247-1-git-send-email-david@tethera.net>\r
+References: <874n2kipq4.fsf@qmul.ac.uk>\r
+ <1396056046-2247-1-git-send-email-david@tethera.net>\r
+User-Agent: Notmuch/0.17+164~gcd5fd5a4837e (http://notmuchmail.org)\r
+ Emacs/24.3.1 (x86_64-pc-linux-gnu)\r
+Date: Sat, 29 Mar 2014 11:25:12 +0200\r
+Message-ID: <87wqfdbblz.fsf@nikula.org>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Sat, 29 Mar 2014 09:25:27 -0000\r
+\r
+On Sat, 29 Mar 2014, David Bremner <david@tethera.net> wrote:\r
+> ---\r
+>\r
+> This obviously needs at least one piece of tidying, and some tests.\r
+>\r
+> My motivation here is the ability to make gzipped backups from within\r
+> notmuch, e.g. as part of an upgrade procedure.\r
+\r
+Is it reasonable to require zlib? It's possible to have conditional\r
+build for this, but it will be laborous to do it cleanly. I could be\r
+persuaded either way, perhaps more easily to just requiring it.\r
+\r
+If you do require zlib, the missing dependencies error message from\r
+configure should include a check for have_zlib and a note about it\r
+similar to other required dependencies.\r
+\r
+Also, it would seem natural to support gzipped input in restore as\r
+well... do you think it would be silly to support gzipped output only\r
+internally until we've added gzipped input too? This could be in a later\r
+release in the future.\r
+\r
+\r
+BR,\r
+Jani.\r
+\r
+\r
+\r
+>\r
+>\r
+> Makefile.local | 2 +-\r
+> configure | 19 ++++++++++++++++++-\r
+> notmuch-client.h | 4 +++-\r
+> notmuch-dump.c | 41 +++++++++++++++++++++++++++--------------\r
+> 4 files changed, 49 insertions(+), 17 deletions(-)\r
+>\r
+> diff --git a/Makefile.local b/Makefile.local\r
+> index cb7b106..e5a20a7 100644\r
+> --- a/Makefile.local\r
+> +++ b/Makefile.local\r
+> @@ -41,7 +41,7 @@ PV_FILE=bindings/python/notmuch/version.py\r
+> # Smash together user's values with our extra values\r
+> FINAL_CFLAGS = -DNOTMUCH_VERSION=$(VERSION) $(CPPFLAGS) $(CFLAGS) $(WARN_CFLAGS) $(extra_cflags) $(CONFIGURE_CFLAGS)\r
+> FINAL_CXXFLAGS = $(CPPFLAGS) $(CXXFLAGS) $(WARN_CXXFLAGS) $(extra_cflags) $(extra_cxxflags) $(CONFIGURE_CXXFLAGS)\r
+> -FINAL_NOTMUCH_LDFLAGS = $(LDFLAGS) -Lutil -lutil -Llib -lnotmuch $(AS_NEEDED_LDFLAGS) $(GMIME_LDFLAGS) $(TALLOC_LDFLAGS)\r
+> +FINAL_NOTMUCH_LDFLAGS = $(LDFLAGS) -Lutil -lutil -Llib -lnotmuch $(AS_NEEDED_LDFLAGS) $(GMIME_LDFLAGS) $(TALLOC_LDFLAGS) $(ZLIB_LDFLAGS)\r
+> FINAL_NOTMUCH_LINKER = CC\r
+> ifneq ($(LINKER_RESOLVES_LIBRARY_DEPENDENCIES),1)\r
+> FINAL_NOTMUCH_LDFLAGS += $(CONFIGURE_LDFLAGS)\r
+> diff --git a/configure b/configure\r
+> index 1d430b9..02ef785 100755\r
+> --- a/configure\r
+> +++ b/configure\r
+> @@ -340,6 +340,18 @@ else\r
+> errors=$((errors + 1))\r
+> fi\r
+> \r
+> +printf "Checking for zlib development files... "\r
+> +have_zlib=0\r
+> +if pkg-config --exists zlib; then\r
+> + printf "Yes.\n"\r
+> + have_zlib=1\r
+> + zlib_cflags=$(pkg-config --cflags zlib)\r
+> + zlib_ldflags=$(pkg-config --libs zlib)\r
+> +else\r
+> + printf "No.\n"\r
+> + errors=$((errors + 1))\r
+> +fi\r
+> +\r
+> printf "Checking for talloc development files... "\r
+> if pkg-config --exists talloc; then\r
+> printf "Yes.\n"\r
+> @@ -844,6 +856,10 @@ XAPIAN_LDFLAGS = ${xapian_ldflags}\r
+> GMIME_CFLAGS = ${gmime_cflags}\r
+> GMIME_LDFLAGS = ${gmime_ldflags}\r
+> \r
+> +# Flags needed to compile and link against zlib\r
+> +ZLIB_CFLAGS = ${zlib_cflags}\r
+> +ZLIB_LDFLAGS = ${zlib_ldflags}\r
+> +\r
+> # Flags needed to compile and link against talloc\r
+> TALLOC_CFLAGS = ${talloc_cflags}\r
+> TALLOC_LDFLAGS = ${talloc_ldflags}\r
+> @@ -882,6 +898,7 @@ CONFIGURE_CFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\\r
+> -DUTIL_BYTE_ORDER=\$(UTIL_BYTE_ORDER)\r
+> \r
+> CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\\r
+> + \$(ZLIB_CFLAGS) \\\r
+> \$(TALLOC_CFLAGS) -DHAVE_VALGRIND=\$(HAVE_VALGRIND) \\\r
+> \$(VALGRIND_CFLAGS) \$(XAPIAN_CXXFLAGS) \\\r
+> -DHAVE_STRCASESTR=\$(HAVE_STRCASESTR) \\\r
+> @@ -892,5 +909,5 @@ CONFIGURE_CXXFLAGS = -DHAVE_GETLINE=\$(HAVE_GETLINE) \$(GMIME_CFLAGS) \\\r
+> -DHAVE_XAPIAN_COMPACT=\$(HAVE_XAPIAN_COMPACT) \\\r
+> -DUTIL_BYTE_ORDER=\$(UTIL_BYTE_ORDER)\r
+> \r
+> -CONFIGURE_LDFLAGS = \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(XAPIAN_LDFLAGS)\r
+> +CONFIGURE_LDFLAGS = \$(GMIME_LDFLAGS) \$(TALLOC_LDFLAGS) \$(ZLIB_LDFLAGS) \$(XAPIAN_LDFLAGS)\r
+> EOF\r
+> diff --git a/notmuch-client.h b/notmuch-client.h\r
+> index d110648..e1efbe0 100644\r
+> --- a/notmuch-client.h\r
+> +++ b/notmuch-client.h\r
+> @@ -450,7 +450,9 @@ typedef enum dump_formats {\r
+> int\r
+> notmuch_database_dump (notmuch_database_t *notmuch,\r
+> const char *output_file_name,\r
+> - const char *query_str, dump_format_t output_format);\r
+> + const char *query_str,\r
+> + dump_format_t output_format,\r
+> + notmuch_bool_t gzip_output);\r
+> \r
+> #include "command-line-arguments.h"\r
+> #endif\r
+> diff --git a/notmuch-dump.c b/notmuch-dump.c\r
+> index 21702d7..029f90a 100644\r
+> --- a/notmuch-dump.c\r
+> +++ b/notmuch-dump.c\r
+> @@ -21,10 +21,12 @@\r
+> #include "notmuch-client.h"\r
+> #include "hex-escape.h"\r
+> #include "string-util.h"\r
+> +#include <zlib.h>\r
+> +\r
+> \r
+> static int\r
+> -database_dump_file (notmuch_database_t *notmuch, FILE *output,\r
+> - const char *query_str, int output_format)\r
+> +database_dump_file (notmuch_database_t *notmuch, gzFile output,\r
+> + const char *query_str, int output_format)\r
+> {\r
+> notmuch_query_t *query;\r
+> notmuch_messages_t *messages;\r
+> @@ -69,7 +71,7 @@ database_dump_file (notmuch_database_t *notmuch, FILE *output,\r
+> }\r
+> \r
+> if (output_format == DUMP_FORMAT_SUP) {\r
+> - fprintf (output, "%s (", message_id);\r
+> + gzprintf (output, "%s (", message_id);\r
+> }\r
+> \r
+> for (tags = notmuch_message_get_tags (message);\r
+> @@ -78,12 +80,12 @@ database_dump_file (notmuch_database_t *notmuch, FILE *output,\r
+> const char *tag_str = notmuch_tags_get (tags);\r
+> \r
+> if (! first)\r
+> - fputs (" ", output);\r
+> + gzputs (output, " ");\r
+> \r
+> first = 0;\r
+> \r
+> if (output_format == DUMP_FORMAT_SUP) {\r
+> - fputs (tag_str, output);\r
+> + gzputs (output, tag_str);\r
+> } else {\r
+> if (hex_encode (notmuch, tag_str,\r
+> &buffer, &buffer_size) != HEX_SUCCESS) {\r
+> @@ -91,12 +93,12 @@ database_dump_file (notmuch_database_t *notmuch, FILE *output,\r
+> tag_str);\r
+> return EXIT_FAILURE;\r
+> }\r
+> - fprintf (output, "+%s", buffer);\r
+> + gzprintf (output, "+%s", buffer);\r
+> }\r
+> }\r
+> \r
+> if (output_format == DUMP_FORMAT_SUP) {\r
+> - fputs (")\n", output);\r
+> + gzputs (output, ")\n");\r
+> } else {\r
+> if (make_boolean_term (notmuch, "id", message_id,\r
+> &buffer, &buffer_size)) {\r
+> @@ -104,7 +106,7 @@ database_dump_file (notmuch_database_t *notmuch, FILE *output,\r
+> message_id, strerror (errno));\r
+> return EXIT_FAILURE;\r
+> }\r
+> - fprintf (output, " -- %s\n", buffer);\r
+> + gzprintf (output, " -- %s\n", buffer);\r
+> }\r
+> \r
+> notmuch_message_destroy (message);\r
+> @@ -121,24 +123,33 @@ database_dump_file (notmuch_database_t *notmuch, FILE *output,\r
+> int\r
+> notmuch_database_dump (notmuch_database_t *notmuch,\r
+> const char *output_file_name,\r
+> - const char *query_str, dump_format_t output_format)\r
+> + const char *query_str,\r
+> + dump_format_t output_format,\r
+> + notmuch_bool_t gzip_output)\r
+> {\r
+> - FILE *output = stdout;\r
+> + gzFile output;\r
+> + const char *mode = gzip_output ? "w9" : "wT";\r
+> +\r
+> int ret;\r
+> \r
+> if (output_file_name) {\r
+> - output = fopen (output_file_name, "w");\r
+> + output = gzopen (output_file_name, mode);\r
+> if (output == NULL) {\r
+> fprintf (stderr, "Error opening %s for writing: %s\n",\r
+> output_file_name, strerror (errno));\r
+> return EXIT_FAILURE;\r
+> }\r
+> + } else {\r
+> + output = gzdopen (fileno (stdout), mode);\r
+> }\r
+> \r
+> ret = database_dump_file (notmuch, output, query_str, output_format);\r
+> \r
+> - if (output != stdout)\r
+> - fclose (output);\r
+> + /* XXX check error return */\r
+> + gzflush (output, Z_FINISH);\r
+> +\r
+> + if (output_file_name)\r
+> + gzclose_w (output);\r
+> \r
+> return ret;\r
+> }\r
+> @@ -158,6 +169,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])\r
+> int opt_index;\r
+> \r
+> int output_format = DUMP_FORMAT_BATCH_TAG;\r
+> + notmuch_bool_t gzip_output = 0;\r
+> \r
+> notmuch_opt_desc_t options[] = {\r
+> { NOTMUCH_OPT_KEYWORD, &output_format, "format", 'f',\r
+> @@ -165,6 +177,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])\r
+> { "batch-tag", DUMP_FORMAT_BATCH_TAG },\r
+> { 0, 0 } } },\r
+> { NOTMUCH_OPT_STRING, &output_file_name, "output", 'o', 0 },\r
+> + { NOTMUCH_OPT_BOOLEAN, &gzip_output, "gzip", 'z', 0 },\r
+> { 0, 0, 0, 0, 0 }\r
+> };\r
+> \r
+> @@ -181,7 +194,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[])\r
+> }\r
+> \r
+> ret = notmuch_database_dump (notmuch, output_file_name, query_str,\r
+> - output_format);\r
+> + output_format, gzip_output);\r
+> \r
+> notmuch_database_destroy (notmuch);\r
+> \r
+> -- \r
+> 1.9.0\r
+>\r
+> _______________________________________________\r
+> notmuch mailing list\r
+> notmuch@notmuchmail.org\r
+> http://notmuchmail.org/mailman/listinfo/notmuch\r
+\r
+-- \r
+Jani\r
projects / notmuch-archives.git / commitdiff