--- /dev/null
+Return-Path: <amdragon@mit.edu>\r
+X-Original-To: notmuch@notmuchmail.org\r
+Delivered-To: notmuch@notmuchmail.org\r
+Received: from localhost (localhost [127.0.0.1])\r
+ by olra.theworths.org (Postfix) with ESMTP id 1DAFF431FBF\r
+ for <notmuch@notmuchmail.org>; Wed, 2 Apr 2014 13:43:54 -0700 (PDT)\r
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org\r
+X-Spam-Flag: NO\r
+X-Spam-Score: -0.7\r
+X-Spam-Level: \r
+X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5\r
+ tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled\r
+Received: from olra.theworths.org ([127.0.0.1])\r
+ by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)\r
+ with ESMTP id HNJ+rtZaPRll for <notmuch@notmuchmail.org>;\r
+ Wed, 2 Apr 2014 13:43:46 -0700 (PDT)\r
+Received: from dmz-mailsec-scanner-1.mit.edu (dmz-mailsec-scanner-1.mit.edu\r
+ [18.9.25.12])\r
+ (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))\r
+ (No client certificate requested)\r
+ by olra.theworths.org (Postfix) with ESMTPS id 4043E431FAE\r
+ for <notmuch@notmuchmail.org>; Wed, 2 Apr 2014 13:43:46 -0700 (PDT)\r
+X-AuditID: 1209190c-f794a6d000000c27-bf-533c767fe4f3\r
+Received: from mailhub-auth-1.mit.edu ( [18.9.21.35])\r
+ (using TLS with cipher AES256-SHA (256/256 bits))\r
+ (Client did not present a certificate)\r
+ by dmz-mailsec-scanner-1.mit.edu (Symantec Messaging Gateway) with SMTP\r
+ id 76.64.03111.F767C335; Wed, 2 Apr 2014 16:43:43 -0400 (EDT)\r
+Received: from outgoing.mit.edu (outgoing-auth-1.mit.edu [18.9.28.11])\r
+ by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id s32KhffS031181; \r
+ Wed, 2 Apr 2014 16:43:42 -0400\r
+Received: from awakening.csail.mit.edu (awakening.csail.mit.edu [18.26.4.91])\r
+ (authenticated bits=0)\r
+ (User authenticated as amdragon@ATHENA.MIT.EDU)\r
+ by outgoing.mit.edu (8.13.8/8.12.4) with ESMTP id s32Khc8G004820\r
+ (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT);\r
+ Wed, 2 Apr 2014 16:43:40 -0400\r
+Received: from amthrax by awakening.csail.mit.edu with local (Exim 4.80)\r
+ (envelope-from <amdragon@mit.edu>)\r
+ id 1WVS0Y-0001dl-H8; Wed, 02 Apr 2014 16:43:38 -0400\r
+Date: Wed, 2 Apr 2014 16:43:38 -0400\r
+From: Austin Clements <amdragon@MIT.EDU>\r
+To: Tomi Ollila <tomi.ollila@iki.fi>\r
+Subject: Re: [Patch v5 3/6] util: add gz_readline\r
+Message-ID: <20140402204337.GA4678@mit.edu>\r
+References: <1396401381-18128-1-git-send-email-david@tethera.net>\r
+ <1396401381-18128-4-git-send-email-david@tethera.net>\r
+ <20140402032644.GB25677@mit.edu>\r
+ <m2wqf73co4.fsf@guru.guru-group.fi>\r
+MIME-Version: 1.0\r
+Content-Type: text/plain; charset=us-ascii\r
+Content-Disposition: inline\r
+In-Reply-To: <m2wqf73co4.fsf@guru.guru-group.fi>\r
+User-Agent: Mutt/1.5.21 (2010-09-15)\r
+X-Brightmail-Tracker:\r
+ H4sIAAAAAAAAA+NgFprNKsWRmVeSWpSXmKPExsUixCmqrFtfZhNs0LdD0+JGazejxfWbM5kt\r
+ 3qycx+rA7HH460IWj2erbjF7bDn0njmAOYrLJiU1J7MstUjfLoEr487U2WwFTcYVL5o2MjYw\r
+ PlPrYuTkkBAwkVj+rJ0ZwhaTuHBvPVsXIxeHkMBsJoktT79AORsYJa72nIVyTjFJzLm9kxXC\r
+ WcIocej4N3aQfhYBFYlHL5awgdhsAhoS2/YvZwSxRYDiD9rWs4LYzAJ2Eke+d4HFhQUMJf4v\r
+ e8YCYvMKaEss7JkIteEQo8TG82dYIRKCEidnPmGBaNaSuPHvJVMXIweQLS2x/B8HSJhTwECi\r
+ 4+UbsJmiQLumnNzGNoFRaBaS7llIumchdC9gZF7FKJuSW6Wbm5iZU5yarFucnJiXl1qka6iX\r
+ m1mil5pSuokRHO6SPDsY3xxUOsQowMGoxMNrIWUTLMSaWFZcmXuIUZKDSUmUN7cEKMSXlJ9S\r
+ mZFYnBFfVJqTWgz0BwezkgivShRQjjclsbIqtSgfJiXNwaIkzvvW2ipYSCA9sSQ1OzW1ILUI\r
+ JivDwaEkwdtQCtQoWJSanlqRlplTgpBm4uAEGc4DNHw2yGLe4oLE3OLMdIj8KUZdjk8b1jQy\r
+ CbHk5eelSonzXgApEgApyijNg5sDS1OvGMWB3hLmDQNZxwNMcXCTXgEtYQJawr3OCmRJSSJC\r
+ SqqB0WGvU9qefdUfr2s6bu0VXRR8bVHym4ozxd/Nn58SdmdZqC/86KzQR8N/c59yXnvNODdc\r
+ RPaj3cwtbClqtxvdUmYFlbrWrdrJE8gsuCwqIb2x9UjIullNDh6PPr5+tzvNuufeoa1nyu4w\r
+ Wv7fE7hU783tLGNek3M/34T/WKvAIzyxrWE6f9gtfiWW4oxEQy3mouJEAHDnz1IuAwAA\r
+Cc: notmuch@notmuchmail.org\r
+X-BeenThere: notmuch@notmuchmail.org\r
+X-Mailman-Version: 2.1.13\r
+Precedence: list\r
+List-Id: "Use and development of the notmuch mail system."\r
+ <notmuch.notmuchmail.org>\r
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>\r
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>\r
+List-Post: <mailto:notmuch@notmuchmail.org>\r
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>\r
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,\r
+ <mailto:notmuch-request@notmuchmail.org?subject=subscribe>\r
+X-List-Received-Date: Wed, 02 Apr 2014 20:43:54 -0000\r
+\r
+Quoth Tomi Ollila on Apr 02 at 7:43 pm:\r
+> On Wed, Apr 02 2014, Austin Clements <amdragon@MIT.EDU> wrote:\r
+> \r
+> > Quoth David Bremner on Apr 01 at 10:16 pm:\r
+> >> The idea is to provide a more or less drop in replacement for readline\r
+> >> to read from zlib/gzip streams. Take the opportunity to replace\r
+> >> malloc with talloc.\r
+> >> ---\r
+> >> util/Makefile.local | 2 +-\r
+> >> util/util.h | 12 +++++++++\r
+> >> util/zlib-extra.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++\r
+> >> util/zlib-extra.h | 11 ++++++++\r
+> >> 4 files changed, 100 insertions(+), 1 deletion(-)\r
+> >> create mode 100644 util/util.h\r
+> >> create mode 100644 util/zlib-extra.c\r
+> >> create mode 100644 util/zlib-extra.h\r
+> >> \r
+> >> diff --git a/util/Makefile.local b/util/Makefile.local\r
+> >> index 29c0ce6..e2a5b65 100644\r
+> >> --- a/util/Makefile.local\r
+> >> +++ b/util/Makefile.local\r
+> >> @@ -4,7 +4,7 @@ dir := util\r
+> >> extra_cflags += -I$(srcdir)/$(dir)\r
+> >> \r
+> >> libutil_c_srcs := $(dir)/xutil.c $(dir)/error_util.c $(dir)/hex-escape.c \\r
+> >> - $(dir)/string-util.c $(dir)/talloc-extra.c\r
+> >> + $(dir)/string-util.c $(dir)/talloc-extra.c $(dir)/zlib-extra.c\r
+> >> \r
+> >> libutil_modules := $(libutil_c_srcs:.c=.o)\r
+> >> \r
+> >> diff --git a/util/util.h b/util/util.h\r
+> >> new file mode 100644\r
+> >> index 0000000..8663cfc\r
+> >> --- /dev/null\r
+> >> +++ b/util/util.h\r
+> >> @@ -0,0 +1,12 @@\r
+> >> +#ifndef _UTIL_H\r
+> >> +#define _UTIL_H\r
+> >> +\r
+> >> +typedef enum util_status {\r
+> >> + UTIL_SUCCESS = 0,\r
+> >> + UTIL_ERROR = 1,\r
+> >> + UTIL_OUT_OF_MEMORY,\r
+> >> + UTIL_EOF,\r
+> >> + UTIL_FILE,\r
+> >> +} util_status_t;\r
+> >> +\r
+> >> +#endif\r
+> >> diff --git a/util/zlib-extra.c b/util/zlib-extra.c\r
+> >> new file mode 100644\r
+> >> index 0000000..cb1eba0\r
+> >> --- /dev/null\r
+> >> +++ b/util/zlib-extra.c\r
+> >> @@ -0,0 +1,76 @@\r
+> >> +/* zlib-extra.c - Extra or enhanced routines for compressed I/O.\r
+> >> + *\r
+> >> + * Copyright (c) 2014 David Bremner\r
+> >> + *\r
+> >> + * This program is free software: you can redistribute it and/or modify\r
+> >> + * it under the terms of the GNU General Public License as published by\r
+> >> + * the Free Software Foundation, either version 3 of the License, or\r
+> >> + * (at your option) any later version.\r
+> >> + *\r
+> >> + * This program is distributed in the hope that it will be useful,\r
+> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of\r
+> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\r
+> >> + * GNU General Public License for more details.\r
+> >> + *\r
+> >> + * You should have received a copy of the GNU General Public License\r
+> >> + * along with this program. If not, see http://www.gnu.org/licenses/ .\r
+> >> + *\r
+> >> + * Author: David Bremner <david@tethera.net>\r
+> >> + */\r
+> >> +\r
+> >> +#include "zlib-extra.h"\r
+> >> +#include <talloc.h>\r
+> >> +#include <stdio.h>\r
+> >> +#include <string.h>\r
+> >> +\r
+> >> +/* mimic POSIX/glibc getline, but on a zlib gzFile stream, and using talloc */\r
+> >> +util_status_t\r
+> >> +gz_getline (void *talloc_ctx, char **bufptr, size_t *bufsiz, ssize_t *bytes_read,\r
+> >\r
+> > Talloc chunks know their size, so rather than taking bufsize, use\r
+> > talloc_get_size (or talloc_array_length if you switch to talloc array\r
+> > functions below).\r
+> \r
+> Now yoy David have a chance to drop the bufsiz argument altogether, as the\r
+> info is available in *bufptr:s talloc context...\r
+> \r
+> >\r
+> >> + gzFile stream)\r
+> >> +{\r
+> >> + size_t len = *bufsiz;\r
+> >> + char *buf = *bufptr;\r
+> >> + size_t offset = 0;\r
+> >> +\r
+> >> + if (len == 0 || buf == NULL) {\r
+> >> + /* same as getdelim from gnulib */\r
+> >> + len = 120;\r
+> >\r
+> > This is presumably because glibc's malloc has an 8 byte header. Fun\r
+> > fact: talloc has a 104 byte header (on 64-bit and including the malloc\r
+> > header).\r
+> \r
+> hmm, what should we choose here? 152 ? Some bikeshedding on IRC ?\r
+\r
+How about we bikeshed about not bikeshedding about this?\r
+\r
+> >> + buf = talloc_size (talloc_ctx, len);\r
+> >> + if (buf == NULL)\r
+> >> + return UTIL_OUT_OF_MEMORY;\r
+> >> + }\r
+> >> +\r
+> >> + while (1) {\r
+> >> + if (! gzgets (stream, buf + offset, len - offset)) {\r
+> >> + int zlib_status = 0;\r
+> >> + (void) gzerror (stream, &zlib_status);\r
+> >> + switch (zlib_status) {\r
+> >> + case Z_OK:\r
+> >> + /* follow getline behaviour */\r
+> >> + *bytes_read = -1;\r
+> >\r
+> > Is this really what getline does when the last line of a file isn't\r
+> > \n-terminated?\r
+> \r
+> Maybe the previous call returned non-\n -terminated string and\r
+> for this call there was 0 bytes left to return ???\r
+\r
+But my point is that the previous call *won't* return a\r
+non-\n-terminated string. If my file looks like "a\nb\nc", this will\r
+return "a\n", then "b\n", and then fail (unless I'm following the code\r
+wrong). This is *not* what getline does (the manpage is confusing,\r
+but I just tested it).\r
+\r
+> Tomi\r
+> \r
+> >> + return UTIL_EOF;\r
+> >> + break;\r
+\r
+Unnecessary break.\r
+\r
+> >> + case Z_ERRNO:\r
+> >> + return UTIL_FILE;\r
+> >> + break;\r
+\r
+And here.\r
+\r
+> >> + default:\r
+> >> + return UTIL_ERROR;\r
+> >> + }\r
+> >> + }\r
+> >> +\r
+> >> + offset += strlen (buf + offset);\r
+> >> +\r
+> >> + if ( buf[offset - 1] == '\n' )\r
+> >\r
+> > Too many spaces!\r
+> >\r
+> >> + break;\r
+> >> +\r
+> >> + len *= 2;\r
+> >> + buf = talloc_realloc (talloc_ctx, buf, char, len);\r
+> >\r
+> > Or talloc_realloc_size, to match the initial talloc_size.\r
+> > Alternatively, the initial talloc_size could be a talloc_array.\r
+> >\r
+> >> + if (buf == NULL)\r
+> >> + return UTIL_OUT_OF_MEMORY;\r
+> >> + }\r
+> >> +\r
+> >> + *bufptr = buf;\r
+> >> + *bufsiz = len;\r
+> >> + *bytes_read = offset;\r
+> >> + return UTIL_SUCCESS;\r
+> >> +}\r
+> >> diff --git a/util/zlib-extra.h b/util/zlib-extra.h\r
+> >> new file mode 100644\r
+> >> index 0000000..ed46ac1\r
+> >> --- /dev/null\r
+> >> +++ b/util/zlib-extra.h\r
+> >> @@ -0,0 +1,11 @@\r
+> >> +#ifndef _ZLIB_EXTRA_H\r
+> >> +#define _ZLIB_EXTRA_H\r
+> >> +\r
+> >> +#include <zlib.h>\r
+> >> +#include "util.h"\r
+> >\r
+> > I'd put "util.h" first so we're more likely to catch missing header\r
+> > dependencies (obviously util.h doesn't have any right now, but in the\r
+> > future).\r
+> >\r
+> > Also, I'd put a blank line after the #includes.\r
+> >\r
+> >> +/* Like getline, but read from a gzFile. Allocation is with talloc */\r
+> >> +util_status_t\r
+> >> +gz_getline (void *ctx, char **lineptr, size_t *line_size, ssize_t *bytes_read,\r
+> >> + gzFile stream);\r
+> >> +\r
+> >> +#endif\r
projects / notmuch-archives.git / commitdiff