From 6c9c74edc2b91ca67ec7726cab0810b0987cb11d Mon Sep 17 00:00:00 2001 From: Austin Clements Date: Wed, 8 Aug 2012 09:57:26 +2000 Subject: [PATCH] Re: [PATCH 1/4] show: indicate length of omitted body content (json) --- 91/264092644dafe4ec08f0c42b01980fc8a98a92 | 171 ++++++++++++++++++++++ 1 file changed, 171 insertions(+) create mode 100644 91/264092644dafe4ec08f0c42b01980fc8a98a92 diff --git a/91/264092644dafe4ec08f0c42b01980fc8a98a92 b/91/264092644dafe4ec08f0c42b01980fc8a98a92 new file mode 100644 index 000000000..109d2bd9e --- /dev/null +++ b/91/264092644dafe4ec08f0c42b01980fc8a98a92 @@ -0,0 +1,171 @@ +Return-Path: +X-Original-To: notmuch@notmuchmail.org +Delivered-To: notmuch@notmuchmail.org +Received: from localhost (localhost [127.0.0.1]) + by olra.theworths.org (Postfix) with ESMTP id 4A0D0431FC0 + for ; Tue, 7 Aug 2012 06:57:35 -0700 (PDT) +X-Virus-Scanned: Debian amavisd-new at olra.theworths.org +X-Spam-Flag: NO +X-Spam-Score: -0.7 +X-Spam-Level: +X-Spam-Status: No, score=-0.7 tagged_above=-999 required=5 + tests=[RCVD_IN_DNSWL_LOW=-0.7] autolearn=disabled +Received: from olra.theworths.org ([127.0.0.1]) + by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024) + with ESMTP id yjMv52OfeAe8 for ; + Tue, 7 Aug 2012 06:57:34 -0700 (PDT) +Received: from dmz-mailsec-scanner-5.mit.edu (DMZ-MAILSEC-SCANNER-5.MIT.EDU + [18.7.68.34]) + by olra.theworths.org (Postfix) with ESMTP id 50531431FAF + for ; Tue, 7 Aug 2012 06:57:34 -0700 (PDT) +X-AuditID: 12074422-b7f1f6d00000090b-9e-50211ecd0f4b +Received: from mailhub-3.mit.edu ( [18.9.21.44]) + by dmz-mailsec-scanner-5.mit.edu (Symantec Messaging Gateway) with SMTP + id AC.4E.02315.DCE11205; Tue, 7 Aug 2012 09:57:33 -0400 (EDT) +Received: from outgoing-legacy.mit.edu (OUTGOING-LEGACY.MIT.EDU [18.7.22.104]) + by mailhub-3.mit.edu (8.13.8/8.9.2) with ESMTP id q77DvXll015115; + Tue, 7 Aug 2012 09:57:33 -0400 +Received: from webmail-10.mit.edu (WEBMAIL-10.MIT.EDU [18.9.23.20]) ) + by outgoing-legacy.mit.edu (8.13.6/8.12.4) with ESMTP id q77E0Nvd003893; + Tue, 7 Aug 2012 10:00:24 -0400 (EDT) +Received: from webmail-10.mit.edu (webmail-10.mit.edu [127.0.0.1]) by + webmail-10.mit.edu (8.13.8) with ESMTP + id q77DvQOn013771; Tue, 7 Aug 2012 09:57:26 -0400 +Received: (from nobody@localhost) + by webmail-10.mit.edu (8.13.8/8.13.8/Submit) id q77DvQu5013770; + Tue, 7 Aug 2012 09:57:26 -0400 +X-Authentication-Warning: webmail-10.mit.edu: nobody set sender to + amdragon@mit.edu using -f +Received: from 209-6-116-242.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com + (209-6-116-242.c3-0.arl-ubr1.sbo-arl.ma.cable.rcn.com [209.6.116.242]) + (User authenticated as amdragon@ATHENA.MIT.EDU) by webmail.mit.edu + (Horde MIME library) with HTTP; Tue, 07 Aug 2012 09:57:26 -0400 +Message-ID: <20120807095726.o066wknhk4sk804k@webmail.mit.edu> +Date: Tue, 07 Aug 2012 09:57:26 -0400 +From: Austin Clements +To: Peter Wang +Subject: Re: [PATCH 1/4] show: indicate length of omitted body content (json) +References: <1344151345-25411-1-git-send-email-novalazy@gmail.com> + <20120806164710.GK22601@mit.edu> + <20120807232414.GA22132@hili.localdomain> +In-Reply-To: <20120807232414.GA22132@hili.localdomain> +MIME-Version: 1.0 +Content-Type: text/plain; + charset=UTF-8; + format="flowed" +Content-Disposition: inline +Content-Transfer-Encoding: 7bit +User-Agent: Internet Messaging Program (IMP) H3 (4.0.3) +X-Brightmail-Tracker: + H4sIAAAAAAAAA+NgFjrEKsWRmVeSWpSXmKPExsUixCmqo3tWTjHAoHOvqcX1mzOZLZ637mVy + YPLYOesuu8ezVbeYA5iiuGxSUnMyy1KL9O0SuDKedl1nKVgvU3Hv1SXmBsa9Yl2MnBwSAiYS + j7s7WCBsMYkL99azdTFycQgJ7GSU2DzpG5SzmVHi5NU1THCZzoenGSGcBYwS30/8YgfpFxJo + YZRYebMUYlaMRNOeY8wQRVOZJBbufwdWxCtgK/Hr0EpWEJtFQFVixbTbTCA2m4CGxLb9yxlB + bBEBZYk/v56xgdjMAtIS3343g9UIC/hLND1sZYEY2s8o0bfkNdjlnAJmEp/+fGOEWCAocXLm + ExaIZhuJEz+2Ai3jABu0/B8HRFheYvvbOcwgtqiAucSDvTsYJzCKzULSPQtJ9yyE7llIuhcw + sqxilE3JrdLNTczMKU5N1i1OTszLSy3SNdXLzSzRS00p3cQIiip2F6UdjD8PKh1iFOBgVOLh + vcClECDEmlhWXJl7iFGSg0lJlLdERDFAiC8pP6UyI7E4I76oNCe1+BCjBAezkgjv4Z1A5bwp + iZVVqUX5MClpDhYlcd5rKTf9hQTSE0tSs1NTC1KLYLIyHBxKErxiwOQhJFiUmp5akZaZU4KQ + ZuLgBBnOAzRcFqSGt7ggMbc4Mx0if4pRUUqclxUkIQCSyCjNg+uFJb1XjOJArwjz6oNU8QAT + Jlz3K6DBTECDveXlQAaXJCKkpBoY9T+Jxi9WyBZRN5K7/8SaySTLIri/eF1HmP2p63mNKmkG + jhaFr3ksXgS9fr9SJbxq/Z+nNoIGyi8ecHDu2NMy5Z1jLpNlmkhS7IGney4ppXEtnL1qIn/P + 0arZKacFN9dO2fdaO0j3rVh5DrvVuwwfli619C0e54psG2aqi127cfq82LfKk5JKLMUZiYZa + zEXFiQD1UuH6VQMAAA== +Cc: notmuch@notmuchmail.org +X-BeenThere: notmuch@notmuchmail.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: "Use and development of the notmuch mail system." + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +X-List-Received-Date: Tue, 07 Aug 2012 13:57:35 -0000 + +Quoting Peter Wang : +> On Mon, 6 Aug 2012 12:47:10 -0400, Austin Clements wrote: +>> What's the overall goal of adding this? Are you planning to add size +>> information to one of the frontends? +> +> Yes, to my frontend. +> +>>> > diff --git a/devel/schemata b/devel/schemata +>> > index 9cb25f5..3df2764 100644 +>> > --- a/devel/schemata +>> > +++ b/devel/schemata +>> > @@ -69,7 +69,10 @@ part = { +>> > # A leaf part's body content is optional, but may be included if +>> > # it can be correctly encoded as a string. Consumers should use +>> > # this in preference to fetching the part content separately. +>> > - content?: string +>> > + content?: string, +>> > + # If a leaf part's body content is not included, the content-length +>> > + # may be included instead. +>> +>> You mentioned elsewhere that the content-length returned is an +>> estimate. If that's the case, this comment should say as much. Is it +>> actually the case, though? g_mime_part_get_content_object is +>> remarkably poorly documented for such an important function, but based +>> on format_part_raw, it seems like the content-length your code returns +>> will be exactly the number of bytes returned by the raw format for a +>> leaf part. +> +> It's the exact length of the _encoded_ content. If the transfer +> encoding is base64, multiplying by 3/4 will get a close estimate of the +> decoded content length. I assume quoted-printable encoding would only +> be used if the content is mostly ASCII, so the encoded length can serve +> as the estimated decoded length then. + +Ah, I see. format_part_raw misled me; apparently the +g_mime_data_wrapper_write_to_stream is key there, since *that* decodes +the transfer encoding of the data wrapper's underlying, raw stream. + +In that case, the comment could either mention that this is the length +of the transfer encoded content or it could say it's an approximation +of the decoded length. The advantage of only claiming the latter is +that it would leave open the possibility of, say, multiplying by .75 +for base64 transfer encoding to get a better decoded estimate (your +assumption about quoted-printable sounds completely reasonable). +Alternatively, we could add the transfer encoding in the future and +let the caller do such approximations. + +>> > diff --git a/notmuch-show.c b/notmuch-show.c +>> > index 3556293..5c54257 100644 +>> > --- a/notmuch-show.c +>> > +++ b/notmuch-show.c +>> > @@ -664,6 +664,14 @@ format_part_json (const void *ctx, sprinter_t +>> *sp, mime_node_t *node, +>> > sp->map_key (sp, "content"); +>> > sp->string_len (sp, (char *) part_content->data, part_content->len); +>> > g_object_unref (stream_memory); +>> > + } else { +>> > + GMimeDataWrapper *wrapper = g_mime_part_get_content_object +>> (GMIME_PART (node->part)); +>> > + GMimeStream *stream = g_mime_data_wrapper_get_stream (wrapper); +>> > + ssize_t length = g_mime_stream_length (stream); +>> > + if (length >= 0) { +>> > + sp->map_key (sp, "content-length"); +>> > + sp->integer (sp, length); +>> > + } +>> +>> Do wrapper or stream need to be g_object_unref'd? +> +> No. +> +>> Any idea what the performance overhead of this is? I'm just curious. +>> It might be approximately nothing, since GMime's parser is eager. +> +> The start and end bounds of the stream are already known so there's +> approximately nothing for g_mime_stream_length to do. The other +> functions simply return field values. + +Sounds good. + +> I'll drop the changes for text output. +> +> Peter + -- 2.26.2