From: Tomi Ollila <tomi.ollila@iki.fi>
Date: Sun, 24 Nov 2013 16:42:46 +0000 (+0200)
Subject: Re: notmuch sha1 implementation broken on (some) big-endian architectures
X-Git-Url: http://git.tremily.us/?a=commitdiff_plain;h=86813cc019ae18cf05abfe70c7bcb2b6b88dd2a2;p=notmuch-archives.git

Re: notmuch sha1 implementation broken on (some) big-endian architectures
---

diff --git a/28/0f92a7c51b4351d045d417cc801a25cb0ad542 b/28/0f92a7c51b4351d045d417cc801a25cb0ad542
new file mode 100644
index 000000000..b806685b8
--- /dev/null
+++ b/28/0f92a7c51b4351d045d417cc801a25cb0ad542
@@ -0,0 +1,105 @@
+Return-Path: <tomi.ollila@iki.fi>
+X-Original-To: notmuch@notmuchmail.org
+Delivered-To: notmuch@notmuchmail.org
+Received: from localhost (localhost [127.0.0.1])
+	by olra.theworths.org (Postfix) with ESMTP id 09E6D431FDF
+	for <notmuch@notmuchmail.org>; Sun, 24 Nov 2013 08:43:04 -0800 (PST)
+X-Virus-Scanned: Debian amavisd-new at olra.theworths.org
+X-Spam-Flag: NO
+X-Spam-Score: 0
+X-Spam-Level: 
+X-Spam-Status: No, score=0 tagged_above=-999 required=5 tests=[none]
+	autolearn=disabled
+Received: from olra.theworths.org ([127.0.0.1])
+	by localhost (olra.theworths.org [127.0.0.1]) (amavisd-new, port 10024)
+	with ESMTP id dpsQUGyQC-+m for <notmuch@notmuchmail.org>;
+	Sun, 24 Nov 2013 08:42:56 -0800 (PST)
+Received: from guru.guru-group.fi (guru.guru-group.fi [46.183.73.34])
+	by olra.theworths.org (Postfix) with ESMTP id 66466431FDD
+	for <notmuch@notmuchmail.org>; Sun, 24 Nov 2013 08:42:56 -0800 (PST)
+Received: from guru.guru-group.fi (localhost [IPv6:::1])
+	by guru.guru-group.fi (Postfix) with ESMTP id CB64A10005E;
+	Sun, 24 Nov 2013 18:42:46 +0200 (EET)
+From: Tomi Ollila <tomi.ollila@iki.fi>
+To: David Bremner <david@tethera.net>, notmuch@notmuchmail.org
+Subject: Re: notmuch sha1 implementation broken on (some) big-endian
+	architectures
+In-Reply-To: <87eh666mhp.fsf@zancas.localnet>
+References: <87eh666mhp.fsf@zancas.localnet>
+User-Agent: Notmuch/0.16+175~g19e97d6 (http://notmuchmail.org) Emacs/24.3.1
+	(x86_64-unknown-linux-gnu)
+X-Face: HhBM'cA~<r"^Xv\KRN0P{vn'Y"Kd;zg_y3S[4)KSN~s?O\"QPoL
+	$[Xv_BD:i/F$WiEWax}R(MPS`^UaptOGD`*/=@\1lKoVa9tnrg0TW?"r7aRtgk[F
+	!)g;OY^,BjTbr)Np:%c_o'jj,Z
+Date: Sun, 24 Nov 2013 18:42:46 +0200
+Message-ID: <m2ob593g55.fsf@guru.guru-group.fi>
+MIME-Version: 1.0
+Content-Type: text/plain
+X-BeenThere: notmuch@notmuchmail.org
+X-Mailman-Version: 2.1.13
+Precedence: list
+List-Id: "Use and development of the notmuch mail system."
+	<notmuch.notmuchmail.org>
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>
+List-Post: <mailto:notmuch@notmuchmail.org>
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
+X-List-Received-Date: Sun, 24 Nov 2013 16:43:04 -0000
+
+On Sun, Nov 24 2013, David Bremner <david@tethera.net> wrote:
+
+> The following code, when linked with libnotmuch.a and libutil.a does a
+> passable imitation of sha1sum on amd64 (and I guess also i386) but
+> computes a different digest on powerpc and probably sparc and s390x.
+>
+> In the long run we should maybe outsource hash computations to
+> e.g. librhash, but I'd like a simpler fix for 0.17, if possible
+>
+> P.S. I blame Austin for adding the "missing-headers" test which found
+> this bug ;).
+
+This is interesting problem, I would have guessed that this would
+fails on LITTLE_ENDIAN machines easier, if ever...
+
+... especially as there is line
+
+lib/libsha1.c:52:#if (PLATFORM_BYTE_ORDER == IS_LITTLE_ENDIAN)
+
+... but...
+
+I could not find any (other) matches for PLATFORM_BYTE_ORDER nor
+IS_LITTLE_ENDIAN in source code or in /usr/include/**/*.h or
+in /usr/lib/gcc/**/*.h
+
+I did some testing and it seems that 
+#if (FOOOO == BBBAAARRR)
+<code>
+#endif
+
+will have <code> in output file in case neither of the above are
+defined... :/
+
+So, this could work:
+
+#if <endian.h>  // for BYTE_ORDER && LITTLE_ENDIAN
+
+and then
+
+#if (BYTE_ORDER == LITTLE_ENDIAN)
+...
+
+to replace lib/libsha1.c:53 (53 now after endian.h added)
+
+
+Please test on BIG_ENDIAN machine...
+
+
+In case this works, then we'd need to inform users that their
+long/missing Message ID:s are now coded differently in their
+databases...
+
+
+Tomi