avoid possible overflow in delta size filtering computation

On a 32-bit system, the maximum possible size for an object is less than
4GB, while 64-bit systems may cope with larger objects.  Due to this
limitation, variables holding object sizes are using an unsigned long
type (32 bits on 32-bit systems, or 64 bits on 64-bit systems).

When large objects are encountered, and/or people play with large delta
depth values, it is possible for the maximum allowed delta size
computation to overflow, especially on a 32-bit system.  When this
occurs, surviving result bits may represent a value much smaller than
what it is supposed to be, or even zero.  This prevents some objects
from being deltified although they do get deltified when a smaller depth
limit is used.  Fix this by always performing a 64-bit multiplication.

Signed-off-by: Nicolas Pitre <nico@cam.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff --git a/builtin-pack-objects.c b/builtin-pack-objects.c
index fb5e14d56e5b9fec0a054eb9708be41a59ac0f13..84a13c7ccd4c04df2d45b7130ef7005643e42cb6 100644 (file)
--- a/builtin-pack-objects.c
+++ b/builtin-pack-objects.c
@@ -1265,7 +1265,7 @@ static int try_delta(struct unpacked *trg, struct unpacked *src,
                max_size = trg_entry->delta_size;
                ref_depth = trg->depth;
        }
-       max_size = max_size * (max_depth - src->depth) /
+       max_size = (uint64_t)max_size * (max_depth - src->depth) /
                                                (max_depth - ref_depth + 1);
        if (max_size == 0)
                return 0;