0

zlib/adler32.c code is:

#define BASE 65521U     /* largest prime smaller than 65536 */
#define NMAX 5552
/* NMAX is the largest n such that 255n(n+1)/2 + (n+1)(BASE-1) <= 2^32-1 */

but I think it should be calculated like this:

255n(n+1)/2 + n < 2^32 - 1

What is the purpose of adding (n+1)(BASE-1)?

minchai
  • 13
  • 3
  • Left as an exercise for the reader. – Mark Adler Apr 25 '22 at 04:46
  • Think about the assumptions behind your calculation. – Mark Adler Apr 26 '22 at 01:09
  • My understanding is that the cumulative sum cannot exceed the value range of the integer type (integer overflow). – minchai Apr 27 '22 at 07:05
  • True. What else did you assume? – Mark Adler Apr 28 '22 at 00:29
  • 1. Assume that each input byte is the maximum value (255): input = [255, 255, 255...] 2. Assume that adler and sum2 are the maximum values: adler = 65520; sum2 = 65520 – minchai Apr 28 '22 at 06:32
  • Those _are_ the right assumptions, but somehow you got the math wrong! I thought for sure you were making the common mistake of assuming initial values of 1 and 0 instead of 65520 and 65520, because if you incorrectly assume 1 and 0, you get _exactly_ your "should be calculated like this". I don't know how you got the math wrong with the correct assumptions. – Mark Adler Apr 30 '22 at 16:45

0 Answers0