0

I need get checksum Adler32 and store to variable in bash. It will be used in automatic script and it will be useful if no additional app/liberty will used which user need to install. Is it possible to use common / basic command Bash command to get this value?

errorfree
  • 103
  • 1
  • 1
  • 8
  • Calculating the checksum using a bash only solution would be very inefficient; reading _one byte once_ is required, as bash strings cannot hold null bytes. – M. Nejat Aydin Mar 23 '22 at 16:58

3 Answers3

1

This is monumentally slow (about 60,000 times slower than C), but it shows that yes, it is possible.

#!/bin/bash
sum1=1
sum2=0
while LANG=C IFS= read -r -d '' -n 1 ch; do
  printf -v val '%d\n' "'$ch"
  (( val = val < 0 ? val + 256 : val, sum1 = (sum1 + val) % 65521, sum2 = (sum2 + sum1) % 65521 ))
done
(( adler = sum1 + 65536 * sum2 ))
echo $adler

Hopefully someone who actually knows bash could vastly improve on this.

Mark Adler
  • 101,978
  • 13
  • 118
  • 158
  • I think `val = val < 0 ? val + 256 : val` is not required. For instance, `LANG=C IFS= read -r -d '' -n 1 ch < <(printf '\0xff'); printf '%d\n' "'$ch"` prints out `255`, not `-1`. But that won't help much for its slowness (in bash). – M. Nejat Aydin Mar 24 '22 at 08:46
  • ok, so if this problem can not be solve in standard bash shell. What solution will you propse? What solution will you propose? Is it possible to add some lib or extension and use some dedicated pack for that? – errorfree Mar 24 '22 at 10:34
  • On my bash I get -1, not 255. So I do need the conversion to unsigned. – Mark Adler Mar 24 '22 at 16:21
  • @errorfree I used bash to test that, so I don't know what's not standard. – Mark Adler Mar 24 '22 at 16:25
1

Maybe this solution?:

python -c "import zlib; print(zlib.adler32(\"${file}\"))"
errorfree
  • 103
  • 1
  • 1
  • 8
  • You said no additional things to install (though I don't know what "liberty" means). But if you know that python won't need to be installed, then that is a _way_ better solution than trying to do it in bash. – Mark Adler Mar 24 '22 at 16:27
  • yes, you are right. Bash is not best way for this case. So, I need use python. – errorfree Mar 25 '22 at 13:19
1

Tried two adler bash functions one with an ordination dictionary and one with printf also tried some bit shifting like instead of sum1=(sum1+val)%65521 -> temp= (sum1+val),sum1=temp >> 16 *15 + (temp & 65355)%65521 wasn't able to improve it a lot, perhaps somebody knows a faster one.

last function is a awk function, it is the fastest, works also on files.

#!/bin/bash

            a=$'Hello World'; b=""
            for ((i=0;i<1000;i++)); do b+=$a; done

            #-- building associative array ord byte character array
            declare -Ai ordCHAR=()
            for ((i=1;i<256;i++)); do printf -v hex "%x" $i; printf -v char "\x"$hex; ordCHAR[$char]=$i; done
            unset hex char i
            #-- building associative array ord byte character array -- END

            #-- with dictionary
            function adler32_A ()
                {
                local char; local -i sum1=1 sum2=0 val
                LC_ALL=C; while read -rN 1 char; do
                val=${ordCHAR[$char]};
                ((sum1=(sum1+val) % 65521, sum2 = (sum2 + sum1) % 65521 ))
                done <<< $1

                #-- removing 0A=\n addition, because of here string
                (( sum2-=sum1, sum2<0 ? sum2+=65521 :0, sum1-=val, sum1<0 ? sum1+=65521 :0 ));
                printf "%08x" $(( (sum2 << 16) + sum1 )) 
                LC_ALL=""
                }

            #-- with printf
            function adler32_B ()
                {
                local char; local -i sum1=1 sum2=0 val
                LC_ALL=C; while read -rN 1 char; 
                    do
                    printf -v val '%d' "'$char"
                    (( sum1 = (sum1 + val) % 65521, sum2 = (sum2 + sum1) % 65521 ))
                    done <<< $1
                #-- removing 0A=\n addition, because of here string
                (( sum2-=sum1, sum2<0 ? sum2+=65521 :0, sum1-=val, sum1<0 ? sum1+=65521 :0 ));
                printf "%x" $((sum1 + 65536 * sum2 ))
                LC_ALL=""
                }

            #-- call adler32_awk [text STR] [evaluate text as path bINT]
            function adler32_awk ()
                {
                local -i bPath=$2;
                awk -b \
            '   BEGIN {RS="^$"; bPath='"$bPath"'; for (i=0;i<256;i++) charOrdARR[sprintf("%c",i)]=i; A=1; B=0;}
                {
                recordSTR=substr($0,1,length($0)-1); if (bPath) {getline byte_data < recordSTR; close(recordSTR);} else byte_data=recordSTR;
                l=length(byte_data); for (i=1;i<=l;i++) {
                    A+=charOrdARR[substr(byte_data,i,1)]; if (A>65520) A-=65521;
                    B+=A; if (B>65520) B-=65521;}
                printf "%x", lshift(B,16)+A; }
            ' <<<$1
                }
            time adler32_A "$b"
            time adler32_B "$b"
            #-- adler 32 of file -> adler32_awk "/home/.../your file" 1
            time adler32_awk "$b"
Schmaehgrunza
  • 39
  • 6