rnddata digest issue

Question

I tried to generate a big file for digest. When I used below openssl rand 8096 | head -c 8096 > rnddata, it was ok.

./hash_test sha1
Digest is: a0a4dab4a16d247fb625ca2abf07373d2123e4a5
openssl dgst -sha1 rnddata
SHA1(rnddata)= a0a4dab4a16d247fb625ca2abf07373d2123e4a5

But the digest result was different when the rnddata's size is equal to or greater then 9000.

openssl rand 9000 | head -c 9000 > rnddata
openssl dgst -sha1 rnddata
SHA1(rnddata)= 4ef5b73ee39f0f2ac6d341c55467c182786cab65
./hash_test sha1
Digest is: 691747a97aee32ddf04b3bf5644b5d74c4b69657

Below is my c code for test

#include <stdio.h>
#include <openssl/evp.h>

main(int argc, char *argv[])
{
    EVP_MD_CTX mdctx;
    const EVP_MD *md;
    FILE *fr;
    size_t bytes;
    char mystring[32];

    fr = fopen("/home/root/rnddata", "r");
    if(fr == NULL) 
        printf("Error opening file\n");

    unsigned char md_value[EVP_MAX_MD_SIZE];
    int md_len, i; 

    OpenSSL_add_all_digests();
    //OpenSSL_add_all_algorithms();
    if(!argv[1]) {
        printf("Usage: mdtest digestname\n");
        exit(1);
    }

    //md = EVP_get_digestbyname(argv[1]);
    md = EVP_sha1();
    if(!md) {
        printf("Unknown message digest %s\n", argv[1]);
        exit(1);
    }

    EVP_MD_CTX_init(&mdctx);
    EVP_DigestInit_ex(&mdctx, md, NULL);
    while ((bytes = fread (mystring, 1, 32, fr)) != 0)
        EVP_DigestUpdate(&mdctx, mystring, bytes);
    EVP_DigestFinal_ex(&mdctx, md_value, &md_len);
    EVP_MD_CTX_cleanup(&mdctx);

    printf("Digest is: ");
    for(i = 0; i < md_len; i++)
        printf("%02x", md_value[i]);
    printf("\n");
    fclose(fr);
}

Any help is appreciated.

LSerni · Answer 1 · 2012-10-05T22:22:37.650

0

I have tested your code, and it works as it should:

for i in $( seq 1 100 ); do
    openssl rand 9000 | head -c 9000 > test
    YOUR=$( ./sha1 test | cut -b 12- )
    SHA1=$( openssl dgst -sha1 test | cut -b 13- )
    if [ "$YOUR" == "$SHA1" ]; then
         echo $YOUR OK
    else
         echo $YOUR $SHA1 FAIL
    fi
done

Also works with lengths 8000, 9100, 10000.

Tested your second code, this too appears to work correctly. This is the modified version I wrote to test on different files by supplying the name on command line:

#include <stdio.h>
#include <openssl/sha.h>

int main(int argc, char **argv)
{
    unsigned char c[SHA_DIGEST_LENGTH];
    int i;

    FILE *inFile = fopen (argv[1], "rb");
    SHA_CTX shaContext;
    int bytes;
    unsigned char data[1024];

    if (inFile == NULL) {
            printf ("%s can't be opened.\n", argv[1]);
            return 0;
    }

    SHA1_Init (&shaContext);

    while ((bytes = fread (data, 1, 1024, inFile)) != 0)
    {
            printf("Read %d bytes\n", bytes);
            SHA1_Update (&shaContext, data, bytes);
    }
    SHA1_Final (c,&shaContext);

    for(i = 0; i < SHA_DIGEST_LENGTH; i++)
            printf("%02x", c[i]);

    printf (" %s\n", argv[1]);
    fclose (inFile);

    return 0;
}

There must be something in how the code is compiled that yields the wrong results.

I checked with gcc version 4.7.1 20120723 [gcc-4_7-branch revision 189773] (SUSE Linux) on kernel 3.4.6 x86_64.

I have modified the code above to print some information about its operation. I tested it with a file made up of 9000 zeroes (dd if=/dev/zero of=/tmp/zeroes bs=1 count=9000) and got:

Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 808 bytes
980eb7886d6d6ce206b9ce5371042d32c96d2714 /tmp/zeroes

If block sizes do match, then we'll try printing the sum of all bytes (which ought to be always zero), or check the buffer one byte at a time.

If you have the same SHA1 as me, we'll try with a sequence of 9000 0x80's.

edited Oct 05 '12 at 22:22

answered Oct 04 '12 at 16:54

LSerni

55,617
10
65
107

Thanks.But why is some lengths different? – gary Oct 05 '12 at 01:49
Sorry, I do not understand. I tried different lengths to see what happened. For each length I chose, all files were of that length, and they checked out correctly. – LSerni Oct 05 '12 at 12:33
root@am335x-evm:~# sh test.sh e5f9b87b7171d4975b1503e901435736e99a3c6503 )= 9de5f6df2d0345433e8c65c3567b40df6f2eb363 FAIL e5f9b87b7171d4975b1503e901435736e99a3c6503 )= 9de5f6df2d0345433e8c65c3567b40df6f2eb363 FAIL – gary Oct 05 '12 at 19:52
You can see my testing result at the bottom. – gary Oct 05 '12 at 20:03
I was using Cross-compiler, and ran it in my embedded Linux system. – gary Oct 05 '12 at 21:41
[linux-devkit]:~/crypto> arm-arago-linux-gnueabi-gcc --version arm-arago-linux-gnueabi-gcc (GCC) 4.5.3 20110311 (prerelease) And my embedded Linux version is Linux version 3.2.0 – gary Oct 05 '12 at 21:57
Sorry for delay. I have tested it with using your code. The result is the same as yours. – gary Oct 08 '12 at 20:26
...now that I come to think of it: your code reads the file `/home/root/rnddata`. You *did* stay in `/home/root` (instead of, say, `/root` or `/private/root`) when creating the `rnddata` file, did you? Otherwise you tested *two different files*... – LSerni Oct 08 '12 at 22:25
Yes, I did created the rnddata in /home/root/ – gary Oct 09 '12 at 02:03

score 0 · Answer 2 · answered Oct 05 '12 at 20:03

I have tested these two code as well as command line. Below is my testing result.

root@am335x-evm:~# openssl rand 9000 | head -c 9000 > rnddata
root@am335x-evm:~# ./hash_test sha1
Digest is: 691747a97aee32ddf04b3bf5644b5d74c4b69657
root@am335x-evm:~# ./sha1_test 
691747a97aee32ddf04b3bf5644b5d74c4b69657 rnddata
root@am335x-evm:~# openssl dgst -sha1 rnddata
SHA1(rnddata)= 4ef5b73ee39f0f2ac6d341c55467c182786cab65

score 0 · Answer 3 · answered Oct 10 '12 at 15:39

I have tested my code on Ubuntu 10.04. The result is same between command line and C code.

gary@ubuntu:~/crypto$ gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.4.3-4ubuntu5' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-plugin --enable-objc-gc --enable-targets=all --disable-werror --with-arch-32=i486 --with-tune=generic --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.4.3 (Ubuntu 4.4.3-4ubuntu5) 


gary@ubuntu:/usr/local/ssl/bin$ ./openssl 
OpenSSL> version
OpenSSL 1.0.0a 1 Jun 2010
OpenSSL> ^C
gary@ubuntu:/usr/local/ssl/bin$ ./openssl 
OpenSSL> dgst -sha1 /home/gary/crypto/rnddata
SHA1(/home/gary/crypto/rnddata)= 34c7dd77ce6fbc56875bca2bd916f42464b9bab0



gcc sha1_test.c -o h1 -I/usr/local/ssl/include -L/usr/local/ssl/lib -l

gary@ubuntu:~/crypto$ ./h1 rnddata 
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 1024 bytes
Read 808 bytes
34c7dd77ce6fbc56875bca2bd916f42464b9bab0 rnddata

rnddata digest issue

3 Answers3