Why does different scrypt implementation produce different results?

Question

I implemented a simple gode for generating passwords deterministically using Go, using this scrypt implementation. My code is here, this is a an extraction of the relevant parts:

const keyLen = 8

// recommended cost parameters for interactive login in 2009
const N = 16384
const r = 8
const p = 1

func main() {    
    dk, _ := scrypt.Key([]byte("mypassword"), []byte("mysalt"), N, r, p, keyLen)
    for _, v := range dk {
        fmt.Printf("%x ", v)
    }

    fmt.Println()
}

Running the code I get the output:

19 e5 59 39 fe 1 2b ef

However when I tried C implementations based libscrypt I would NOT get the same output. I would get:

68 56 66 b2 86 19 2f 6e

The C code is here and this is the main part:

int N = 16384;
int r = 8;
int p = 1;
const int keyLen = 8;

int main(void) {
    unsigned char digest[keyLen];
    const unsigned char password[] = "mypassword";
    const unsigned char salt[] = "mysalt";

    libscrypt_scrypt(password, sizeof(password), salt, sizeof(salt), N, r, p, digest, keyLen);

    for (int i = 0; i < keyLen; ++i) {
        printf("%x ", digest[i]);
    }
    printf("\n");

    return 0;
}

I also came across yet another implementation scrypt-jane, which gave yet another result when hashing. Although I realise this is probably because scrypt jane allows you to chose which hashing function is used. I didn't use a random salt in any of the cases.

My own hunch which I am trying to verify here is that there is no clear standard on what hashing functions are used with scrypt. If so how can I make sure an implementation in Go, C, python or JavaScript all produce the same result? I rather avoid porting the code myself, as I understand that is risky business in cryptography.

http://tools.ietf.org/html/draft-josefsson-scrypt-kdf-01 has test vectors — CodesInChaos, May 09 '14 at 14:59
The most common primitives used in scrypt are PBKDF2-HMAC-SHA256 and Salsa20/8. — CodesInChaos, May 09 '14 at 15:00
from what I can tell skimming the source code both the libscrypt and golang version uses pbkdf2, sha256 and salsa20/8. So I don't understand the difference unless I have made a mistake in how the parameters are passed. — Erik Engheim, May 09 '14 at 16:44
It could be that strings in those two languages use different default text encodings (utf-8, etc). Check to see that the incoming password/salt byte-arrays match. — hunter, May 09 '14 at 17:56
I wrote code to check against the test vectors. That uncovered that the C program had to have a mistake. — Erik Engheim, May 09 '14 at 20:14

score 6 · Accepted Answer · answered May 09 '14 at 20:20

The C code that calls the libscrypt_scrypt method within the library contains two bugs, which causes the wrong results to be calculated. The offending parts are:

sizeof(password)
sizeof(salt)

The sizeof will include the termnating 0 at the end of the strings so instead of "mypassword" and "mysalt" getting length 10 and 6 respectively they get 11 and 7. Thus two extra 0's are included when calculating the hash.

So the problem can be fixed by either writing:

strlen((const char *)password)
strlen((const char *)salt)

or

sizeof(password) - 1
sizeof(salt) - 1

for the salt and password arguments to the scrypt hash function.

Why does different scrypt implementation produce different results?

1 Answers1