How to get py-scrypt's "simple password verifier" example functions to work?

Question

I am using the example script provide by py-scrypt to build a simple password verifier. Below is my test script.

Test Script:

#!/usr/bin/python3
# -*- coding: utf-8 -*-

import scrypt
import os

def hash2_password(a_secret_message, password, maxtime=0.5, datalength=64):
    #return scrypt.encrypt(a_secret_message, password, maxtime=maxtime)
    return scrypt.encrypt(os.urandom(datalength), password, maxtime=maxtime)

def verify2_password(data, password, maxtime=0.5):
    try:
        secret_message = scrypt.decrypt(data, password, maxtime)
        print('\nDecrypted secret message:', secret_message)
        return True
    except scrypt.error:
        return False


password2 = 'Baymax'
secret_message2 = "Go Go"
data2 = hash2_password(secret_message2, password2, maxtime=0.1, datalength=64)
print('\nEncrypted secret message2:')
print(data2)

password_ok = verify2_password(data2, password2, maxtime=0.1)
print('\npassword_ok? :', password_ok)

Issues: I often get an error messages, e.g.:

Traceback (most recent call last):
  File "~/example_scrypt_v1.py", line 56, in <module>
    password_ok = verify2_password(data2, password2, maxtime=0.1)
  File "~/example_scrypt_v1.py", line 43, in verify2_password
    secret_message = scrypt.decrypt(data, password, maxtime)
  File "~/.local/lib/python3.5/site-packages/scrypt/scrypt.py", line 188, in decrypt
    return str(out_bytes, encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xca in position 0: invalid continuation byte

where the last lines varies to e.g.:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaf in position 3: invalid start byte

or

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xee in position 1: invalid continuation byte

or no error message but return False

password_ok? : False

When I comment return scrypt.encrypt(os.urandom(datalength), password, maxtime=maxtime) to remove the random secret message generator and uncomment return scrypt.encrypt(a_secret_message, password, maxtime=maxtime) to use a non-random secret message, the function verify2_password works.

Question: How do I get the random secret message element to work? What is causing it's failure?

Answer 1

Explanation for UnicodeDecodeError Exception

Reason 1

I think I understand why Scrypt is issuing a UnicodeDecodeError. Quoting Python's UnicodeDecodeError :

The UnicodeDecodeError normally happens when decoding an str string from a certain coding. Since codings map only a limited number of str strings to unicode characters, an illegal sequence of str characters will cause the coding-specific decode() to fail.

Also in Python's Unicode HOWTO section Python’s Unicode Support --> The String Type, it writes

In addition, one can create a string using the decode() method of bytes. This method takes an encoding argument, such as UTF-8, and optionally an errors argument

The errors argument specifies the response when the input string can’t be converted according to the encoding’s rules. Legal values for this argument are 'strict' (raise a UnicodeDecodeError exception), 'replace' (use U+FFFD, REPLACEMENT CHARACTER), 'ignore' (just leave the character out of the Unicode result), or 'backslashreplace' (inserts a \xNN escape sequence).

In short, whenever Python's .decode() method fails to map str strings to unicode characters, and when it uses the strict argument, the .decode() method will return a UnicodeDecodeError exception.

I tried to find the .decode() method in the .decrypt() method of py-scrypt/scrypt/scrypt.py. Initially, I could not locate it. For Python3, the .decrypt() method return statement was: return str(out_bytes, encoding)

However, further checking Python's explanation on the str class, I found the explanation saying that:

if object is a bytes (or bytearray) object, then str(bytes, encoding, errors) is equivalent to bytes.decode(encoding, errors).

This meant that without defining the error argument in str(bytes, encoding), this str class defaulted to returning bytes.decode(encoding, errors='strict') and returned the UnicodeDecodeError exception whenever it failed to map str strings to unicode characters.

Reason 2

In the "simple password verifier" example, the input argument of Scrypt.encrypt() was defined as os.urandom(datalength) which returned a <class 'bytes'>. When this <class 'bytes'> was encrypted, and subsequently decrypted by Scrypt.decrypt(), the returned decrypted value must also be a <class 'bytes'> . According to the doc_string of the .decrypt() method, for Python3 this method will return a str instance if encoded with encoding. If encoding=None, it will return a bytes instance. As Script.decrypt() defaults to encoding='utf-8' in function verify2_password(), Script.decrypt() attempts to return a <class str> resulted in the UnicodeDecodeError.

Solution to the "simple password verifier" example script given in py-scrypt:

1. The verify_password() function should contain the argument encoding=None .
2. scrypt.decrypt() should contain the argument encoding=encoding .

Revised Example Script:

#!/usr/bin/python3
# -*- coding: utf-8 -*-

import scrypt
import os

def encrypt_password(password, maxtime=0.5, datalength=64):
    passphrase = os.urandom(datalength)
    print('\npassphrase = ', passphrase, type(passphrase))
    return scrypt.encrypt(passphrase, password, maxtime=maxtime)

def verify_password(encrpyted_passphrase, password, maxtime=0.5, encoding=None):
    try:
        passphrase = scrypt.decrypt(encrpyted_passphrase, password, maxtime,
                                    encoding=encoding)
        print('\npassphrase = ', passphrase, type(passphrase))
        return True
    except scrypt.error:
        return False


password = 'Baymax'
encrypted_passphrase = encrypt_password(password, maxtime=0.5, datalength=64)
print('\nEncrypted PassPhrase:')
print(encrypted_passphrase)

password_ok = verify_password(encrypted_passphrase, password, maxtime=0.5)
print('\npassword_ok? :', password_ok)