Read Fernet Key Causes ValueError: Fernet key must be 32 url-safe base64-encoded bytes

Question

In this function I am trying to read a Fernet key from a file, or create one if the file doesn't contain a key.

from cryptography.fernet import Fernet
import csv


with open("Keys.txt","rU") as csvfile:
    reader=csv.reader(csvfile)
    KeyFound=0
    print(KeyFound)
    for row in reader:
        try:
            print(row[0])
        except IndexError:
            continue
        if len(row[0])>4:
            print("KEY FOUND")
            KeyFound=1
            print(KeyFound)
            Key=row[0]
            print(Key)
            print(KeyFound)
        else:
            pass
if KeyFound==0:
    Key = Fernet.generate_key()
    print(Key)
    print("Created Key")
    csvfile.close()
#Writing Key to textfile
with open("Keys.txt", "w+") as csvfile:
    headers = ['key']
    writer=csv.DictWriter(csvfile, fieldnames=headers)
    writer.writeheader()
    writer.writerow({'key': Key})
    csvfile.close()
print(Key)
Ecy = Fernet(Key)

I am having difficulty reading the file. When the file is read the key is read in as:

b'nNjpIl9Ax2LRtm-p6ryCRZ8lRsL0DtuY0f9JeAe2wG0='

Yet I receive this error:

ValueError: Fernet key must be 32 url-safe base64-encoded bytes.

In this line:

Ecy = Fernet(Key)

Any help would be appreciated.

Just executing `f = Fernet(b'nNjpIl9Ax2LRtm-p6ryCRZ8lRsL0DtuY0f9JeAe2wG0=')` works fine for me. — Reinier Torenbeek, Dec 22 '18 at 16:42
I'm sure it does work fine If you directly put the byte string into the command instead of reading it from the textfile. Must be something to do with the csv reader. — Lyra Orwell, Dec 22 '18 at 16:47
So is that byte string that you copied not exactly the output from `print(Key)`? — Reinier Torenbeek, Dec 22 '18 at 16:50
No it is but if you look at the error the key must not be in the correct format. — Lyra Orwell, Dec 22 '18 at 16:54

snakecharmerb · Accepted Answer · 2022-08-25T09:24:10.543

The problem here is how the key is being written to the file.

Fernet.generate_key() returns a bytes instance:

>>> key = Fernet.generate_key()
>>> key
b'ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg='

The key is being written to the file as is:

>>> with open('keys.csv', 'w+') as f:
...     headers = ['key']
...     writer = csv.DictWriter(f, fieldnames=headers)
...     writer.writeheader()
...     writer.writerow({'key': key})
... 
49
>>>

If we look in the file, we can see the contents aren't what we expect - the b that indicates a python bytestring has been written to the file:

$  cat keys.csv 
key
b'ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg='

csv.writer calls str on any values that aren't already strings. If str is called on a bytes instance you get the stringified repr of the bytes instances rather than the decoded value of the bytes instance, which is what you want^*.

>>> str(key)
"b'ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg='"  # <- note the extra quotes...
>>> key.decode('utf-8')
'ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg='

So the solution is to call the bytes instance's decode method before csv.writer receives it.

>>> with open('keys.csv', 'w+') as f:
...     headers = ['key']
...     writer = csv.DictWriter(f, fieldnames=headers)
...     writer.writeheader()
...     writer.writerow({'key': key.decode('utf-8')})
... 
46
>>>

This gives us the file contents that we want:

$  cat keys.csv 
key
ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg=

And the rest of the code works as expected:

>>> with open('keys.csv') as f:
...     reader = csv.reader(f)
...     next(reader)      # <- skip the header row
...     for row in reader:
...         csv_key = row[0]
...         print(Fernet(csv_key))
... 
['key']                   # <- the headers are printed as a side effect of skipping
<cryptography.fernet.Fernet object at 0x7f3ad62fd4e0>

One debugging tip. When using print() to debug your code, it's sometimes better to print an object's repr, rather than the result of calling str on the object (which is what print() does). This is especially the case if the object is a string. For example:

>>> bad_key = str(key)
>>> print(bad_key)                                
b'ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg='      # <- Looks ok...
>>> print(repr(bad_key))
"b'ZmDfcTF7_60GrrY167zsiPd67pEvs0aGOv2oasOM1Pg='"    # <- See the problem
>>> 
>>> good_str = 'foo'
>>> bad_str = 'foo '
>>> print(bad_str)
foo                             # <- looks like good_str
>>> print(repr(bad_str))
'foo '                          # <- see the trailing space

^* If you call Python with the -b flag - python -b myscript.py - Python will emit a BytesWarning he first time you try to call str on a bytes instance. If the -bb flag is used, an exception will be raised.

Thank you very much, I ill now add repr() to my debugging repotoire. — Lyra Orwell, Dec 24 '18 at 09:25

Read Fernet Key Causes ValueError: Fernet key must be 32 url-safe base64-encoded bytes

1 Answers1

Linked