jose encryption returns binary string but decrypting with decoded string gives error in python

Question

I am using jose for python jwe encryption.

Here is my code from the example

import jose
from time import time
from Crypto.PublicKey import RSA

key = RSA.generate(2048)

claims = {'name': 'Jack'}

pub_jwk = {'k': key.publickey().exportKey('PEM')}

jwe = jose.encrypt(claims, pub_jwk)
jwt = jose.serialize_compact(jwe)

Here the value of jwt is binary string

b'eyJlbmMiOiAiQTEyOENCQy1IUzI1NiIsICJhbGciOiAiUlNBLU9BRVAiLCAiX192IjogMn0.N1RFIEaRIGxCgSkT8HhQI4bO66XOL2RVfn4tMu8BBfGBO79AFKzHUYIRuVqpBX9YcUrsn66n3ccH5O2HO-CuCEPZ6EBM47IBUW1NAdFnm4uc3_X3EAngGTe2hnkLp0RzByYUcaLp2bMn7TWptRmvDGrADaI3uliZCV_ahLeWWFySFjIm_LaLBUzH1okZ-uPqvKQRXDEsdmBSTH5KlsQZHOdRa6uZz_iILmZY6Pp-9XtOSldTLiGasIA_9DNfljP5UtImOhAax_piA7hHeacGAtBNZJVZCWZZajLI6HKz5hVs4aZy7I2EIK6ogL0ubBNMeCQ0dZ70SWjvBTcTbtV2jw.65XaQ1rCSIn25Gc73CJe0g.oo81kAasMwPTISH5XEnnY5Mym3PPXMVs-FtYwgboHUE.5TR5Au7A7JYU7x0iYoPhGQ'

When I decrypt with the same jwt value it gives me exact result.

enc = jose.decrypt(jose.deserialize_compact(jwt), priv_jwk)

Here when I try to make json of encrypted value

 data = {'jwt': jwt}
 json.dumps(data)

It gives me error Object of type 'bytes' is not JSON serializable

I can decode the encryption like:

jwt = jose.serialize_compact(jwe)

but decrypting with this encryption value will raise error. I don't want to encode jwt in decrypting process.

Is there anyway I can get string instead of byte while encryption so that I can dump it to JSON.

Answer 1

First of all, the Demonware/JOSE project was written for Python 2 originally, you are presumably using the Python 3 branch. There are unresolved issues with the implementation, which to me speaks of the package author not actually understanding the issue very well. JWT tokens, in their compact serialization are just a series of URL-safe Base64 strings concatenated with . characters.

Whenever you encrypt or sign a new token, you have to decode bytes to strings (just decode the bytes value as ASCII). When verifying or decrypting, you'll need to encode the strings to bytes again.

E.g. to encode the jwt value in a JSON object, you'll need to decode:

data = {'jwt': jwt.decode('ascii')}

The whole point of a Javascript Web Token (JWT) is to be exchanged with another party as text. The library compounds matters by claiming the method returns a string.

You could shorten this use the default utf-8 codec, as ASCII is a subset of UTF-8:

data = {'jwt': jwt.decode()}

In the opposite direction, you'll have to encode the compact string to bytes again:

data = json.loads(json_document)
jose.decrypt(jose.deserialize_compact(data['jwt'].encode()), priv_jwk)

But you are basically using outdated software; the Demonware/jose project has not been updated in 3 years time. It also relies on the outdated, unmaintained pycrypto package. You do not want to use.

Instead, take a look at Authlib or JWCrypto, two modules that are actively maintained, and use the cryptography project to handle the tricky cryptography primitives (there is also pyjwt and python-jose, but those projects do not (yet) support JWE encryption, only JWS signed tokens).

Of these, Authlib offers by far the cleanest and clearest API. E.g. to both generate a public / private key pair and create your encrypted token with Authlib, using the default encryption and signing algorithm chosen by Demonware/JOSE, you'd do:

from authlib.jose import jwt
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa

# generate a private key with corresponding public key, then
# export the keys as a PEM serializations.
private_key = rsa.generate_private_key(
    public_exponent=65537,
    key_size=2048,
    backend=default_backend()
)
private_key_pem = private_key.private_bytes(
    serialization.Encoding.PEM,
    serialization.PrivateFormat.PKCS8,
    serialization.NoEncryption()
)
public_key = private_key.public_key()
public_key_pem = public_key.public_bytes(
    serialization.Encoding.PEM,
    serialization.PublicFormat.SubjectPublicKeyInfo
)

# create an encrypted token (JWT using JWE)
claims = {'name': 'Jack'}
header = {'enc': 'A128CBC-HS256', 'alg': 'RSA-OAEP'}
token = jwt.encode(header, claims, public_key_pem).decode()

# decrypt the token again
claims_from_token = jwt.decode(token.encode(), private_key_pem)

Note that Authlib too returns a bytes value here, so you'd have to decode and encode this too if you wanted to further embed this in JSON and pass the token data from a JSON payload back to the jwt.decode() method.

Using the same public and private key serializations with JWCrypto:

import json
from jwcrypto import jwt, jwk

# Create a JWK key object from the private key
jwk_key = jwk.JWK.from_pem(private_key_pem)

# create a JWT() instance to handle ecryption and serialization
header = {'enc': 'A128CBC-HS256', 'alg': 'RSA-OAEP'}
jwt_token = jwt.JWT(claims, header)
jwt_token.make_encrypted_token(jwk_key)
token = jwt_token.serialize()

# deserialize again, using a new JWT instance but with different arguments;
# Yes, this is a confusing API.
# it's always a good idea to limit what algorithms you'll accept
# note that this is a list of both *signing* and *encryption* algorithms, not
# the possible values of the "alg" key in a JOSE header.
jwt_token = jwt.JWT(key=jwk_key, jwt=token, algs=['A128CBC-HS256', 'RSA-OAEP'])
claims_from_token = json.loads(jwt_token.claims)

Note that this library does return a string when serializing, but you have to do the JSON decoding of the claims manually. The API is also mixing serializing and de-serializing into one class, creating a very confusing mix of arguments, methods, and properties.