JWK and python cryptography package - can't reproduce matching pub key

Question

I have a JWK generated on a test website:

key = {
        "kty": "EC",
        "d": "MXrxKTl_o9yIQlExYy9c1LcWZX_OwX3aw-oGP0flUdo",
        "use": "sig",
        "crv": "secp256k1",
        "kid": "Im53aoD8zJoHzOXmfIAUkncONCIeR1pgy_nhvQrwN3s",
        "x": "hHXNLbjBY_SFeP-tOPoyoGGYjISm-m3aVJLpc3suka0",
        "y": "yYIjrvo_lqrsdxq-oMQQxBG8eyIUKmF9XazdwdGTwSY",
        "alg": "ES256"
    }

I should convert this into PEM formatting, with python:

curve = ec.SECP256R1()
signature_algorithm = ec.ECDSA(hashes.SHA256())
padding_factor = (4 - len(key['d']) % 4) % 4
padded_secret = key['d']+ '='*padding_factor
secret_bytes = base64.urlsafe_b64decode(padded_secret)
secret_int =  int.from_bytes(secret_bytes, 'big')
priv_key = ec.derive_private_key(secret_int, curve, default_backend())
pem_priv = priv_key.private_bytes(
        encoding=serialization.Encoding.PEM,
        format=serialization.PrivateFormat.TraditionalOpenSSL,
        encryption_algorithm=serialization.NoEncryption()
)

At this point I test:

pub_key = pem_priv.public_key()
x = (pub_key.public_numbers().x)
x_bytes = x.to_bytes(32, byteorder="big")
x_encoded = base64.urlsafe_b64encode(number_bytes)
self.assertTrue(key["x"]==x_encoded.decode())

This fails.