Why are signatures created with ecdsa Python library not valid with coincurve?

Question

I'm switching from the pure Python ecdsa library to the much faster coincurve library for signing data. I would also like to switch to coincurve for verifying the signatures (including the old signatures created by the ecdsa library).

It appears that signatures created with ecdsa are not (always?) valid in coincurve. Could someone please explain why this is not working? Also, it seems that cryptography library is able to validate both ecdsa signatures and coincurve signatures without issues, consistently.

What is even more confusing, if you run below script a few times, is that sometimes it prints point 3 and other times it does not. Why would coincurve only occasionally find the signature valid?

pip install ecdsa cryptography coincurve

import ecdsa
import hashlib
import coincurve
from coincurve.ecdsa import deserialize_compact, cdata_to_der
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives.asymmetric.utils import Prehashed

ecdsa_private_key = ecdsa.SigningKey.generate(ecdsa.SECP256k1, None, hashlib.sha256)
ecdsa_pub = ecdsa_private_key.get_verifying_key()
message = b"Hello world!"
digest = hashlib.sha256(message).digest()
serialized_signature = ecdsa_private_key.sign_digest_deterministic(digest, hashfunc=hashlib.sha256)

signature = cdata_to_der(deserialize_compact(serialized_signature))
cc_private_key = coincurve.PrivateKey(ecdsa_private_key.to_string())
cc_pub = cc_private_key.public_key
crypto_pub = ec.EllipticCurvePublicKey.from_encoded_point(ec.SECP256K1(), cc_pub.format(True))

if ecdsa_pub.verify_digest(serialized_signature, digest) is True:
    print("1. ecdsa can validate its own signature")

crypto_pub.verify(signature, digest, ec.ECDSA(Prehashed(hashes.SHA256())))
print("2. cryptography can validate ecdsa signature (raises exception if not valid)")

if cc_pub.verify(signature, digest, None) is False:
    print("3. coincurve will not validate ecdsa signature")

signature = cc_private_key.sign(digest, None)

crypto_pub.verify(signature, digest, ec.ECDSA(Prehashed(hashes.SHA256())))
print("4. cryptography can validate coincurve signature (raises exception if not valid)")

if cc_pub.verify(signature, digest, None) is True:
    print("5. coincurve will validate its own signature")

This may or may not be the problem, but you really shouldn't be writing "is True" or "is False". You should be `if ecdsa_pub.verify_digest(...):` or `if not cc.pub.verify(...):` and the like. Unless a Python function is documented to return precisely `True` or `False`, it may return something that behaves like false (0, None, []) or something that behaves like True (everything else). — Frank Yellin, Dec 25 '21 at 06:35

Topaco · Accepted Answer · 2021-12-25T14:41:21.860

Bitcoin and the coincurve library use canonical signatures while this is not true for the ecdsa library.

What does canonical signature mean?
In general, if (r,s) is a valid signature, then (r,s') := (r,-s mod n) is also a valid signature (n is the order of the base point).
A canonical signature uses the value s' = -s mod n = n - s instead of s, i.e. the signature (r, n-s), if s > n/2, s. e.g. here.

All signatures from the ecdsa library that were not been successfully validated by the coincurve library in your test program have an s > n/2 and thus are not canonical, whereas those that were successfully validated are canonical.

So the fix is simply to canonize the signature of the ecdsa library, e.g.:

def canonize(s_bytes):
  n = 115792089237316195423570985008687907852837564279074904382605163141518161494337
  s = int.from_bytes(s_bytes, byteorder='big')
  if s > n//2:
    s = n - s
  return s.to_bytes(32, byteorder='big')

...  
serialized_signature = serialized_signature[:32] + canonize(serialized_signature[32:]) # Fix: canonize
signature = cdata_to_der(deserialize_compact(serialized_signature))
...

With this fix, the coincurve library successfully validates all signatures from the ecdsa library in your test program.

Turns out `coincurve` had the proper function to normalize the signature. This was my solution thanks to your insight: https://github.com/lbryio/lbry-sdk/blob/42cf4b44a218177781002f9f8b81f1a7ef2e8fb9/lbry/wallet/bip32.py#L159-L167 — Lex Berezhny, Jan 01 '22 at 19:38

Why are signatures created with ecdsa Python library not valid with coincurve?

1 Answers1