Why does the same HSM key verify to a different ethereum address every time?

Question

I am interfacing with an HSM which is generating and signing with the ethereum standard (secp256k1). I am interfacing with the HSM using a package called Graphene. I pull out the public key using its "pointEC" attribute: 0xc87c1d67c1909ebf8b54c9ce3d8e0f0cde41561c8115481321e45b364a8f3334b6e826363d8e895110fc9ca2d75e84cc7c56b8e9fbcd70c726cb44f5506848fa

Which I can use to generate the address: 0x21d20b04719f25d2ba0c68e851bb64fa570a9465

But when I try to use the key to sign a personal message from a dApp, the signature always evaluates to a different address. For example, the nonce/message: wAMqcOCD2KKz2n0Dfbu1nRYbeLw_qbLxrW1gpTBwkq Has the signature:

0x2413f8d2ab4df2f3d87560493f21f0dfd570dc61136c53c236731bf56a9ce02cb23692e6a5cec96c62359f6eb4080d80328a567d14387f487f3c50d9ce61503b1c

But it recovers a valid address of 0xFC0561D848b0cDE5877068D94a4d803A0a933785

This is all presumably with the same private/public key. Granted, I merely appended the "1c" recovery value, but even when I attempt with other values I have no luck. Here's a couple more examples:

Nonce: WRH_ApTkfN7yFAEpbGwU9BiE2M6eKTZMklPYK50djnx
Sig: 0x70242adabfe27c12e54abced8de87b45f511a194609eb27b215b153594b5697b7fb5e7279285663f80c82c2a2f2920916f76fd845cdecb45ace19f76b0622ac41c
Address: 0x1A086eD40FF90E75764260E2Eb42fab4Db519E53

Nonce: TZV6qhplddJgcKaN7qtpcIhudFhiQ
Sig: 0x3607beb3d58ff35ca1059f3ea44f41e79e76d8ffe35a4f716e78030f0fe2ca1da51f138c31d4ec4b9fc3546c4de1185736a4c4c7030a8b1965e30cb0af6ba2ee1c
Address: 0xa61A518cf73163Fd92461942c26C67c203bda379

My code to sign the message:

        let alg: graphene.MechanismType;
        alg = graphene.MechanismEnum.ECDSA;
        const session = get_session();

        let key: graphene.Key | null = null;
        //#region Find signing key
        const objects = session.find({label: GEN_KEY_LABEL});
        for (let i = 0; i < objects.length; i++) {
            const obj = objects.items(i);
            if ((obj.class === graphene.ObjectClass.PRIVATE_KEY ||
                obj.class === graphene.ObjectClass.SECRET_KEY) &&
                obj.handle.toString('hex') == params.handle
            ) {
                key = obj.toType<graphene.Key>();
                break;
            }
        }
        if (!key) {
            throw new Error("Cannot find signing key");
        }
        var sign = session.createSign(alg,key);
        if (!params.data) {
            console.log("No data found. Signing 'test' string");
            params.data = 'test';
        }
        sign.update(Buffer.from(params.data.toString().trim()));
        var signature = sign.final();



        console.log(signature.toString('hex'));

Keep in mind, it fails with even just 1 key present.

You might need to share the code you're using to sign the message and the code you're using to recover the signer. — user94559, Apr 24 '19 at 15:02
Hard to say without seeing the code, but what you're doing with `web3` may be expecting the data that's signed to be first hashed with keccak256 and then prefixed with "\x19Ethereum Signed Message:32\n". It certainly doesn't look like you're doing that in your signing code. — user94559, Apr 24 '19 at 17:08
The data being signed is already in that format. All this does is send it off to the HSM to sign and return the signature. — Carlos Hernandez, Apr 24 '19 at 17:14
So what's the actual value of `params.data.toString()`? What exactly is being passed to `sign.update`? — user94559, Apr 24 '19 at 17:17
`Buffer.from(params.data.toString().trim())` may be mangling the data if what's in `params.data` is already something like a buffer because it's a hash. — user94559, Apr 24 '19 at 17:20
In general, it would help if you shared the rest of your relevant code (e.g. where you assign to `params.data`, where you hash the "nonce", prefix it, hash again, and then what you're passing to what function in `web3.js`). There's too much guessing involved in a process where it's critical that every byte be preserved properly. — user94559, Apr 24 '19 at 17:23
I think you're confused. Obviously the data isn't good because the correct address isn't coming back out. :-) A common reason for this is that you're telling the recovery code that you signed the message "foo" when you actually signed the message "bar". I'm asking for the details that are necessary for debugging this sort of issue. — user94559, Apr 24 '19 at 17:42
What I mean is that it the rest of it was working before, the code I posted is the only data manipulation happening it's the same whether I use a buffer or a string, whether i trim it or not, etc. — Carlos Hernandez, Apr 24 '19 at 18:41
I provided full source code for this issue in [Using AWS CloudHSM to sign transactions question](https://ethereum.stackexchange.com/questions/73192/using-aws-cloudhsm-to-sign-transactions) — Wazen Shbair, Mar 13 '20 at 08:31

Maarten Bodewes · Answer 1 · 2019-04-24T17:01:23.777

3

The address is just calculated over the public key, while the signature is generated using ECDSA. ECDSA which consists of a random value r and a signature s that is specific to that random (and, of course, the private key). More information here (Wikipedia on ECDSA).

You don't see this because they are simply encoded to a statically sized (unsigned, big integer) values and then concatenated together to be called "the signature" (hence the size of the signature being twice that of the key size, 64 bytes instead of 32 bytes). Verification will parse the signature and use the separate values again. With ethereum and BitCoin an additional byte may be prefixed to the signature so that it is possible to retrieve back the public key and then recalculate the address. This also alters the signature generation so you're not talking plain ECDSA anymore.

There is also the X9.62 signature format, which still does consists of two separate integers, encoded using ASN.1 / DER encoding. Those signatures only look partially random because of the overhead required to separate / encode the two integers.

edited Apr 24 '19 at 17:01

answered Apr 24 '19 at 15:40

Maarten Bodewes

90,524
13
150
263

How would I sign using the correct value for the address generated from the public key? – Carlos Hernandez Apr 24 '19 at 15:47
Um, I'd have to look that up :| Not a blockchain expert, just a crypto expert that just looked up the other day how addresses are calculated. That may take some time. Note though that signatures are not compared directly, you simply *verify* them for correctness. – Maarten Bodewes Apr 24 '19 at 15:47
I just don't understand how to sign in a way that the signature will verify to the generated address. Can the public key be taken from the signature? – Carlos Hernandez Apr 24 '19 at 15:54
See the answer here: https://crypto.stackexchange.com/q/18105/1172. Note the comment under the second answer: "It turned out that in my case (Bitcoin message signature) they append an extra byte to the signature in order to identify which of the two points is the public key. Once this is done they calculate a 160bit hash of the public key. This hash is known as a bitcoin address. Then they check if the generated address is equal with the given one and if it is the message is authenticated. By using this way of doing they win some bytes. – Jan Moritz Jul 15 '14 at 6:04" – Maarten Bodewes Apr 24 '19 at 16:16
Note that the signature format has a byte prefixed to it: see more info [here](https://bitcoin.stackexchange.com/a/14265/29414). [This](https://stackoverflow.com/a/20400041/589259) seems to be an implementation of that link. I presume that each signature will generate one of the possible public key formats, which you then have to alter to get the org. key back. – Maarten Bodewes Apr 24 '19 at 16:28
I assume that's what the recovery code's doing, however, it seems that it's recovering a different public key ever time, as it spits out an address that should've come from the public key. – Carlos Hernandez Apr 24 '19 at 16:29
You're just prefixing a value 28, while the least significant bit of that determines what kind of conversion you need to do on the altered ECDSA signature (and actually, the value of 27 needs to be subtracted or added, not 28). Hence you will not fix this without fixing both the signature generation function *and* the key extraction method. – Maarten Bodewes Apr 24 '19 at 20:30

score 0 · Accepted Answer · answered Jul 25 '19 at 19:06

Turns out I was using a deprecated Buffer.from function, as the updated version requires you to specify the format of the incoming data.

E.g. Buffer.from("04021a","hex")

Since it was the final 'input' and calculation, it took me forever to realize that the data was being incorrectly transformed at that point. I thought I had checked and rechecked the data in every step multiple times, but missed the most in-your-face part.

Also, I learned that to create a proper signature and prevent transaction malleability, you have to keep resigning so that the value of 's' ends up being less than: (0xfffffffffffffffffffffffffffffffebaaedce6af48a03bbfd25e8cd0364141)/2

Then when putting 'r' and 's' through a address-recovery function, it should try to recover the address with v=27 or v=28 (0x1a or 0x1b), basically at this point it's trial and error. Most of the time, it'll recover the correct address with v=27.

can you share more information. I am using proper buffer.from, but i am facing this issue. I am using Graphene and facing the exact same problem. This is the way, I am finding rs and v value. ethTx.r = sig.slice(0, 32) ethTx.s = sig.slice(32, 64) ethTx.v = chainId ? recovery + (chainId * 2 + 35) : recovery + 27; — user2805885, Dec 20 '19 at 07:07
If you get s>n/2 you don't need to compute a new signature, just replace it with n-s where n is the curve-group order; see https://ethereum.stackexchange.com/questions/55245/why-is-s-in-transaction-signature-limited-to-n-21 . Whether v=27 or 28 is correct (and recovers your publickey) depends on your publickey and should be half and half for a fair distribution; how many did you try? — dave_thompson_085, Sep 02 '22 at 05:50

Why does the same HSM key verify to a different ethereum address every time?

2 Answers2