Why does my ECDH shared secret generation sometimes fail?

Question

I'm working on Apple Pay's in-app Wallet Provisioning. I've made a test project that matches the test vectors I get from Apple's documentation (utilizing System.Security and BouncyCastle), so pretty sure I'm on the right track.

However, when I switch to my generated ephemeral key pair (rather than the given ones for testing), the private key often times causes an error while generating the ECDH shared secret. Oddly enough, it seems to work about half the time without error.

Here is the code for generating the ephemeral key pair:

ephemKeyPair = ECDiffieHellman.Create(CURVE); // System.Security.Cryptography.ECDiffieHellman
// save our public and private key bytes
ephemPublicKey = GetPublicKeyFromEncodedBytes(ephemKeyPair.PublicKey.ExportSubjectPublicKeyInfo());
ephemPrivateKey = ephemKeyPair.ExportParameters(true).D;
...
private static byte[] GetPublicKeyFromEncodedBytes(byte[] publicKeyEncoded)
{
     byte[] bytes = new byte[65];
     Array.Copy(publicKeyEncoded, 26, bytes, 0, bytes.Length);
     return bytes;
}

When I attempt to generate my shared secret (which is based off of the Apple public key + my ephemeral private key), it throws the error Scalar is not in the interval [1, n - 1] (Parameter 'd') about half the time. Here's the calling code:

GenerateSharedSecret(ephemPrivateKey /* generated above */, applePublicKey /* extracted from input, currently a test .pem file. this has been verified in the Apple docs*/);

and the code to generate the shared secret:

public void GenerateSharedSecret(byte[] privateKeyIn, byte[] publicKeyIn)
{
     ECDHCBasicAgreement agreement = new ECDHCBasicAgreement();
     X9ECParameters? curve = null;
     ECDomainParameters? ecParam = null;
     ECPrivateKeyParameters? privKey = null;
     ECPublicKeyParameters? pubKey = null;
     Org.BouncyCastle.Math.EC.ECPoint point;
     curve = NistNamedCurves.GetByName("P-256");
     ecParam = new ECDomainParameters(curve.Curve, curve.G, curve.N, curve.H, curve.GetSeed());
     BigInteger bigInt = new BigInteger(privateKeyIn);
     privKey = new ECPrivateKeyParameters(bigInt, ecParam); // *** ERROR HERE ***
     point = ecParam.Curve.DecodePoint(publicKeyIn);
     pubKey = new ECPublicKeyParameters(point, ecParam);

     agreement.Init(privKey);
     BigInteger secret = agreement.CalculateAgreement(pubKey);
     sharedSecret = secret.ToByteArrayUnsigned();
}

Am I generating the ephemeral keys improperly somehow or something else entirely?

**EDIT 3/15/23

Ok, so I rewrote portions of this based on the feedback of Maarten. Rather than using some System.Security objects and some BouncyCastle objects, I'm using all BC now and it made things much smoother. I haven't yet validated the efficacy of the encrypted data but I'm not receiving those errors any longer and it's more straightforward now.

Updated code:

private readonly X9ECParameters curve = NistNamedCurves.GetByName("P-256");
private ECDomainParameters ecParam { get; set; }
private ECPrivateKeyParameters privateKeyParameters { get; set; }
private AsymmetricCipherKeyPair ephemKeyPair { get; set; }
...
private void GenerateEphemeralKeyPair()
{
    ecParam = new ECDomainParameters(curve.Curve, curve.G, curve.N, curve.H, curve.GetSeed());
    var secureRandom = new SecureRandom();
    var keyParams = new ECKeyGenerationParameters(ecParam, secureRandom);
    var generator = new ECKeyPairGenerator("ECDH");
    generator.Init(keyParams);

    ephemKeyPair = generator.GenerateKeyPair();
    ECPublicKeyParameters publicKeyParameters = (ECPublicKeyParameters) 
    ephemKeyPair.Public;
    privateKeyParameters = (ECPrivateKeyParameters) ephemKeyPair.Private;
    ephemPublicKey = publicKeyParameters.Q.GetEncoded();
    ephemPrivateKey = privateKeyParameters.D.ToByteArrayUnsigned();
}
...
private void GenerateSharedSecret(byte[] publicKeyIn)
{
   Org.BouncyCastle.Math.EC.ECPoint point = ecParam.Curve.DecodePoint(publicKeyIn);
   ECPublicKeyParameters pubKey = new ECPublicKeyParameters(point, ecParam);

   ECDHCBasicAgreement agreement = new ECDHCBasicAgreement();
   agreement.Init(privateKeyParameters);

   BigInteger secret = agreement.CalculateAgreement(pubKey);

   sharedSecret = secret.ToByteArrayUnsigned();
   sharedSecretAsHex = Convert.ToHexString(sharedSecret);
}

Maarten Bodewes · Accepted Answer · 2023-03-14T16:21:41.167

0

The problem is here: BigInteger bigInt = new BigInteger(privateKeyIn);. This will result in a negative value about half the time. See the examples on how to fix this by adding a zero byte at the end (.NET is mostly little endian).

Beware that BigInteger assumes little endian. You may first have to reverse the byte order. This is for instance the documentation for the D parameter on .NET:

Represents the private key D for the elliptic curve cryptography (ECC) algorithm, stored in big-endian format.

edited Mar 14 '23 at 16:21

answered Mar 14 '23 at 13:25

Maarten Bodewes

90,524
13
150
263

Thank you! This took care of it. I just added a `bigInt = bigInt.Abs()` assignment to ensure it was always positive. – Paul S Mar 14 '23 at 15:42
Hmm, 0xFF is 255 in unsigned, but -1 in two complement... ABS(-1) != 255. – Maarten Bodewes Mar 14 '23 at 15:46
Heh, I didn't get a reaction on that. You do understand that using `Abs()` will not return the right value? Nor is the byte order correct, you need to convert between big and little endian (found the required documentation, D is already a byte array) – Maarten Bodewes Mar 14 '23 at 16:21
Ok, thank you. Since this is a BigInteger, can I simply swap all the bytes from front-to-back to get the endianness changed? I don't see any way to init a BigInteger from big endian or how to export the parameter in little endian. – Paul S Mar 14 '23 at 19:12
If the first byte is smaller than 0x80 (unsigned of course) then just reverse the byte array, otherwise create a new byte array that is one byte larger and put the reverse bytes in there, starting at index 0. Then the final index byte remains zero. I'd make it a function, uh, `ToSignedLittleEndian`?. – Maarten Bodewes Mar 14 '23 at 19:15

Why does my ECDH shared secret generation sometimes fail?

1 Answers1