OpenSSL - digital signature generated through code does not match signature generated on command line

Question

UPDATE

I didn't realize this question was still getting eyeballs.

I found and fixed the problem some time ago, and it was basically me not understanding how to do things properly. I wasn't using the right file type for the certificate, I was not reading the private key properly (should have been using PEM_read_RSAPrivateKey, not PEM_read_PrivateKey), I was not computing the signature properly, etc.

It just took me more time than it should have to figure it out.

Thanks for everyone who took the time to answer.

ORIGINAL

I'm working on some code to digitally sign some requests, but the signature I am generating from the code is incorrect, and it doesn't jibe with a signature generated from the command line. I suspect the problem is that I am not reading the private key in properly, but I would expect to get some errors back if that were the case, and I'm not.

This is the first time I've worked with openssl, and I'm sure I'm doing things incorrectly, but I don't yet understand enough to ask intelligent questions.

I've created a MessageSigner class to handle the libcrypto chores. The private key and certificate are attributes of the class:

class MessageSigner 
{
  ...
  private:

  EVP_PKEY *private_key;
  X509     *certificate;
};

I initialize these to NULL when an instance is created:

MessageSigner::MessageSigner() : 
  private_key( NULL ),
  certificate( NULL )
{
}

I load the private key as follows:

void MessageSigner::addKeyFile( const std::string& filename, const std::string& passphrase )
{
  FILE *fp = ::fopen( keyfile.c_str(), "r" );
  if ( !fp )
    // throw exception

  private_key = PEM_read_PrivateKey( fp, NULL, NULL, passphrase.c_str() );
  ::fclose( fp );

  if ( !private_key )
    // throw exception
}

And I am generating the signature as

void MessageSigner::signMessage( const std::vector< unsigned char >& msg, std::vector< unsigned char >& signature )
{
  unsigned char *msgbuf = new unsigned char [msg.size()];
  std::copy( msg.begin(), msg.end(), msgbuf );

  unsigned char *sigbuf;
  size_t sigbuf_length;

  EVP_MD_CTX *ctx = EVP_MT_CTX_create();
  if ( !ctx )
    // throw exception

  if ( EVP_DigestSignInit( ctx, NULL, EVP_sha256(), NULL, private_key ) != 1)
    // throw exception

  if ( EVP_DigestSignUpdate( ctx, msgbuf, msg.size() ) != 1 )
    // throw exception

  if ( EVP_DigestSignFinal( ctx, NULL, &sigbuf_length ) != 1 )
    // throw exception

  sigbuf = (unsigned char *)OPENSSL_malloc( sizeof *sigbuf * sigbuf_length );
  if ( !sigbuf )
    // throw exception

  if ( EVP_DigestSignFinal( ctx, sigbuf, &sigbuf_length ) != 1 )
    // throw exception

  std::copy( sigbuf, sigbuf + sigbbuf_length, std::back_inserter( signature ) );

  EVP_MD_CTX_destroy( ctx );
  OPENSSL_free( sigbuf ); // yes, there's potential for a memory leak, but I'm just trying to get this bastard to work.
}

Again, the signature generated from this code does not jibe with the result of

openssl rsautl -sign -inkey keyfile.pem -keyform PEM -in msg.txt -out signature

I'm convinced the problem is with how I'm loading the private key, but all the examples I've seen so far indicate that this should work.

I've been staring at this and pulling my hair our for two days. Any suggestions, hints, guidance, rude remarks, etc., will be much appreciated.

Thanks.

EDIT

Some examples should illustrate what I'm up against.

Here's the SignedInfo generated for a given message:

<ds:SignedInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
  <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2006/12/xml-exc-c14n#">
    <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="alws soapenv"/>
  </ds:CanonicalizationMethod>
  <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/>
  <ds:Reference URI="#TS-5b171864-232b-11e9-846f-00505695541c">
    <ds:Transforms>
      <ds:Transform Algorithm="http://www.w3.org/2006/12/xml-exc-c14n#">
        <ec:InclusiveNamespaces xmlns:ec="http://www.w3.org/2001/10/xml-exc-c14n#" PrefixList="wsse alws soapenv"/>
      </ds:Transform>
    </ds:Transforms>
    <ds:DigestValue>eeLn6ak1glbbbWE48q7olsxO0CO/fL85bZ+8hzcjrvE=
</ds:DigestValue>
    <ds:DigestMethod Algorithm="https//www.w3.org/2001/04/xmlenc#sha256"/>
  </ds:Reference>
</ds:SignedInfo>

The digest value is calculated from the timestamp:

<wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="TS-5b171864-232b-11e9-846f-00505695541c">
  <wsu:Created>2019-01-28T18:34:33Z</wsu:Created>
  <wsu:Expires>2019-01-28T18:34:38Z</wsu:Expires>
</wsu:Timestamp>

When I calculate the digest for that timestamp from the command line as so:

$ openssl dgst -sha256 -binary timestamp > timestamp.dgst

I get the same digest value:

$ openssl base64 -in timestamp.dgst
eeLn6ak1glbbbWE48q7olsxO0CO/fL85bZ+8hzcjrvE=

So far so good. That hash is added to SignedInfo, then I take the hash of SignedInfo:

cQaWLGHi8D/c1kXPG9i49xzAupeBuypvMvuMQlzA/wo=

And use it to generate the signature. Now, here's where things go completely off the rails. When I generate the signature using the above code, I get:

HtQ4LkYq4Eao4bMOpV4SBpMxHi2a+0ilxDXS9jIQZWdCC8HCNlpvVU4rWMZG2Zd/
LplCWmUHIaB35FKv6uKjCjJPVDAJT2agyp7FnSKxaBI44Y/YsdvKyxJTAMiAlF8i
dd1MB8ljYsfayrzq5e76kt2cbHlYkT/RM3SvwJtjZiYsNpfcXD0Bi6JhRshHxQ8s
6/errruOe7jUqbKh7UOPJokadCX0OTSSwRgcs+sm7VjnS9MYILaGzFFT3Js9xI6d
TL4B6A/JGIkEqLO+GA1lrokAeIBr9OVUu7OEzaBb7DaiP9Gv1diu0j1sbZ4uT5Cf
CjYJPYU72Xx8F+MKdSJteg==

When I use the command-line tool, I get:

PvfCDqPl86/8USbFU0XR5r1Dhl5JbWd2va3L4W1IW1zw6xdes04F4lYjol6gMKio
jyr8DdmWBquroVlo4vW8kmhr6760qMcpK6mfsZ26ftu7XRC+Z4b9ge6ICOemsGlE
04Yoh9EpECP+ei5yS4E1sbntteiSoQcjotmVcIbPaEG5DIDcd4JKfoCWmsnuZESs
qctIJAQy4YY9HJsVGJ2JG7QashFcEQJabtInFgYeKuxla0ZSXBfOBkwHZT/cSv+k
n/NqPMCyEl4B2LiPBVa36GaTUd6fx0SXnIh0Fm+jw6b6j3EjU0QfMJ/JBAlL+oWZ
fXO/pS5L7W+OWk8Fh//iKA==

When I verify the signature generated on the command line, I get the original hash back:

 $ openssl base64 -d -in signature.b64 > signature.reversed

 $ openssl rsautl -verify -inkey cert.pem -certin -in signature.reversed > signature.reversed.dgst

$ openssl base64 -in signature.reversed.dgst
cQaWLGHi8D/c1kXPG9i49xzAupeBuypvMvuMQlzA/wo=

When I try to verify the signature generated by my code, I do not get the original hash back:

$ openssl base64 -d -in badsig.b64 > badsig

$ openssl rsautl -verify -inkey cert.pem -certin -in badsig > badsig.dgst

$ openssl base64 -in badsig.dgst
MDEwDQYJYIZIAWUDBAIBBQAEIHEGlixh4vA/3NZFzxvYuPccwLqXgbsqbzL7jEJc
wP8K

What's especially frustrating is that I've since added code to verify the signature after I create it, and that code passes. But it's obviously wrong - the remote service rejects our requests because the hashes don't match, and it doesn't match with what's generated by the openssl command line tool. I've verified that both the command line tool and the library I'm using are the same version (1.0.1).

I've added code to dump the private key to compare against what's in the keyfile, and it matches, so I'm reading the key in correctly.

There appear to be no less than 3 different ways to sign a message using libcrypto routines - EVP_Sign(), EVP_DigestSign(), EVP_PKEY_sign() - and I'm not sure which one is the correct one to use in my case. All three give me bad signatures (meaning, when decrypted against the public key, do not result in the original hash value).

So, again, if anyone can point to anything I'm doing that's obviously wrong, I would appreciate it.

MORE EDIT

Also attempting to verify using dgst as follows, again using the SignedInfo block above:

$ openssl dgst -sha256 -binary -sign fx-realtime.fundsxpress.com.pem -out signature signedinfo

$ openssl dgst -sha256 -binary -verify publickey.pem -signature signature signedinfo
Verified OK

Reversing the signature my code generates:

$ echo -n 'XfgP1A08UTwz3sUHIVvvV+fq1n3act6+lVBZ8ieDtgh28k1r1/M0tm9MntvK+Hm4
> Be+LjguX2gxhZ4PvVcoCBCugDIsrhxplDeB4bYeY2PEedQL6+IZFX+kFrz6o3RQa
> W7sXK7czogxWpdLAmKnhDJOk2BmKFihkRMTjo9D4z/qylZI9nnX29HNdg3uV2BYw
> zHh8GvYO8fy1ugqfFW80na+hLBAtBP6fwTTv10DS2L8n+ixQcnxlKW5pyBOXlR/r
> mZEqwU+A996G0573HkGFeFvXzArlRFg/7mkKoyUHyqyDzkf5eC+vTnpEy1CP75Yc
> lvd7ldSrwREisPnyxu47sg=='> computed_signature.b64

$ openssl base64 -e -in computed_signature.b64 > computed_signature

$ openssl dgst -sha256 -binary -verify publickey.pem -signature computed_signature signedinfo
Verification Failure

A back insert iterator into a ***const vector***? I haven't tried this trick myself, but I'm surprised that this compiled, at all... Either that, or I just suffered a stroke, and not reading this correctly... In either case, copying a `std::vector` into a `new`ed buffer, like that, accomplishes absolutely nothing useful, whatsoever. — Sam Varshavchik, Jan 25 '19 at 23:42
@SamVarshavchik: you’re not having a stroke. I didn’t want to cut and paste and botched it. — John Bode, Jan 26 '19 at 21:12

score 0 · Answer 1 · answered Jan 27 '19 at 17:11

0

The code is fine and the signature generated is fine. What you are missing is that every time you generate a signature it will be different each time. If you run the signatures through the verify they will all work. I don’t known the reason why but I assume there is a security reason why this is done this way.

answered Jan 27 '19 at 17:11

Shane Powell

13,698
2
49
61

The signature generated is *not* fine - that's the problem. It's being rejected by the service. When I decrypt the generated signature with our public key, I do not get the original hash back. – John Bode Jan 28 '19 at 18:38
I think there is a disconnect here. You can't "get the original hash back". A signature is not a "hash". You need to verify the signature using EVP_DigestVerifyXxx using the public key of the key pair that generated the signature. – Shane Powell Jan 28 '19 at 19:47
See: https://www.openssl.org/docs/man1.1.1/man3/EVP_DigestVerify.html for the EVP_DigestVerifyXxx API – Shane Powell Jan 28 '19 at 19:49
Also your openssl commands are not the same as your code. If you wish to use the EVP_DigestXxx via the command line you use the "openssl dgst" command. "openssl rsautl" uses the RSA_private_encrypt / RSA_public_decrypt api. – Shane Powell Jan 28 '19 at 20:29

score 0 · Answer 2 · answered Apr 01 '20 at 17:06

Its not clear from the details here how the signed data (input) is treated.

the openssll commands will read these as binary (not text) and there may be newline data that is changed if you use text reads (ie on windows 2 chars CR/LF -> transition to 1 CR) before you sign the input.

not signing the same input will create a diff digest (obviously).

OpenSSL - digital signature generated through code does not match signature generated on command line

2 Answers2