0

I'm generating a keypair in a softHSM keystore, and uses it to sign a software.

Now I need to extract the public key and move it to my target.
I can sign and verify with pkcs11-tool, but I have not succeeded with using openssl verifying.

All I get is

80DB511C4A7F0000:error:02000068:rsa routines:ossl_rsa_verify:bad signature:../crypto/rsa/rsa_sign.c:430:

This is the series of commands I use to generate the keys.

softhsm2-util --init-token --free --label "token-label" --so-pin mysecret1 --pin mysecret1

pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so --login --login-type so --so-pin mysecret1 --init-pin --new-pin mysecret1

# create a public-private key pair. 
pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so -l --token-label token-label -k --key-type rsa:2048      --usage-sign --id 1002 --label rsatest     --pin mysecret1

pkcs11-tool --modul /usr/lib/softhsm/libsofthsm2.so --id 1002 --read-object --type pubkey -o rsa.der
openssl ec -pubin -inform DER -in rsa.der -outform PEM -out rsa.pem

pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so --login --pin mysecret1 --sign --id 1002 -m RSA-PKCS   --input text.txt --output /tmp/rsa.signature

pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so --id 1002  --verify -m RSA-PKCS   --input-file text.txt --signature-file  /tmp/rsa.signature

openssl dgst -sha512 -keyform PEM -verify rsa.pem -signature /tmp/rsa.signature text.txt

I have tried several combinations of PEM and DER files. Also to change the -sha512 parameter.

A demo project with Dockerfile can be found here: https://gitlab.com/kjeld.flarup/consoletest

Kjeld Flarup
  • 1,471
  • 10
  • 15
  • 1
    I'm seeing `parse_pss_params` in the source code of the badly documented and not so well programmed pkcs11-tool, so I guess you need to use the RSA-PSS signature algorithm. Unfortunately I cannot directly see the params used, apparently they are associated with the private key. Usually they are SHA-1, SHA-256 or SHA-512 and sometimes SHA-384 (the latter doesn't make any sense whatsoever, but yeah). I'd try SHA-256 for the data and MGF1 first, and then SHA-256 with SHA-1 for the MGF1. – Maarten Bodewes Feb 15 '23 at 10:32

1 Answers1

0

A little inspired by https://stackoverflow.com/users/589259/maarten-bodewes I searched for RSA-PSS and found that I could specify SHA512 in the mechanism. Then it worked.

pkcs11-tool --module /usr/lib/softhsm/libsofthsm2.so --login --pin mysecret1 --sign --id 1002 -m SHA512-RSA-PKCS --input text.txt --output /tmp/rsa.signature
Kjeld Flarup
  • 1,471
  • 10
  • 15