How to use CloudHSM to implement mutual TLS ( client side ) in Ruby

Question

I'm asking this question after reading all of CloudHSM topics on StackOverflow, Cryptography, Information Security and CloudHSM forum but couldn't find anything helpful. Any idea or code snippet is helpful.

We have a Ruby application that is requesting to a web server via X.509 certificates and we should generate/host private key inside CloudHSM.

I followed CloudHSM documentation step by step and configured TLS offloading via NGINX and Apache HTTPD to understand how it is working, now I'm working on mutual TLS with CloudHSM.

My web server requires client certificate, I can validate that via cURL:

curl --cert app-selfsigned.crt --key app-selfsigned.key  -k https://127.0.0.1/index.html

Also I can use this Ruby code to authenticate via certifications on disk:

require 'faraday'
require 'openssl'

def ssl_options
  cert_file = File.read "app-selfsigned.crt"
  key_file = File.read "app-selfsigned.key"
  ssl_options = {
    verify: false,
    client_cert: OpenSSL::X509::Certificate.new(cert_file),
    client_key: OpenSSL::PKey::RSA.new(key_file)
  }
end

def connection
  dest = "https://127.0.0.1/"
  connection = Faraday::Connection.new(dest, ssl: ssl_options)
  connection.get
end

irb -I . -r rubytest.rb
connection

cURL and Ruby test via certification on disk:

I need to host `app-selfsigned.key` key inside the CloudHSM, how can I do that?

1) Can I do that via CloudHSM OpenSSL Dynamic Engine? if yes, even after cloudhsm engine installation(/opt/cloudhsm/lib/libcloudhsm_openssl.so), should I load and install engine every time in my code?

2) Or should I use PKCS#11 via p11-kit or pkcs11-openssl packages and p11tool command or Ruby PKCS#11?

3) Should I add anything related to n3fips_password inside my Ruby application?

Here is Ruby code that I'm trying to use CloudHSM with it (I'm using FAKE PEM key instead of real private key to point to real private key with label nginx-selfsigned_imported_key inside CloudHSM):

require 'faraday'
require 'openssl'

def initialize_openssl
  key_label = "nginx-selfsigned_imported_key"
  # OpenSSL Engine:
  OpenSSL::Engine.load
  e = OpenSSL::Engine.by_id('cloudhsm')
  e.ctrl_cmd("SO_PATH", "/opt/cloudhsm/lib/libcloudhsm_openssl.so")
  e.ctrl_cmd("ID", "cloudhsm")
  e.ctrl_cmd("LOAD")
  e.load_private_key("CKA_LABEL=#{ key_label }")
end

def ssl_options
  cert_file = File.read "app-selfsigned.crt"
  key_file = File.read "app-selfsigned_fake_PEM.key"
  {
    verify: false,
    client_cert: OpenSSL::X509::Certificate.new(cert_file),
    client_key: OpenSSL::PKey::RSA.new(key_file)
  }
end

def connection
  dest = "https://127.0.0.1/"
  Faraday::Connection.new(dest, ssl: ssl_options)
end

def connect
  initialize_openssl
  c = connection
  c.get
end

irb -I . -r rubytest_cloudhsm.rb
initialize_openssl

but I get this error:

OpenSSL::Engine::EngineError: invalid cmd name
from /root/self-signed/app-selfsigned/rubytest_cloudhsm.rb:9:in `ctrl_cmd'
from /root/self-signed/app-selfsigned/rubytest_cloudhsm.rb:9:in `initialize_openssl'
from (irb):1
from /bin/irb:12:in `<main>'

Adding them line by line:

Ruby error with CloudHSM:

Debug logs

OpenSSL Dynamic Engine has been installed sucessfuly:

export n3fips_password=<Crypto User Username>:<CU Password>
openssl engine -tt cloudhsm
# (cloudhsm) CloudHSM hardware engine support
#       SDK Version: 2.03
# [ available ]
openssl engine -vvvv dynamic -pre SO_PATH:/opt/cloudhsm/lib/libcloudhsm_openssl.so -pre ID:cloudhsm -pre LOAD
# (dynamic) Dynamic engine loading support
# [Success]: SO_PATH:/opt/cloudhsm/lib/libcloudhsm_openssl.so
# [Success]: ID:cloudhsm
# [Success]: LOAD
# Loaded: (cloudhsm) CloudHSM hardware engine support
openssl speed -engine cloudhsm
# SDK Version: 2.03
# engine "cloudhsm" set.
# Doing md2 for 3s on 16 size blocks:
# 557992 md2's in 2.99s
openssl version
# OpenSSL 1.0.2k-fips  26 Jan 2017
rpm -qa | grep -i openssl
# openssl-1.0.2k-16.amzn2.1.1.x86_64
# openssl-libs-1.0.2k-16.amzn2.1.1.x86_64

OpenSSL CloudHSM Dynamic Engine shared object is in the correct place:

ls -ltrha /usr/lib64/openssl/engines/libcloudhsm.so
lrwxrwxrwx 1 root root 40 Aug  7 09:56 /usr/lib64/openssl/engines/libcloudhsm.so -> /opt/cloudhsm/lib/libcloudhsm_openssl.so

libcloudhsm_openssl.so:

OS:

cat /etc/os-release
# NAME="Amazon Linux"
# VERSION="2"
# ID="amzn"
# ID_LIKE="centos rhel fedora"
# VERSION_ID="2"
# PRETTY_NAME="Amazon Linux 2"
# ANSI_COLOR="0;33"
# CPE_NAME="cpe:2.3:o:amazon:amazon_linux:2"
# HOME_URL="https://amazonlinux.com/"
uname -a
# Linux hsm.example.net 4.14.133-113.112.amzn2.x86_64 #1 SMP Tue Jul 30 18:29:50 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

rpm -qa | grep -i cloudhsm-client
# cloudhsm-client-2.0.3-3.el7.x86_64
# cloudhsm-client-dyn-2.0.3-3.el6.x86_64

I can check private and public key with or without CloudHSM engines:

app-selfsigned.crt: Public Key
app-selfsigned.key: Private key has been exported from CloudHSM 
app-selfsigned_fake_PEM.key: Fake private key pointing to real private key inside CloudHSM generated by getCaviumPrivKey -k 14 -out app-selfsigned_fake_PEM.key

# Test without CloudHSM:
openssl s_server -cert app-selfsigned.crt -key app-selfsigned.key
# Using default temp DH parameters
# ACCEPT
openssl s_server -cert app-selfsigned.crt -key app-selfsigned_fake_PEM.key
# Using default temp DH parameters
# ACCEPT


# Test with CloudHSM engine
openssl s_server -cert app-selfsigned.crt -key app-selfsigned.key -engine cloudhsm
# SDK Version: 2.03
# engine "cloudhsm" set.
# Using default temp DH parameters
# ACCEPT
openssl s_server -cert app-selfsigned.crt -key app-selfsigned_fake_PEM.key -engine cloudhsm
# SDK Version: 2.03
# engine "cloudhsm" set.
# Using default temp DH parameters
# ACCEPT

To make sure I'm requesting the correct key inside CloudHSM, I configured `app-selfsigned.crt` and `app-selfsigned_fake_PEM.key` for TLS offloading via NGINX:

/etc/nginx/nginx.conf:

ssl_engine cloudhsm;
ssl_certificate "/etc/pki/nginx/app-selfsigned.crt";
ssl_certificate_key "/etc/pki/nginx/private/app-selfsigned_fake_PEM.key";

nginx -t
# SDK Version: 2.03
# nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
# nginx: configuration file /etc/nginx/nginx.conf test is successful
systemctl start nginx && systemctl status nginx
# Aug 13 11:21:33 hsm.example.net nginx[13046]: SDK Version: 2.03

Check cert file via OpenSSL:

openssl x509 -in app-selfsigned.crt -text -noout
# Serial Number: c7:c4:07:a6:78:22:2e:ff
# Subject: C=AA, ST=AA, L=AA, O=AA, OU=AA, CN=a.com/emailAddress=a@a.com

Certificate check via Firefox:

Checking private key inside CloudHSM via key_mgmt_util and cloudhsm_mgmt_util:

/opt/cloudhsm/bin/key_mgmt_util
loginHSM -u CU -s CUADMIN -p CUPASSWORD
findSingleKey -k 14
# Cfm3FindSingleKey returned: 0x00 : HSM Return: SUCCESS
getKeyInfo -k 14
# Cfm3GetKey returned: 0x00 : HSM Return: SUCCESS
# Owned by user: 6

/opt/cloudhsm/bin/cloudhsm_mgmt_util /opt/cloudhsm/etc/cloudhsm_mgmt_util.cfg
enable_e2e
loginHSM CU CUADMIN CUPASSWORD

getAttribute 14 0
# OBJ_ATTR_CLASS
# 0x00000003
# 3: Private key in a public–private key pair. 
getAttribute 14 2
# OBJ_ATTR_PRIVATE
# 0x00000001
# 1: True. This attribute indicates whether unauthenticated users can list the attributes of the key. Since the CloudHSM PKCS#11 provider currently does not support public sessions, all keys (including public keys in a public-private key pair) have this attribute set to 1.
getAttribute 14 3
# OBJ_ATTR_LABEL
# nginx-selfsigned_imported_key
getAttribute 14 256
# OBJ_ATTR_KEY_TYPE
# 0x00000000
# 0: RSA.

If you are using CloudHSM mutual TLS in another language please paste your code here so I can get the idea and implement it in Ruby.

Thanks in advance.

score 3 · Accepted Answer · answered Oct 13 '19 at 10:09

3

To anyone who see this question later, here is a sample Ruby code works with Amazon CloudHSM:

require 'openssl'
require 'base64'

FAKE_KEY = "/root/ruby/ruby_key_inside_hsm/ruby_hsm_fake_private.key"
REAL_KEY = "/root/ruby/ruby_key_inside_hsm/ruby_hsm_real_private_exported.key"
PUB_KEY = "/root/ruby/ruby_key_inside_hsm/pubkey.pem"

STR = "test string"

def encrypt(str)
  pubkey = OpenSSL::PKey::RSA.new(File.read(PUB_KEY))
  Base64.encode64(pubkey.public_encrypt(str))
end

def decrypt(str, key)
  OpenSSL::Engine.load
  privkey = OpenSSL::PKey::RSA.new(File.read(key))
  privkey.private_decrypt(Base64.decode64(str))
end


def estr
  encrypt(STR)
end

def real_dec
  decrypt(estr, REAL_KEY)
end

def hsm_dec
  OpenSSL::Engine.load
  OpenSSL::Engine.by_id('cloudhsm')
  decrypt(estr, FAKE_KEY)
end

Right now we are working on it to add it to the production environment.

answered Oct 13 '19 at 10:09

hi, how do you export fake_private_key? I cannot use private_decrypt or sign – Tim Hsu Dec 04 '21 at 07:47
Hi @TimHsu to export the fake private key from HSM you can use getCaviumPrivKey command described in these two documents: https://docs.aws.amazon.com/cloudhsm/latest/userguide/key_mgmt_util-getCaviumPrivKey.html https://docs.aws.amazon.com/cloudhsm/latest/userguide/ssl-offload-import-or-generate-private-key-and-certificate.html I'm not sure if I understood your question correctly. Feel free to ask any question regarding CloudHSM, I would be glad to help. – Dec 05 '21 at 08:04
I solved it. thank you very much. – Tim Hsu Dec 06 '21 at 06:33
HI @Amin Khoshnood, I found OpenSSL::Engine.by_id("cloudhsm") is affecting all crypto operations to use cloudhsm. could you confirm it? – Tim Hsu Apr 17 '22 at 13:20
@TimHsu, yes this is correct. This was one of our challenges. We initiated the CloudHSM engine via OpenSSL::Engine.by_id("cloudhsm") in Rails applications and later found out that it causes the Rails application to send all of the crypto operations to HSM even TLS handshake sign and verify requests. This overloads HSMs very fast and AWS doesn't provide any monitoring. So the best option is to implement an HSM wrapper in a separate Ruby, Python or Go script and initialize cloudhsm engine in that script. This will assure you that only specific operations will go to the CloudHSM cluster. – Apr 18 '22 at 14:13
It's ruby's offical OpenSSL limitation. I start writing c extension for Ruby today. OpenSSL c API allows you to specify engine – Tim Hsu Apr 18 '22 at 16:16
Oh, I didn't know that. Thanks a lot for sharing this information. Did you upgrade to CloudHSM v5 client or are you still using v3? We upgraded to v5 recently and I loved it. It's more optimum and much faster than v3 and it's perfect for containers. – Apr 20 '22 at 06:17
I use v5 directly. I never had experience using CloudHSM – Tim Hsu Apr 20 '22 at 10:32

How to use CloudHSM to implement mutual TLS ( client side ) in Ruby

I need to host app-selfsigned.key key inside the CloudHSM, how can I do that?

Debug logs

To make sure I'm requesting the correct key inside CloudHSM, I configured app-selfsigned.crt and app-selfsigned_fake_PEM.key for TLS offloading via NGINX:

Checking private key inside CloudHSM via key_mgmt_util and cloudhsm_mgmt_util:

1 Answers1

I need to host `app-selfsigned.key` key inside the CloudHSM, how can I do that?

To make sure I'm requesting the correct key inside CloudHSM, I configured `app-selfsigned.crt` and `app-selfsigned_fake_PEM.key` for TLS offloading via NGINX: