RabbitMQ TLS clustering - "insufficient security"

Question

I am currently setting up a RabbitMQ cluster where a requirement is that all communication within the cluster be encrypted.

I followed the guide at https://www.rabbitmq.com/clustering-ssl.html - created a file with the cert and key, and passed the SSL arguments to rabbitmq-server as environment variables:

RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS='-pa /usr/lib/erlang/lib/ssl-7.1/ebin -proto_dist inet_tls -ssl_dist_opt server_certfile /etc/ssl/certs/rabbit.pem'
RABBITMQ_CTL_ERL_ARGS='-pa /usr/lib/erlang/lib/ssl-7.1/ebin -proto_dist inet_tls -ssl_dist_opt server_certfile /etc/ssl/certs/rabbit.pem'

The server starts up fine and listens with TLS enabled, but I can not issue any commands to it via rabbitmqctl:

# rabbitmqctl status
Status of node rabbit@rabbit01 ...
Error: unable to connect to node rabbit@rabbit01: nodedown

DIAGNOSTICS
===========

attempted to contact: [rabbit@rabbit01]

rabbit@rabbit01:
  * connected to epmd (port 4369) on rabbit01
  * epmd reports node 'rabbit' running on port 47965
  * TCP connection succeeded but Erlang distribution failed
  * suggestion: hostname mismatch?
  * suggestion: is the cookie set correctly?
  * suggestion: is the Erlang distribution using TLS?

On the server's side, I don't get anything much in the logs other than the thoroughly unhelpful:

=ERROR REPORT==== 30-Dec-2015::13:08:58 ===
SSL: hello: tls_handshake.erl:167:Fatal error: insufficient security

A bit of searching around has indicated this can be due to a cipher suite mismatch, but my understanding here was that they would both be using the same SSL library.

Erlang dues have SSL support enabled as checked on https://www.rabbitmq.com/troubleshooting-ssl.html, and I can make a secure connection with the certificate with openssl's s_server and s_client.

I'm not sure what the next step would be to troubleshooting this, and couldn't find any known issue anywhere around clustering - unfortunately, most of RabbitMQ's documentation regarding TLS is for the AMQP protocol and not for the internal clustering.

I have tested using openSSL to connect to the running RabbitMQ instance as well and get a very similar error:

# openssl s_client -connect localhost:47965
CONNECTED(00000003)
140004605863584:error:1407742F:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert insufficient security:s23_clnt.c:770:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 295 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---

 openssl s_client -connect localhost:47965 -tls1_2
CONNECTED(00000003)
140400037775008:error:1409442F:SSL routines:SSL3_READ_BYTES:tlsv1 alert insufficient security:s3_pkt.c:1262:SSL alert number 71
140400037775008:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 0 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID:
    Session-ID-ctx:
    Master-Key:
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1451944018
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

You say you tested with s_client and s_server, but did you test connecting with s_client to the port that erlang is running on? Use options `-show_certs` and `-issuer_checks`. — Law29, Jan 01 '16 at 23:44
Tested that and still getting the same "insufficient security" errors. — user5416, Jan 04 '16 at 21:49
That output looks suspiciously like you are connected to port with no SSL at all. I get almost the same when I connect to a plain webserver. Check that it is an SSL/TLS port, and check if you have any server-side logs. — Law29, Jan 04 '16 at 22:37
If I connect to a port without SSL/TLS with s_client, it seems to work fine without complaining at all (if I run, for example, `nc -kl 1234` and then ` openssl s_client -connect localhost:1234 -tls1_2` then it just returns `CONNECTED(00000003)`). — user5416, Jan 05 '16 at 18:27

score 0 · Answer 1 · answered Oct 03 '19 at 10:00

It can be a matter of cipher suite or TLS version mismatch (e.g. a client is limited to TLSv1 but RabbitMQ is configured to require TLSv1.2) but can also be due to subtle key usage fields in the certificate. The latter is particularly tricky to notice.

See [1][2][3][4][5].

RabbitMQ TLS clustering - "insufficient security"

1 Answers1