0

I'm not even sure I asked the question right...

I have three servers running minio in distributed mode. I need all three servers to run with TLS enabled. It's easy enough to run certbot, generate a cert for each node, drop said certs into /etc/minio/certs/ and go! but here's where I start running into issues.

The servers are thus:

node1.files.example.com
node2.files.example.com
node3.files.example.com

I'm launching minio using the following command:

MINIO_ACCESS_KEY=minio \
MINIO_SECRET_KEY=secret \
/usr/local/bin/minio server \
-C /etc/minio --address ":443" \
https://node{1...3}.files.example.com:443/volume/{1...4}/

This works and I am able to connect to all three servers from a webbrowser using https with good certs. however, users will connect to the server using the parent domain "files.example.com" (using distributed DNS)

I already ran certbot and generated the certs for the parent domain... and I copied the certs into /etc/minio/certs/ as well as /etc/minio/certs/CAs/ (calling both files "files.example.com-public.crt" and "files.example.com-public.key" respectively)... this did not work. when I try to open the parent domain "files.example.com" I get a cert error (chich I can bypass) indicating the certificate is for the node in which I have connected and not for the parent domain.

I'm pretty sure this is just a matter of putting the cert in the right place and naming it correctly... right? does anyone know how to do that? I also have an idea there might be a way to issue a cert that covers multiple domains... is that how I'm supposed to do this? how?

I already hit up minio's slack channel and posted on their github, but no ones replying to me. not even, "this won't work."

any ideas?

rudepeople
  • 57
  • 1
  • 7
  • it looks like I can accomplish what I need to do using the following: `certbot certonly --standalone -d node1.files.example.com -d files.example.com --staple-ocsp -m op@example.com --agree-tos` the trouble is, we already have a minio server running on files.example.com using that cert. I'm worried that issueing that command will invalidate the existing cert before we're ready to take the other server down. is there a way to dry run this? just adding --dry-run to the command didn't work (got python errors)... – rudepeople Jun 17 '20 at 18:14
  • actually, this brings up another possible issue... DNS. we're going to be using distributed DNS to point to the cluster. each node needs to have a cert... untill now I've been running certbot on all three individually to generate their individual node certs which has worked because I have seperate DNS entries for each node individually. but the parent domain points to all three, but node1 comes up first... won't that trip certbot? or am I overthinking this... can I safely run certbot on the first node and just copy the cert to the other two? – rudepeople Jun 17 '20 at 18:23
  • As it is not really about programming at this stage your question might be offtopic here. I know nothing about "minio" but about your latest question on certbot, yes you can generate the certificate on any host that can achieve the DNS or HTTP validation by Let's Encrypt and then copy the certificate and associated key to any host that needs it (and for witch the names matched). Remember that you can request wildcard certificates: one for `*.example.com` could be used by all hosts in `example.com`. – Patrick Mevzek Jun 17 '20 at 18:35
  • So if I'm understanding this correctly, it's best to run certbot the following command on node1: `certbot certonly --standalone -d *.files.example.com -d files.example.com --staple-ocsp -m op@example.com --agree-tos` here's the problem, when I do I get the following error: `Client with the currently selected authenticator does not support any combination of challenges that will satisfy the CA. You may need to use an authenticator plugin that can do challenges over DNS.` I assume I need to install the applicable dns plugin for my provider... except there isn't one for my distro (ubuntu)... – rudepeople Jun 17 '20 at 20:00
  • clearification, I'm using nsone. there is a nsone plugin... but certbot hasn't ported it to their PPA so I'm stuck. – rudepeople Jun 17 '20 at 20:02

2 Answers2

0

I gave up and ran certbot in manual mode. it had to install apache on one of the nodes, then certbot had me jump through a couple of minor hoops (namely it had me create a new txt record with my DNS provider, and then create a file with a text string on the server for verification). I then copied the created certs into my minio config directory (/etc/minio/certs/) on all three nodes. that's it.

to be honest, I'd rather use the plugin as it allows for an automated cert renewal, but I'll live with this for now.

rudepeople
  • 57
  • 1
  • 7
0

You could also run all of them behind a reverse proxy to handle the TLS termination using a wildcard domain certificate (ie. *.files.example.com). The reverse proxy would centralize the certificates, DNS, and certbot script if you prefer, etc to a single node, essentially load balancing the TLS and DNS for the minio nodes. The performance hit of "load-balancing" TLS like this may be acceptable depending on your workload, considering the simplification to your current DNS and TLS cert setup.

[Digital Ocean example using nginx and certbot plugins] https://www.digitalocean.com/community/tutorials/how-to-create-let-s-encrypt-wildcard-certificates-with-certbot

cbcoutinho
  • 634
  • 1
  • 12
  • 28