Bazel Remote Cache not working with certificates/TLS

Question

I'm runnig Bazel remote cache from https://github.com/buchgr/bazel-remote inside docker, by runnig the following commands to start the http and grpc server:

docker pull buchgr/bazel-remote-cache
docker run -u 1000:1000 -v /path/to/cache/dir:/data -p 9093:8080 -p 9094:9092 buchgr/bazel-remote-cache

So far I can run some tests from my client by running one of the following commands:

bazelisk test  --remote_cache=http://<SERVER_IP>:<SERVER_PORT> <TEST_TARGET>
bazelisk test  --remote_cache=grpc://<SERVER_IP>:<SERVER_PORT> <TEST_TARGET>

But when I put my server to run over TLS by adding the following flags to the server invocation:

-v /path/to/certificate_authority:/etc/bazel-remote/ca_cert
-v /path/to/server_cert:/etc/bazel-remote/server_cert
-v /path/to/server_key:/etc/bazel-remote/server_key
--tls_ca_file=/etc/bazel-remote/ca_cert
--tls_cert_file=/etc/bazel-remote/server_cert
--tls_key_file=/etc/bazel-remote/server_key

Although I created a root CA certificate to sign both my server's and clients' certificates, I'm not able to access my server via https or grpcs. The following errors are being raised for https:

avax.net.ssl.SSLHandshakeException: General OpenSslEngine problem


sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target


sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

And the following for grpcs:

javax.net.ssl.SSLHandshakeException: General OpenSslEngine problem

java.security.cert.CertificateException: No subject alternative names present

ERROR: Failed to query remote execution capabilities: General OpenSslEngine problem

Is this problem related to the fact the root CA certificate is self signed itself or am I doing something else stupid?

I used the very same commands from the following link: https://github.com/bazelbuild/bazel/blob/master/src/test/testdata/test_tls_certificate/README.md

Answer 1

For grpcs, your problem is the hostname the client is using doesn't match the certificate. When I connect to localhost (like those directions default to), it works:

bazel test --remote_cache=grpcs://localhost:9094 --tls_client_certificate=/path/to/client.crt --tls_client_key=/path/to/client.pem --tls_certificate=/path/to/ca.crt //...

But if I switch to 127.0.0.1, it fails like you're seeing. If you have a hostname, changing SERVER_CN to match it should work. IP addresses are a bit trickier though, because you have to use a subject alternative name to tie the certificate to the IP. The most direct way I know is creating a file extensions.cnf with this line:

subjectAltName=IP:<SERVER_IP>

and then using that file with -extfile when generating the certificate. You can just add this command at the end of the directions to generate a new certificate including it:

openssl x509 -req -passin pass:1111 -days 358000 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt -extfile extensions.cnf

I don't think bazel supports client certificates with https. You can fix the problem you're running into using a custom truststore, but I can't figure out how to pass the client certificate. The only part of the code that uses that option looks like it's GRPC-specific. The documentation does not make that clear though...

A few more x509 basics: CA certificates are usually self-signed, because as the root of trust clients don't care how they're signed. The server certificate generated by those directions is not self-signed, contrary to the echo statement. Clients validate that the certificate they receive from the server matches the entity they're attempting to connect to (so that a certificate issued for yourwebsite.com can't be used to pretend to be google.com, for example). The CN is one field that can tie those together, subjectAltName is an extension that can be used to tie a certificate to multiple entities in various formats.