I'm cracking my head for past few days over following problem. I would like to setup end-to-end encryption from client to pod running on EKS and preserve client ip. I've checked that to preserve ip, I would need to use proxy protocol V2. At first I tested this with TLS termination at ingress, configured proxy protocol in Nginx Ingress controller and on NLB and everything worked properly. As next step, I've enabled ssl-passtrough in Nginx controller to terminate SSL on pod and this is where things break. Anybody has success with such flow or some ideas what can be wrong with ssl-passtrough configuration?
When I access nlb DNS record via browser, I get such error
An error occurred during a connection to myexample.com. SSL received a record that exceeded the maximum permissible length. Error code: SSL_ERROR_RX_RECORD_TOO_LONG
If I do curl,I get
* Connected to myexample.com (xxx.xxx.xxx.xxx) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
* Closing connection 0
curl: (35) error:1400410B:SSL routines:CONNECT_CR_SRVR_HELLO:wrong version number
Here is my configuration changes:
Nginx service
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: '*'
service.beta.kubernetes.io/aws-load-balancer-type: nlb
labels:
helm.sh/chart: ingress-nginx-3.4.1
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/version: 0.40.2
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/component: controller
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
type: LoadBalancer
externalTrafficPolicy: Cluster
ports:
- name: http
port: 80
protocol: TCP
targetPort: http
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/component: controller
Nginx deployment started with - --enable-ssl-passthrough
Nginx Configmap has
data:
use-proxy-protocol: "true"
real-ip-header: "proxy_protocol"
Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: tls-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
rules:
- host: myexample.com
http:
paths:
- path: /
backend:
serviceName: secure-app
servicePort: 8443
Sample app
apiVersion: apps/v1
kind: Deployment
metadata:
name: secure-app
spec:
replicas: 1
selector:
matchLabels:
app: secure-app
template:
metadata:
labels:
app: secure-app
spec:
containers:
- name: secure-app
image: nginxdemos/nginx-hello:plain-text
ports:
- containerPort: 8443
volumeMounts:
- name: secret
mountPath: /etc/nginx/ssl
readOnly: true
- name: config-volume
mountPath: /etc/nginx/conf.d
volumes:
- name: secret
secret:
secretName: app-tls-secret
- name: config-volume
configMap:
name: secure-config
---
apiVersion: v1
kind: Service
metadata:
name: secure-app
spec:
ports:
- port: 8443
targetPort: 8443
protocol: TCP
name: https
selector:
app: secure-app
---
apiVersion: v1
kind: ConfigMap
metadata:
name: secure-config
data:
app.conf: |-
server {
listen 8443 ssl;
server_name myexample.com;
ssl_certificate /etc/nginx/ssl/tls.crt;
ssl_certificate_key /etc/nginx/ssl/tls.key;
default_type text/plain;
location / {
return 200 "hello from pod $hostname\n";
}
}