0

I have a kubernetes cluster where control planes are working in HA through keepalived's VIP configured among them(installed only on control nodes). Everything works as expected when firewall is disabled. all internal communications and nodePorts behave inconsistently when firewall is enabled. As a basic startup i have enabled certain mandatory ports which are required for kubernetes but this doesn't help for applications to communicate. Is there any way or add any firewall rules that helps me to access the nodePorts/application services?

Incase if i'm unclear, this is a small summary

Control Plane 1: 172.16.23.110

Control Plane 2: 172.16.23.111

Control Plane 3: 172.16.23.112

Keepalived VIP: 172.16.23.116

Worker Nodes : 172.16.23.120-23.125

I'm trying to access service through nodePort like https://172.16.23.116:30443

i have added below firewall case across all nodes(all nodes are centos7.6)

cat /etc/firewalld/zones/internal.xml

<rule>
    <protocol value="vrrp" />
    <accept />
</rule>

Kindly help

user2039152
  • 146
  • 8
  • 1
    The firewall rules that you pasted is unclear. did you allow port `30443` on `172.16.23.116`? – Veerendra K Feb 13 '20 at 13:01
  • yes i have already enabled all required ports, from any non-VIP i can access it even when there is a firewall running. like for example: https://control-plane-1-ip:30443 works fine even when there is firewall enabled. – user2039152 Feb 13 '20 at 13:36
  • It sounds like you are trying to reimplement MetalLB? Maybe just use that? – coderanger Feb 13 '20 at 19:18

1 Answers1

1

I am not sure, that I understood question correctly.

But according to the official documentation you need to have the following ports be opened:

Control-plane node(s):

  • TCP Inbound 6443*
  • TCP Inbound 2379-2380
  • TCP Inbound 10250
  • TCP Inbound 10251
  • TCP Inbound 10252

Worker node(s):

  • TCP Inbound 10250
  • TCP Inbound 30000-32767

Here are the examples of commands for Firewalld, taken from here:

Control-plane node(s):

firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --permanent --add-port=2379-2380/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10251/tcp
firewall-cmd --permanent --add-port=10252/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --add-masquerade --permanent
# only if you want NodePorts exposed on control plane IP as well
firewall-cmd --permanent --add-port=30000-32767/tcp
systemctl restart firewalld

Worker node(s):

firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --permanent --add-port=8472/udp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --add-masquerade --permanent
systemctl restart firewalld
Artem Golenyaev
  • 2,568
  • 12
  • 20
  • yes i have already enabled all required ports, from any non-VIP i can access it even when there is a firewall running. like for example: control-plane-1-ip:30443 works fine even when there is firewall enabled. – user2039152 Feb 13 '20 at 13:35
  • Then it seems problem is in Keepalived VIP. Is there anything in logs for Keepalived VIP? How is it configured? – Artem Golenyaev Feb 13 '20 at 13:42
  • nothing much there.. just usual logs.. `Feb 13 08:47:25 rkha-master1220 Keepalived_healthcheckers[1387]: HTTP status code success to [172.16.22.131]:6443 url(1). Feb 13 08:47:25 rkha-master1220 Keepalived_healthcheckers[1387]: Remote Web server [172.16.22.131]:6443 succeed on service. Feb 13 08:47:25 rkha-master1220 Keepalived_healthcheckers[1387]: Adding service [172.16.22.131]:6443 to VS [172.16.23.116]:6443` – user2039152 Feb 13 '20 at 13:49
  • @OhHiMark, could you update question with your firewall configuration? Probably, you need to add IP of Keepalive VIP to rules – Artem Golenyaev Feb 20 '20 at 13:52
  • @ArtemGolenyaev Sorry, I misstagged you instead of the OP. I believe you wanted to ask user2039152. – Wytrzymały Wiktor Feb 20 '20 at 14:12
  • OhHiMark no I haven't solved it yet. i stopped exploring due to other activities. probably will resume in few days @ArtemGolenyaev how can i include that? – user2039152 Feb 25 '20 at 09:16