Unable to run calico ingress and egress rules

Question

I have been trying to experiment with the calico network rules and I'm finding it tough to get the ingress and the egress rules to work with order in calico after denying all ingress and egress rules.

 kubectl get pods --show-labels
NAME         READY   STATUS    RESTARTS   AGE   LABELS
hello-web3   1/1     Running   0          45m   app=foo
hello-web4   1/1     Running   0          45m   app=bar
hello-web5   1/1     Running   0          15s   app=foobar
hello-web6   1/1     Running   0          4s    app=barbar

My network policy is as follows

---
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: ppdp-default
spec:
  selector: all()
  order: 2000
  types:
  - Ingress
  - Egress
---
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: ppdp-egress-trusted
spec:
  selector: app == 'foo'
  order: 1000
  types:
  - Egress
  egress:
  - action: Allow
    destination:
      selector: app == 'bar'
---
apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: ppdp-ingress-trusted
spec:
  selector: app == 'foobar'
  order: 100
  types:
  - Ingress
  ingress:
  - action: Allow
    source:
      selector: app == 'barbar'

Output for Ingress:

(base) ➜ ✗ kubectl exec --stdin --tty hello-web5 -- sh
/ # ^C
/ # wget -qO- --timeout=2 http://hello-web6:8080
^C
/ # wget -qO- --timeout=2 http://hello-web6:8080
wget: bad address 'hello-web6:8080'
/ # command terminated with exit code 1
---
(base) ➜ ✗ kubectl exec --stdin --tty hello-web6 -- sh
/ # wget -qO- --timeout=2 http://hello-web5:8080
wget: bad address 'hello-web5:8080'
/ # command terminated with exit code 1

Output for Egress

(base) ➜ ✗ kubectl exec --stdin --tty hello-web3 -- sh
/ # wget -qO- --timeout=2 http://hello-web4:8080
^C
/ # command terminated with exit code 130

Am I missing anything? Any help would be of great use.

Thanks in advance

Answer 1

In a namespace, I want to deny traffic among all pods in the first place and then allow egress or ingress traffic between specific pods (matching labels)

While I don't know why you're using the order or calico network policies the goal described in the comments can be achieved with Kubernetes network policies supported by Calico CNI. I prepared a simple example of how network policies work with those. So let's start with a list of pods that I have created in dev namespace:

➜  ~ kgp -n dev --show-labels -owide

NAME          READY   STATUS    RESTARTS   AGE   IP              NODE       NOMINATED NODE   READINESS GATES   LABELS
centos        1/1     Running   0          30m   10.244.120.68   minikube   <none>           <none>            host=centos
echo-server   1/1     Running   0          30m   10.244.120.69   minikube   <none>           <none>            host=echo-server

Notice the labels. In my example we are going to allow ingress traffic to echo-server from host named centos. Let's first test if the connections works between those:

[root@centos /]# curl 10.244.120.69:80 -v
* About to connect() to 10.244.120.69 port 80 (#0)
*   Trying 10.244.120.69...
* Connected to 10.244.120.69 (10.244.120.69) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: 10.244.120.69
> Accept: */*
  "path": "/",
  "headers": {
    "user-agent": "curl/7.29.0",
    "host": "10.244.120.69",
    "accept": "*/*"
  "os": {
    "hostname": "echo-server"
  },
  "connection": {}
* Connection #0 to host 10.244.120.69 left intact

Now let's deny all traffic in that namespace:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-all
  namespace: dev
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

Once this policy is in place previous test fails:

➜  ~ keti -n dev centos bash 
[root@centos /]# curl 10.244.120.69:80 -v
* About to connect() to 10.244.120.69 port 80 (#0)
*   Trying 10.244.120.69...
* Connection timed out
* Failed connect to 10.244.120.69:80; Connection timed out
* Closing connection 0
curl: (7) Failed connect to 10.244.120.69:80; Connection timed out

Let's now apply the ingress policy that will allow us to reach the echo-server. So basically we selects the pod's label that we want the policy to be applied to and the choose which pod ingress traffic is allowed from:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-echo
  namespace: dev    
spec:
  podSelector:
    matchLabels:
      host: echo-server
  ingress:
    - from:
      - podSelector:
          matchLabels:
            host: centos

Now since we allowed the traffic towards our echo-server, we might feel tempted to test this straight away but this still won't work. While we allowed the ingress traffic toward echo-server we have to remember that we denied both ingress and egress in our deny-all policy. This means that we have to allow the egress traffic from centos pod:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: egress-centos
  namespace: dev    
spec:
  podSelector:
    matchLabels:
      host: centos
  egress:
    - to:
      - podSelector:
          matchLabels:
            host: echo-server      

After that we have successfully allowed traffic for some specific pods in the same namespace while deny every pod that don't match the labels.