Bug 1941941

Summary: [kuryr] Egress network policy with namespaceSelector in Kuryr behaves differently than in OVN-Kubernetes
Product: OpenShift Container Platform Reporter: Michał Dulko <mdulko>
Component: NetworkingAssignee: Michał Dulko <mdulko>
Networking sub component: kuryr QA Contact: Jon Uriarte <juriarte>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: high CC: bbennett, cmarches, gcheresh, ltomasbo, mdulko, rdobosz, rlobillo
Version: 4.7Keywords: Triaged
Target Milestone: ---   
Target Release: 4.7.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: 1930017 Environment:
Last Closed: 2021-05-19 15:15:46 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1921878    
Bug Blocks: 1958103    

Comment 1 Michał Dulko 2021-03-23 08:45:50 UTC
I've only cloned this due to formal reasons - see https://issues.redhat.com/browse/ART-2801.

Comment 2 Michał Dulko 2021-03-23 08:46:23 UTC
*** Bug 1930017 has been marked as a duplicate of this bug. ***

Comment 5 Jon Uriarte 2021-05-13 09:53:51 UTC
Verified in 4.7.0-0.nightly-2021-05-12-004740 on top of OSP 13.0.15 (2021-03-24.1) with amphora
provider and and on top of OSP 16.1.5 (RHOS-16.1-RHEL-8-20210323.n.0) with ovn octavia provider.

SG rules generated by below NP resource definition allow traffic to other namespaces but not
to the outside:

$ cat np_bz1941941.yaml 
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: np-bz1941941
spec:
  podSelector:
    matchLabels:
      run: demo
  policyTypes:
  - Egress
  - Ingress
  ingress:
  - from:
    - podSelector: {}
  egress:
  - to:
    - namespaceSelector: {}
    

OSP 13.0.15
-----------
Steps:

1. Create test and test2 projects both with kuryr/demo pod exposed by a service on port 80:

$ oc new-project test
$ oc run --image kuryr/demo demo
$ oc expose pod/demo --port 80 --target-port 8080

$ oc -n test get all -o wide
NAME       READY   STATUS    RESTARTS   AGE     IP               NODE                          NOMINATED NODE   READINESS GATES
pod/demo   1/1     Running   0          3m53s   10.128.124.120   ostest-tbbzz-worker-0-94qjl   <none>           <none>

NAME           TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo   ClusterIP   172.30.6.65   <none>        80/TCP    10s   run=demo

$ oc new-project test2
$ oc run --image quay.io/kuryr/demo demo2
$ oc expose pod/demo2 --port 80 --target-port 8080

$ oc -n test2 get all -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP              NODE                          NOMINATED NODE   READINESS GATES
pod/demo2   1/1     Running   0          22s   10.128.127.82   ostest-tbbzz-worker-0-94qjl   <none>           <none>

NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo2   ClusterIP   172.30.39.179   <none>        80/TCP    14s   run=demo2


2. Apply np on demo pod in test project:

$ oc project test
$ oc apply -f np_bz1941941.yaml
networkpolicy.networking.k8s.io/np-bz1941941 created

# knp resource is created and no egress rule to 0.0.0.0/0
is created:

$ oc get knp/np-bz1941941 -o json | jq .spec
{                                                                                                                                                                                                                                             
  "egressSgRules": [                                                                                                                                                                                                                          
    {                                                                                                                                                                                                                                         
      "sgRule": {                                                                                                                                                                                                                             
        "description": "Kuryr-Kubernetes NetPolicy SG rule",                                                                                                                                                                                  
        "direction": "egress",                                                                                                                                                                                                                
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.0.0/14"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    }
  ],
  "ingressSgRules": [
    {
      "namespace": "test",
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.124.0/23"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.196.0.0/16"
      }
    }
  ],
  "podSelector": {
    "matchLabels": {
      "run": "demo"
    }
  },
  "policyTypes": [
    "Egress",
    "Ingress"
  ]
}


# Connectivity tests (pods in other namespace are reachable, outside
access is not):

$ oc rsh -n test pod/demo
~ $ curl 172.30.39.179
demo2: HELLO! I AM ALIVE!!!
~ $ curl 10.128.127.82:8080
demo2: HELLO! I AM ALIVE!!!
~ $ curl www.google.com
^C
~ $ ping www.google.com
PING www.google.com (142.250.179.132) 56(84) bytes of data.
^C
--- www.google.com ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 6174ms


OSP 16.1.5
----------
Steps:

1. Create test and test2 projects both with kuryr/demo pod exposed by a service on port 80:

$ oc new-project test
$ oc run --image quay.io/kuryr/demo demo
$ oc expose pod/demo --port 80 --target-port 8080

$ oc -n test get all -o wide
NAME       READY   STATUS    RESTARTS   AGE     IP              NODE                          NOMINATED NODE   READINESS GATES
pod/demo   1/1     Running   0          2m18s   10.128.127.94   ostest-4sw2h-worker-0-r2dwr   <none>           <none>

NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo   ClusterIP   172.30.226.113   <none>        80/TCP    48s   run=demo

$ oc new-project test2
$ oc run --image quay.io/kuryr/demo demo2
$ oc expose pod/demo2 --port 80 --target-port 8080

$ oc -n test2 get all -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP              NODE                          NOMINATED NODE   READINESS GATES
pod/demo2   1/1     Running   0          26m   10.128.129.57   ostest-4sw2h-worker-0-qgk7v   <none>           <none>

NAME            TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo2   ClusterIP   172.30.117.176   <none>        80/TCP    26m   run=demo2


2. Apply np on demo pod in test project:

$ oc project test
$ oc apply -f np_bz1941941.yaml
networkpolicy.networking.k8s.io/np-bz1941941 created

# knp resource is created and no egress rule to 0.0.0.0/0
is created:

$ oc get knp/np-bz1941941 -o json | jq .spec
{
  "egressSgRules": [
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.0.0/14"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    }
  ],
  "ingressSgRules": [
    {
      "namespace": "test",
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.126.0/23"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.196.0.0/16"
      }
    }
  ],
  "podSelector": {
    "matchLabels": {
      "run": "demo"
    }
  },
  "policyTypes": [
    "Egress",
    "Ingress"
  ]
}


# Connectivity tests (pods in other namespace are reachable, outside
access is not):

$ oc rsh -n test pod/demo
~ $ curl 172.30.117.176
demo2: HELLO! I AM ALIVE!!!
~ $ curl 10.128.129.57:8080
demo2: HELLO! I AM ALIVE!!!
~ $ curl www.google.com
^C
~ $ ping www.google.com
PING www.google.com (142.250.179.164) 56(84) bytes of data.
^C
--- www.google.com ping statistics ---
36 packets transmitted, 0 received, 100% packet loss, time 35864ms


Kubernetes NP tests passed as well.

Comment 7 errata-xmlrpc 2021-05-19 15:15:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.7.11 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1550