Bug 1958103 - [kuryr] Egress network policy with namespaceSelector in Kuryr behaves differently than in OVN-Kubernetes
Summary: [kuryr] Egress network policy with namespaceSelector in Kuryr behaves differe...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.7
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 4.6.z
Assignee: Michał Dulko
QA Contact: Jon Uriarte
URL:
Whiteboard:
Depends On: 1941941
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-07 08:11 UTC by OpenShift BugZilla Robot
Modified: 2021-06-08 13:54 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-08 13:54:19 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift kuryr-kubernetes pull 511 0 None open [release-4.6] Bug 1958103: Include service subnet to be open for namespaceSelector set to all. 2021-05-13 10:29:32 UTC
Red Hat Product Errata RHBA-2021:2157 0 None None None 2021-06-08 13:54:39 UTC

Comment 1 Michał Dulko 2021-05-07 08:56:46 UTC
Back to assigned, it also needs https://github.com/openshift/kuryr-kubernetes/pull/511.

Comment 6 Jon Uriarte 2021-06-03 10:40:57 UTC
Verified in 4.6.0-0.nightly-2021-05-31-074224 on top of OSP 13.0.15 (2021-03-24.1) with amphora
provider and and on top of OSP 16.1.5 (RHOS-16.1-RHEL-8-20210323.n.0) with ovn octavia provider.

SG rules generated by below NP resource definition allow traffic to other namespaces but not
to the outside:

$ cat np_bz1958103.yaml 
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: np-bz1958103
spec:
  podSelector:
    matchLabels:
      run: demo
  policyTypes:
  - Egress
  - Ingress
  ingress:
  - from:
    - podSelector: {}
  egress:
  - to:
    - namespaceSelector: {}
    

OSP 13.0.15
-----------
Steps:

1. Create test and test2 projects both with kuryr/demo pod exposed by a service on port 80:

$ oc new-project test
$ oc run --image quay.io/kuryr/demo demo
$ oc expose pod/demo --port 80 --target-port 8080

$ oc -n test get all -o wide
NAME       READY   STATUS    RESTARTS   AGE   IP               NODE                          NOMINATED NODE   READINESS GATES
pod/demo   1/1     Running   0          39s   10.128.117.184   ostest-q5lcm-worker-0-qnktl   <none>           <none>

NAME           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo   ClusterIP   172.30.224.192   <none>        80/TCP    5s    run=demo

$ oc new-project test2
$ oc run --image quay.io/kuryr/demo demo2
$ oc expose pod/demo2 --port 80 --target-port 8080

$ oc -n test2 get all -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP               NODE                          NOMINATED NODE   READINESS GATES
pod/demo2   1/1     Running   0          43s   10.128.119.134   ostest-q5lcm-worker-0-26xt2   <none>           <none>

NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo2   ClusterIP   172.30.70.197   <none>        80/TCP    5s    run=demo2


2. Apply np on demo pod in test project:

$ oc project test
$ oc apply -f np_bz1958103.yaml
networkpolicy.networking.k8s.io/np-bz1958103 created

# knp resource is created and no egress rule to 0.0.0.0/0
is created:

$ oc get knp/np-bz1958103 -o json | jq .spec
{                                                                                                                                                                                                                                             
  "egressSgRules": [                                                                                                                                                                                                                          
    {                                                                                                                                                                                                                                         
      "sgRule": {                                                                                                                                                                                                                             
        "description": "Kuryr-Kubernetes NetPolicy SG rule",                                                                                                                                                                                  
        "direction": "egress",                                                                                                                                                                                                                
        "ethertype": "IPv4",                                                                                                                                                                                                                  
        "remote_ip_prefix": "10.128.0.0/14"                                                                                                                                                                                                   
      }                                                                                                                                                                                                                                       
    },                                                                                                                                                                                                                                        
    {                                                                                                                                                                                                                                         
      "sgRule": {                                                                                                                                                                                                                             
        "description": "Kuryr-Kubernetes NetPolicy SG rule",                                                                                                                                                                                  
        "direction": "egress",                                                                                                                                                                                                                
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    }
  ],
  "ingressSgRules": [
    {
      "namespace": "test",
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.116.0/23"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.196.0.0/16"
      }
    }
  ],
  "podSelector": {
    "matchLabels": {
      "run": "demo"
    }
  },
  "policyTypes": [
    "Egress",
    "Ingress"
  ]
}


# Connectivity tests (pods in other namespace are reachable, outside
access is not):

$ oc rsh -n test pod/demo
~ $ curl 172.30.70.197
demo2: HELLO! I AM ALIVE!!!
~ $ curl 10.128.119.134:8080
demo2: HELLO! I AM ALIVE!!!
~ $ curl www.google.com
^C
~ $ ping www.google.com
PING www.google.com (172.217.164.164) 56(84) bytes of data.
^C
--- www.google.com ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5160ms


OSP 16.1.5
----------
Steps:

1. Create test and test2 projects both with kuryr/demo pod exposed by a service on port 80:

$ oc new-project test
$ oc run --image quay.io/kuryr/demo demo
$ oc expose pod/demo --port 80 --target-port 8080

$ oc -n test get all -o wide
NAME       READY   STATUS    RESTARTS   AGE    IP              NODE                          NOMINATED NODE   READINESS GATES
pod/demo   1/1     Running   0          100s   10.128.117.14   ostest-slbx7-worker-0-x2xqb   <none>           <none>

NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo   ClusterIP   172.30.232.18   <none>        80/TCP    13s   run=demo

$ oc new-project test2
$ oc run --image quay.io/kuryr/demo demo2
$ oc expose pod/demo2 --port 80 --target-port 8080

$ oc -n test2 get all -o wide
NAME        READY   STATUS    RESTARTS   AGE   IP               NODE                          NOMINATED NODE   READINESS GATES
pod/demo2   1/1     Running   0          23s   10.128.119.150   ostest-slbx7-worker-0-x2xqb   <none>           <none>

NAME            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE   SELECTOR
service/demo2   ClusterIP   172.30.62.165   <none>        80/TCP    4s    run=demo2


2. Apply np on demo pod in test project:

$ oc project test
$ oc apply -f np_bz1958103.yaml
networkpolicy.networking.k8s.io/np-bz1958103 created

# knp resource is created and no egress rule to 0.0.0.0/0
is created:

$ oc get knp/np-bz1958103 -o json | jq .spec
{
  "egressSgRules": [
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.0.0/14"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "egress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.0.0/15"
      }
    }
  ],
  "ingressSgRules": [
    {
      "namespace": "test",
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.128.116.0/23"
      }
    },
    {
      "sgRule": {
        "description": "Kuryr-Kubernetes NetPolicy SG rule",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "10.196.0.0/16"
      }
    },
    {
      "sgRule": {
        "description": "Allow traffic from local namespace service demo",
        "direction": "ingress",
        "ethertype": "IPv4",
        "remote_ip_prefix": "172.30.232.18"
      }
    }
  ],
  "podSelector": {
    "matchLabels": {
      "run": "demo"
    }
  },
  "policyTypes": [
    "Egress",
    "Ingress"
  ]
}


# Connectivity tests (pods in other namespace are reachable, outside
access is not):

~ $ curl 172.30.62.165
demo2: HELLO! I AM ALIVE!!!
~ $ curl 10.128.119.150:8080
demo2: HELLO! I AM ALIVE!!!
~ $ curl www.google.com
^C
~ $ ping www.google.com
PING www.google.com (172.217.164.164) 56(84) bytes of data.
^C
--- www.google.com ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5111ms


Filed https://bugzilla.redhat.com/show_bug.cgi?id=1967540 for duplicated egress rules in knp object.

Comment 8 errata-xmlrpc 2021-06-08 13:54:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.32 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2157


Note You need to log in before you can comment on or make changes to this bug.