Bug 1700431 - EgressIP doesn't work with NetworkPolicy unless traffic from default project is allowed
Summary: EgressIP doesn't work with NetworkPolicy unless traffic from default project ...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: 4.2.0
Assignee: Dan Winship
QA Contact: Weibin Liang
URL:
Whiteboard:
Depends On:
Blocks: 1741477 1741499 1766583
TreeView+ depends on / blocked
 
Reported: 2019-04-16 14:36 UTC by Juan Luis de Sousa-Valadas
Modified: 2019-10-29 13:04 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Egress IPs did not work correctly in namespaces with restrictive NetworkPolicies. Consequence: Pods that accepted traffic only from specific sources would not be able to send egress traffic via egress IPs, because the response from the external server would be mistakenly rejected by their NetworkPolicies. Fix: Replies from egress traffic are now correctly recognized as replies rather than as new connections. Result: Egress IPs and NetworkPolicy work together.
Clone Of:
: 1741477 1741499 (view as bug list)
Environment:
Last Closed: 2019-10-16 06:28:06 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github openshift sdn pull 19 'None' closed Bug 1700431: Pass egress IP packets to conntrack 2020-10-27 18:11:17 UTC
Red Hat Product Errata RHBA-2019:2922 None None None 2019-10-16 06:28:21 UTC

Description Juan Luis de Sousa-Valadas 2019-04-16 14:36:59 UTC
Description of problem:
Customer reports when they use networkPolicy combined with egressIP unless


Version-Release number of selected component (if applicable):
3.11

How reproducible:
Always

Steps to Reproduce:
1. Create a project X using egressIP
2. Add egressIP to node A
3. Create a pod in project X which is *not* running on node A.
4. Create a networkPolicy which only allows traffic from itself:

- apiVersion: extensions/v1beta1
  kind: NetworkPolicy
  metadata:
    name: deny-ingress-from-other-namespaces
  spec:
    ingress:
    - from:
      - podSelector: {}
    podSelector: {}
    policyTypes:
    - Ingress
5. Go to the pod in project X and try to reach a resource outside OpenShift. Traffic is dropped.
6. Create a rule that allows traffic from the default project (assumes the default netnamespace netid equals 0 and the project has a label project=default)
- apiVersion: extensions/v1beta1
  kind: NetworkPolicy
  metadata:
    name: allow from default
  spec:
    ingress:
    - from:
      - podSelector: {}
      - namespaceSelector:
          matchLabels:
            project: default
    podSelector: {}
    policyTypes:
    - Ingress
Traffic works
7. Completely remove every networkPolicy. Traffic also works

Actual results:
Packet is dropped somewhere.

Expected results:
Packet goes through the egressIP and comes back

Additional info:
Additionally the customer has an egressNetworkPolicy which allows traffic to destination and denies traffic by default. I believe this egressNetworkPolicy is unrelated to the issue.

Comment 7 Juan Luis de Sousa-Valadas 2019-04-16 16:34:20 UTC
Software versions:

atomic-openshift-node-3.11.88-1.git.0.47f4e98.el7.x86_64
registry.redhat.io/openshift3/ose-node                          v3.11.16            074bf04571e2        6 months ago        1.15 GB

Comment 9 Dan Winship 2019-05-02 20:55:27 UTC
Yeah, can confirm that this is happening... it's not supposed to; conntrack should be recognizing the return traffic as being a reply to the outbound connection, and so it should be accepted regardless of policies. But that's not working.

Comment 17 zhaozhanqi 2019-08-23 02:23:03 UTC
I tried this on 4.2 version 4.2.0-0.nightly-2019-08-22-043819.it's working well, see below. so it's not an issue for 4.2. close this bug

#oc get netnamespaces z1
NAME   NETID     EGRESS IPS
z1     3625249   [139.178.76.100]

#oc get hostsubnet control-plane-0
NAME              HOST              HOST IP         SUBNET          EGRESS CIDRS   EGRESS IPS
control-plane-0   control-plane-0   139.178.76.22   10.128.0.0/23                  [139.178.76.100]

$ oc get networkpolicy -n z1 -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
  kind: NetworkPolicy
  metadata:
    creationTimestamp: "2019-08-23T02:08:49Z"
    generation: 1
    name: deny-ingress-from-other-namespaces
    namespace: z1
    resourceVersion: "241251"
    selfLink: /apis/extensions/v1beta1/namespaces/z1/networkpolicies/deny-ingress-from-other-namespaces
    uid: f25ab63b-c54a-11e9-951d-0050568b11fb
  spec:
    ingress:
    - from:
      - podSelector: {}
    podSelector: {}
    policyTypes:
    - Ingress
kind: List
metadata:
  resourceVersion: ""
  selfLink: ""

$ oc get pod -n z1
NAME            READY   STATUS    RESTARTS   AGE
test-rc-4j2hk   1/1     Running   0          14m
test-rc-5rhqn   1/1     Running   0          14m

$ oc rsh test-rc-4j2hk
~ $ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=54 time=1.97 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=54 time=1.33 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=54 time=1.41 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=54 time=1.38 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=54 time=1.37 ms

Comment 18 Dan Winship 2019-08-24 13:55:42 UTC
@zhaozhanqi FYI the new github bugzilla-checking automation is picky about bugzilla statuses and will only let me backport this if it's "CLOSED ERRATA" or "VERIFIED". Fixing that... Hm... but it won't let me go right from CLOSED to VERIFIED...

Comment 19 errata-xmlrpc 2019-10-16 06:28:06 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2922


Note You need to log in before you can comment on or make changes to this bug.