Bug 1583255 - Entire cluster goes to NotReady using a NetworkPolicy that contains an ingress ipBlock section
Summary: Entire cluster goes to NotReady using a NetworkPolicy that contains an ingres...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 3.10.0
Assignee: Dan Winship
QA Contact: Hongan Li
URL:
Whiteboard:
Depends On:
Blocks: 1585243
TreeView+ depends on / blocked
 
Reported: 2018-05-28 15:05 UTC by David Caldwell
Modified: 2018-07-30 19:17 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: OpenShift's default network plugin has not been updated to implement the new NetworkPolicy features introduced upstream in Kubernetes (policies for controlling egress, and policies based on IP addresses rather than pods or namespaces). Consequence: In 3.9, creating a NetworkPolicy with an "ipBlock" section would cause nodes to crash, and creating a NetworkPolicy that contained only "egress" rules would erroneously cause ingress traffic to be blocked. Fix: The code is now aware of the unsupported NetworkPolicy features (though it does not yet implement them). Result: If a NetworkPolicy contains "ipBlock" rules, those rules are ignored. (This may cause the policy to be treated as "deny all" if the "ipBlock" rule was the only rule in the policy). If a NetworkPolicy contains only "egress" rules, it is ignored completely (and does not affect ingress).
Clone Of:
: 1585243 (view as bug list)
Environment:
Last Closed: 2018-07-30 19:16:51 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Origin (Github) 19869 0 None None None 2018-05-29 16:26:08 UTC
Red Hat Product Errata RHBA-2018:1816 0 None None None 2018-07-30 19:17:40 UTC

Description David Caldwell 2018-05-28 15:05:50 UTC
Description of problem:
If a NetworkPolicy is applied that contains an ingress ipBlock section with cidr range, each node goes into NotReady status. It seems also that core dumps are also generated repeatedly in /var/lib/origin until the disk space is exhausted.

Version-Release number of selected component (if applicable):
OCP 3.9.25
OCP 3.9.27

How reproducible:
Install OpenShift with the networkpolicy plugin. Create a test namespace. Create and apply a networkpolicy such as:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: test
  namespace: test
spec:
 podSelector: {}
 ingress:
  - from:
    - ipBlock:
        cidr: 0.0.0.0/0


Steps to Reproduce:
1. Install OpenShift with the openshift-ovs-networkpolicy SDN plugin.
2. Create a test namespace.
3. Create a syntactically-valid NetworkPolicy containing ingress ipBlock section such as the example above.
4. oc create -f <test_network_policy_name>

Actual results:
All nodes in cluster go to NotReady. Core dumps generated repeatedly.

Expected results:
Network policy to be applied without effecting node status in any way.

Additional info:

Comment 4 Dan Winship 2018-05-29 15:40:14 UTC
while we should not crash, you should be aware that OCP 3.10 does not support NetworkPolicy ipBlocks anyway, so the *expected* result of

  kind: NetworkPolicy
  apiVersion: networking.k8s.io/v1
  metadata:
    name: test
  spec:
    podSelector: {}
    ingress:
    - from:
      - ipBlock:
          cidr: 0.0.0.0/0

is that it will be interpreted the same as:

  kind: NetworkPolicy
  apiVersion: networking.k8s.io/v1
  metadata:
    name: test
  spec:
    podSelector: {}

Comment 5 Dan Winship 2018-05-29 16:09:42 UTC
new QE tests:

1. A policy that contains both ingress and egress rules behaves exactly the same as it would if you removed the "egress" section entirely. Eg, this:

  kind: NetworkPolicy
  apiVersion: networking.k8s.io/v1
  metadata:
    name: test-ignore-egress-1
  spec:
    podSelector: {}
    ingress:
    - from:
      - ports:
        - port: 80
    egress:
    - to:
      - ports:
        - port: 100

allows incoming connections to port 80 on any pod in the namespace, and does not affect egress traffic in any way.

2. A policy with 'policyTypes: ["Egress"]' is ignored. Eg, this policy:

  kind: NetworkPolicy
  apiVersion: networking.k8s.io/v1
  metadata:
    name: egress-default-deny
  spec:
    policyTypes: ["Egress"]
    podSelector: {}

would have *no effect* on a namespace, and in particular it does not result in a default deny of ingress.

3. A policy with an ingress rule with an "ipBlock" element behaves like it would if that rule was removed. So eg:

  kind: NetworkPolicy
  apiVersion: networking.k8s.io/v1
  metadata:
    name: test-podSelector-and-ipBlock
  spec:
    podSelector: {}
    ingress:
    - from:
      - ipBlock:
          cidr: 0.0.0.0/0
      - podSelector:
          matchLabels:
            type: red

Behaves the same as

  kind: NetworkPolicy
  apiVersion: networking.k8s.io/v1
  metadata:
    name: test-podSelector-only
  spec:
    podSelector: {}
    ingress:
    - from:
      - podSelector:
          matchLabels:
            type: red

and a policy with *only* an ipBlock (as in the original example here) becomes a "deny all" policy.

Comment 6 Dan Winship 2018-05-29 16:26:08 UTC
https://github.com/openshift/origin/pull/19869

Comment 10 Hongan Li 2018-06-06 08:26:01 UTC
Verified in v3.10.0-0.58.0 and the issue has been fixed.

Tested all cases mentioned in comment 4 and comment 5, the test results are as expected.

OS: Red Hat Enterprise Linux Atomic Host release 7.5
kernel: Linux qe-np-master-etcd-1 3.10.0-862.2.3.el7.x86_64 #1 SMP Mon Apr 30 12:37:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

Comment 12 errata-xmlrpc 2018-07-30 19:16:51 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816


Note You need to log in before you can comment on or make changes to this bug.