Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1451881 - headless services causes SDN initialization failure for master-controllers when network change.
headless services causes SDN initialization failure for master-controllers wh...
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking (Show other bugs)
3.5.0
Unspecified Unspecified
high Severity high
: ---
: 3.7.0
Assigned To: jtanenba
Meng Bo
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-05-17 14:05 EDT by Ryan Howe
Modified: 2017-11-28 16:55 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: When initializing openshift sdn fail to allow a nil as a valid service IP Consequence: openshift sdn failed to initialize causing master node to not fail when using headless services Fix: Allow nil for a valid value of srv.Spec.ClusterIP Result: Openshift sdn properly starts when master node is restarted
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-11-28 16:55:46 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 3035961 None None None 2017-05-17 14:30 EDT
Github openshift/ose/pull/722 None None None 2017-05-31 13:54 EDT
Red Hat Product Errata RHSA-2017:3188 normal SHIPPED_LIVE Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update 2017-11-28 21:34:54 EST

  None (edit)
Description Ryan Howe 2017-05-17 14:05:58 EDT
Description of problem:

When changes are made to SDN plugin, the master controller will fail to start when there are headless services in the cluster. 

Error Message: 

[run_components.go:384] SDN initialization failed: Error: Existing service with IP: None is not part of service network: 172.30.0.0/16

https://kubernetes.io/docs/concepts/services-networking/service/#headless-services


Code that checks and errors out. 

https://github.com/openshift/openshift-sdn/blob/master/plugins/osdn/master.go#L111-L120

Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Create headless service 

# oc create -f - << EOF
apiVersion: v1
kind: Service
metadata:
  name: hello-openshift
spec:
  selector:
    name: hello-openshift
  portalIP: None
  clusterIP: None
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
EOF
```

2. Change sdn plugin 

Master
  # sed -i 's/openshift-ovs-subnet/openshift-ovs-multitenant/g' /etc/origin/master/master-config.yaml
  # sed -i 's/openshift-ovs-subnet/openshift-ovs-multitenant/g' /etc/origin/node/node-config.yaml
 
Node
  # sed -i 's/openshift-ovs-subnet/openshift-ovs-multitenant/g' /etc/origin/node/node-config.yaml


3. Restart masters and nodes 


Actual results:

SDN initialization failed: Error: Existing service with IP: None is not part of service network: 172.30.0.0/16


Expected results:
The SDN not to fail with hitting headless services 


Additional info:

The metrics and logging deployer configures headless services that have 

  clusterIP: None

Example: 

https://github.com/openshift/origin-metrics/blob/master/deployer/templates/hawkular-cassandra.yaml#L47
Comment 2 jtanenba 2017-05-31 13:54:33 EDT
This was fixed in 3.5 in PR722 (https://github.com/openshift/ose/pull/722) Commit #2
Comment 4 Yan Du 2017-06-05 06:18:46 EDT
Test on OCP 3.5 env
openshift v3.5.5.23
kubernetes v1.5.2+43a9be4

SDN works well after changing network with headless service.
Comment 8 errata-xmlrpc 2017-11-28 16:55:46 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188

Note You need to log in before you can comment on or make changes to this bug.