Bug 1502866 – 3.6 Nodes will not start with 3.7 master

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1502866 - 3.6 Nodes will not start with 3.7 master

Summary:	3.6 Nodes will not start with 3.7 master

Status:	CLOSED ERRATA

Aliases:	None

Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Networking (Show other bugs)
Sub Component:
Version:	3.7.0
Hardware:	Unspecified Unspecified

Priority	urgent Severity unspecified
Target Milestone:	---
Target Release:	3.7.0
Assigned To:	Dan Winship
QA Contact:	Meng Bo
Docs Contact:

URL:
Whiteboard:
Keywords:	NeedsTestCase

Duplicates:	1499040 (view as bug list)
Depends On:
Blocks:
	Show dependency tree / graph

Reported:	2017-10-16 17:26 EDT by Eric Paris
Modified:	2017-11-28 17:17 EST (History)
CC List:	6 users (show)

See Also:
Fixed In Version:
Doc Type:	No Doc Update
Doc Text:	undefined
Story Points:	---
Clone Of:
Environment:
Last Closed:	2017-11-28 17:17:21 EST
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
oc get clusternetwork default -o yaml (373 bytes, text/plain) 2017-10-16 17:29 EDT, Eric Paris	no flags	Details
journal of failed node start (5.13 KB, text/plain) 2017-10-16 17:29 EDT, Eric Paris	no flags	Details
default clusternetwork after manual intervention (416 bytes, text/plain) 2017-10-16 17:42 EDT, Eric Paris	no flags	Details
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2017:3188	normal	SHIPPED_LIVE	Moderate: Red Hat OpenShift Container Platform 3.7 security, bug, and enhancement update	2017-11-28 21:34:54 EST

Groups: None (edit)

Description Eric Paris 2017-10-16 17:26:00 EDT

The crashloop over and over.

error: SDN node startup failed: failed to get network information: failed to parse ClusterNetwork CIDR : invalid CIDR address:

danw indicated he sees the problem.

Comment 1 Eric Paris 2017-10-16 17:27:23 EDT

17:26 < danw> "oc edit clusternetwork default" and set the "network" and "hostsubnetlength" fields to match the values in clusterNetworks
17:26 < danw> it *might* re-break it every time you restart master though

Comment 2 Eric Paris 2017-10-16 17:29 EDT

Created attachment 1339473 [details]
oc get clusternetwork default -o yaml

Comment 3 Eric Paris 2017-10-16 17:29 EDT

Created attachment 1339474 [details]
journal of failed node start

Comment 4 Eric Paris 2017-10-16 17:42 EDT

Created attachment 1339478 [details]
default clusternetwork after manual intervention

note the trailing 'network' and 'hostsubnetlength' fields which match the CIDR.

Comment 5 Dan Winship 2017-10-16 18:29:55 EDT

https://github.com/openshift/origin/pull/16897

Comment 6 Ravi Sankar 2017-10-16 20:49:05 EDT

*** Bug 1499040 has been marked as a duplicate of this bug. ***

Comment 8 Yan Du 2017-10-24 00:32:42 EDT

Setup 3.6 env 
openshift v3.6.173.0.56
kubernetes v1.6.1+5115d708d7

upgrade master to 3.7
openshift v3.7.0-0.176.0
kubernetes v1.7.6+a08f5eeb62

3.6 nodes could start normally with 3.7 master.

Comment 9 Yan Du 2017-10-24 00:36:45 EDT

And the clusternetowrk works well too.
# oc get node
NAME                                               STATUS                     AGE       VERSION
host-8-xxx.host.centralci.eng.rdu2.redhat.com   Ready,SchedulingDisabled   2h        v1.7.6+a08f5eeb62
host-8-yyy.host.centralci.eng.rdu2.redhat.com   Ready                      2h        v1.6.1+5115d708d7
# oc get clusternetwork
NAME      CLUSTER NETWORKS   SERVICE NETWORK   PLUGIN NAME
default   10.128.0.0/14:9    172.30.0.0/16     redhat/openshift-ovs-subnet

Comment 12 errata-xmlrpc 2017-11-28 17:17:21 EST

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188

Note You need to log in before you can comment on or make changes to this bug.