Bug 1507433

Summary:	oadm migrate storage failed in upgrade process
Product:	OpenShift Container Platform	Reporter:	ge liu <geliu>
Component:	Networking	Assignee:	Ben Bennett <bbennett>
Status:	CLOSED ERRATA	QA Contact:	Meng Bo <bmeng>
Severity:	medium	Docs Contact:
Priority:	unspecified
Version:	3.7.0	CC:	aos-bugs, bbennett, jokerman, mmccomas
Target Milestone:	---
Target Release:	3.7.0
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	No Doc Update
Doc Text:	undefined	Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-12-18 13:23:26 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description ge liu 2017-10-30 09:18:52 UTC

Description of problem:

upgrade ocp v3.7.0-0.143.2 to v3.7.0-0.184.0, 'oadm migrate storage' met error below:

["error: clusternetworks/default: ClusterNetwork \"default\" is invalid: [network: Invalid value: \"\": network must be identical to clusterNetworks[0].cidr, hostsubnetlength: Invalid value: 0x0: hostsubnetlength must be identical to clusterNetworks[0].hostSubnetLength]", "summary: total=1031 errors=1 ignored=0 unchanged=964 migrated=66", "info: to rerun only failing resources, add --include=clusternetworks", "error: 1 resources failed to migrate"]

run the command by manual, still met error:

# oadm migrate storage --confirm --include=clusternetworks
error:      clusternetworks/default: ClusterNetwork "default" is invalid: [network: Invalid value: "": network must be identical to clusterNetworks[0].cidr, hostsubnetlength: Invalid value: 0x0: hostsubnetlength must be identical to clusterNetworks[0].hostSubnetLength]
summary: total=1 errors=1 ignored=0 unchanged=0 migrated=0
info: to rerun only failing resources, add --include=clusternetworks
error: 1 resources failed to migrate


How reproducible:
Always

Steps to Reproduce:

1. steup ocp HA env in aws cluster, upgrade cluster version from v3.7.0-0.143.2 to v3.7.0-0.184.0

==> met error as description

2. run the command by manual, still met error:

# oadm migrate storage --confirm --include=clusternetworks

Actual results:

oadm migrate storage failed in upgrade process

Expected results:

oadm migrate storage succeed in upgrade process

Comment 1 Scott Dodson 2017-10-30 19:56:17 UTC

Perhaps this will be addressed via https://github.com/openshift/openshift-ansible/pull/5864

Comment 2 liujia 2017-10-31 08:49:14 UTC

Hit the issue when do minor version upgrade.

Comment 3 Ben Bennett 2017-10-31 14:53:05 UTC

It would not since the old installer didn't write the new form, and somehow they have the new form (without the old). 


However, this is an upgrade from a dev release to a dev release.

The problem is that we did not require that for some releases in the 3.7 dev cycle we didn't set:

networkConfig:
  clusterNetworkCIDR: 10.128.0.0/14

And we didn't require it.  We realized there were problems with that and changed the code so that it was required.  This would not be an issue for any upgrade from a released version to a released version.

So, I think you need to manually fix it, and it's not really something we need to fix.

Comment 4 ge liu 2017-11-01 06:27:39 UTC

@bbennett, could u help to list the manual steps that could fix this problem? 
and I also think about that online env have requirement that upgrade from a dev release to a dev release, does it met this problem also?
thx

Comment 5 ge liu 2017-11-01 06:37:03 UTC

@bbennett, we have another bug(1489168) need to do upgrade from a dev release to a dev release, so the manual workaround is necessary to us if we want not fix this problem, thx

Comment 6 liujia 2017-11-03 10:52:08 UTC

Verified on openshift-ansible-3.7.0-0.191.0.git.0.bc2ff60.el7.noarch


Do upgrade from v3.7.0-0.190.0 to v3.7.0-0.191.0, upgrade succeed.

Comment 10 errata-xmlrpc 2017-12-18 13:23:26 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:3464