Bug 1549971

Summary: upgrade failed due to unsupported scheduler predicate NoVolumeNodeConflict from kube-1.9
Product: OpenShift Container Platform Reporter: Weihua Meng <wmeng>
Component: Cluster Version OperatorAssignee: Michael Gugino <mgugino>
Status: CLOSED ERRATA QA Contact: Weihua Meng <wmeng>
Severity: high Docs Contact:
Priority: high    
Version: 3.9.0CC: aos-bugs, jiajliu, jokerman, mmccomas
Target Milestone: ---   
Target Release: 3.9.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-03-28 14:30:19 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Weihua Meng 2018-02-28 08:08:44 UTC
Description of problem:
Upgrade failed due to unsupported scheduler predicate NoVolumeNodeConflict
resulting in master controllers service cannot start

Version-Release number of the following components:
openshift-ansible-3.9.1-1.git.0.9862628.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1. Upgrade from ocp v3.7 to v3.9


Actual results:
# systemctl status atomic-openshift-master-controllers.service 
● atomic-openshift-master-controllers.service - Atomic OpenShift Master Controllers
   Loaded: loaded (/usr/lib/systemd/system/atomic-openshift-master-controllers.service; enabled; vendor preset: disabled)
   Active: activating (auto-restart) (Result: exit-code) since 三 2018-02-28 01:55:05 EST; 1s ago
     Docs: https://github.com/openshift/origin
  Process: 32818 ExecStart=/usr/bin/openshift start master controllers --config=${CONFIG_FILE} $OPTIONS (code=exited, status=255)
 Main PID: 32818 (code=exited, status=255)

2月 28 01:55:05 host-172-16-120-76 atomic-openshift-master-controllers[32818]: I0228 01:55:05.363449   32818 factory.go:857] Registering predicate: NoVolumeNodeConflict
2月 28 01:55:05 host-172-16-120-76 atomic-openshift-master-controllers[32818]: F0228 01:55:05.363453   32818 plugins.go:234] Invalid configuration: Predicate type not found for NoVolumeNodeConflict

Expected results:
Master contollers service running after upgrade

Comment 1 Scott Dodson 2018-03-01 14:13:31 UTC
Minor cleanup for the scheduler upgrade, not expected to fix the problem, but related.

https://github.com/openshift/openshift-ansible/pull/7333

Our belief is that ansible is skipping these task for no apparent reason.

Comment 2 Michael Gugino 2018-03-01 19:39:42 UTC
PR updated: 
https://github.com/openshift/openshift-ansible/pull/7333

Cause was stale openshift.common.short_version set by openshift_facts being used inside the lookup plugin for scheduler config.

Comment 4 Weihua Meng 2018-03-03 05:11:27 UTC
Fixed.
openshift-ansible-3.9.2-1.git.0.1a855b3.el7.noarch

# systemctl status atomic-openshift-master-controllers.service -l
● atomic-openshift-master-controllers.service - Atomic OpenShift Master Controllers
   Loaded: loaded (/usr/lib/systemd/system/atomic-openshift-master-controllers.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2018-03-02 23:56:39 EST; 13min ago
     Docs: https://github.com/openshift/origin
 Main PID: 18475 (openshift)

Comment 7 errata-xmlrpc 2018-03-28 14:30:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489