Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1612864

Summary: openshift_upgrade_node_label does not upgrades masters when labels are matched
Product: OpenShift Container Platform Reporter: Jatan Malde <jmalde>
Component: Cluster Version OperatorAssignee: Vadim Rutkovsky <vrutkovs>
Status: CLOSED NOTABUG QA Contact: Gaoyun Pei <gpei>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.9.0CC: aos-bugs, bcankurt, jmalde, jokerman, mmccomas, scuppett, sdodson, vrutkovs
Target Milestone: ---   
Target Release: 3.9.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-22 11:20:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
inventory
none
control plane upgrade playbook logs.
none
control plane upgrade verbose logs none

Description Jatan Malde 2018-08-06 12:32:19 UTC 
Description of problem:

When the label of openshift_upgrade_node_label is set to upgrade the masters in the cluster, the upgrade.yml playbook completes fully and does not upgrade the nodes.


Version-Release number of the following components:

$ rpm -qa | grep ansible
openshift-ansible-3.9.33-1.git.56.19ba16e.el7.noarch
openshift-ansible-roles-3.9.33-1.git.56.19ba16e.el7.noarch
ansible-2.4.6.0-1.el7ae.noarch
openshift-ansible-playbooks-3.9.33-1.git.56.19ba16e.el7.noarch
openshift-ansible-docs-3.9.33-1.git.56.19ba16e.el7.noarch


How reproducible:

Steps to Reproduce:
1.
2.
3.

Actual results:
The correct nodes were listed when checked the verbose logs but the nodes were still node upgraded.

Expected results:
The master node should be upgraded to newer version.

Additional info:
Please attach logs from ansible-playbook with the -vvv flag

The ansible logs are attached in the below comment
Comment 4 Vadim Rutkovsky 2018-08-06 16:02:57 UTC 
Please attach the full inventory
Comment 5 Bahaddin 2018-08-07 10:05:42 UTC 
Hi,

we had to change the hosts stanza for three tasks in the upgrade_node.yml playbook.
oo_nodes_to_upgrade:!oo_masters_to_config - > oo_nodes_to_upgrade:oo_masters_to_config

https://github.com/openshift/openshift-ansible/blob/dab8e8cce33dcd08ec5a03e2f23c94e195ea4054/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml#L3
https://github.com/openshift/openshift-ansible/blob/dab8e8cce33dcd08ec5a03e2f23c94e195ea4054/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml#L12
https://github.com/openshift/openshift-ansible/blob/dab8e8cce33dcd08ec5a03e2f23c94e195ea4054/playbooks/common/openshift-cluster/upgrades/upgrade_nodes.yml#L82

The statement !oo_masters_to_config was causing the playbook to disregard the master nodes, hence a node upgrade was not done although the master nodes were properly selected.
Comment 6 Bahaddin 2018-08-07 10:14:04 UTC 
Created attachment 1473943 [details]
inventory

here's the inventory.
Comment 8 Bahaddin 2018-08-07 12:33:38 UTC 
Created attachment 1473976 [details]
control plane upgrade playbook logs.
Comment 9 Vadim Rutkovsky 2018-08-07 14:17:40 UTC 
(In reply to Bahaddin from comment #8)
> Created attachment 1473976 [details]
> control plane upgrade playbook logs.

We'd need `ansible-playbook -vvv` output to find out more details
Comment 12 Bahaddin 2018-08-09 12:48:17 UTC 
Created attachment 1474672 [details]
control plane upgrade verbose logs

Hello,

I'm attaching the verbose logs..

Best,
Bahaddin
Comment 17 Stephen Cuppett 2018-08-22 11:20:22 UTC 
Instead of 
  playbooks/common/openshift-cluster/upgrades/v3_9/upgrade_control_plane.yml
they should run
  playbooks/byo/openshift-cluster/upgrades/v3_9/upgrade_control_plane.yml

Closing for the moment. If there are issues with the BYO scripts, please reopen with the additional details.