Bug 1542855 - upgrade control plane failed at play "Ensure the requested version packages are available" when specify openshift_additional_repos
Summary: upgrade control plane failed at play "Ensure the requested version packages a...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cluster Version Operator
Version: 3.9.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.9.0
Assignee: Michael Gugino
QA Contact: Weihua Meng
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-02-07 07:57 UTC by Weihua Meng
Modified: 2018-03-28 14:26 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-03-28 14:26:32 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0489 0 None None None 2018-03-28 14:26:55 UTC

Description Weihua Meng 2018-02-07 07:57:09 UTC
Description of problem:
upgrade control plane failed at play "Ensure the requested version packages are available" when specify openshift_additional_repos in hosts file.

The check is targetedd for all hosts(masters+nodes) in task [openshift_version : Get available atomic-openshift version], but only masters group ran task [Update repos on upgrade hosts] during upgrade master phase.

Version-Release number of the following components:
openshift-ansible-3.9.0-0.38.0.git.0.57e1184.el7.noarch
ansible-2.4.2.0-2.el7.noarch

How reproducible:
Always

Steps to Reproduce:
1. specify openshift_additional_repos=[...] in hosts file
2. upgrade OCP v3.7 to v3.9 by openshift-ansible
# ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/upgrades/v3_9/upgrade_control_plane.yml

Actual results:

TASK [include_role] ***********************************************************************************************************************************************************************************************
task path: /usr/share/ansible/openshift-ansible/playbooks/init/version.yml:31

TASK [openshift_version : Check openshift_version for rpm installation] *******************************************************************************************************************************************
task path: /usr/share/ansible/openshift-ansible/roles/openshift_version/tasks/masters_and_nodes.yml:5
included: /usr/share/ansible/openshift-ansible/roles/openshift_version/tasks/check_available_rpms.yml for hostxxx.redhat.com, hostxxx.redhat.com, hostxxx.redhat.com, hostxxx.redhat.com, hostxxx.redhat.com, hostxxx.redhat.com

TASK [openshift_version : Get available atomic-openshift version]

TASK [openshift_version : fail] ***********************************************************************************************************************************************************************************
task path: /usr/share/ansible/openshift-ansible/roles/openshift_version/tasks/check_available_rpms.yml:8
skipping: [hostxxx.redhat.com] => {
    "changed": false, 
    "skip_reason": "Conditional result was False"
}
skipping: [hostxxx.redhat.com] => {
    "changed": false, 
    "skip_reason": "Conditional result was False"
}
fatal: [hostxxx.redhat.com]: FAILED! => {
    "changed": false, 
    "msg": "Package atomic-openshift not found"
}
fatal: [hostxxx.redhat.com]: FAILED! => {
    "changed": false, 
    "msg": "Package atomic-openshift not found"
}
fatal: [hostxxx.redhat.com]: FAILED! => {
    "changed": false, 
    "msg": "Package atomic-openshift not found"
}
fatal: [hostxxx.redhat.com]: FAILED! => {
    "changed": false, 
    "msg": "Package atomic-openshift not found"
}


Failure summary:


  1. Hosts:    hostxxx.redhat.com, hostxxx.redhat.com, hostxxx.redhat.com, hostxxx.redhat.com
     Play:     Ensure the requested version packages are available.
     Task:     openshift_version : fail
     Message:  Package atomic-openshift not found


Expected results:
Upgrade succeeds

Comment 4 Michael Gugino 2018-02-08 20:12:22 UTC
I don't believe the situation as reported is possible.

0 non-master nodes should have tasks run against them during upgrade_control_plane.

The logging provided shows the following playflow:

'Examine etcd serving certificate SAN'

Followed by

'Initialization Checkpoint Start'

This is not possible.

I suspect the installer host is corrupted in some way.

Comment 7 Weihua Meng 2018-02-09 02:38:40 UTC
Hi Michael
I set up a new ansible host, did the same thing with openshift-ansible-3.9.0-0.38.0, with non-ha this time.
Meet the same issue.
inventory file and upgrade log attached.

Comment 8 Michael Gugino 2018-02-09 15:38:38 UTC
@Weihua,

Thanks for uploading the new log.  It looks like a change was made recently, it was really hard to find.  Thank you for your patience.

PR Created: https://github.com/openshift/openshift-ansible/pull/7088

Comment 9 Weihua Meng 2018-02-17 02:16:30 UTC
Fixed.
openshift-ansible-3.9.0-0.45.0.git.0.05f6826.el7.noarch

Comment 11 Weihua Meng 2018-02-21 02:26:48 UTC
Fixed.
openshift-ansible-3.9.0-0.45.0.git.0.05f6826.el7.noarch

Comment 14 errata-xmlrpc 2018-03-28 14:26:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489


Note You need to log in before you can comment on or make changes to this bug.