Bug 1760859

+++ This bug was initially created as a clone of Bug #1760665 +++

Description of problem:

cri-o package was not updated during upgrade.

https://github.com/openshift/openshift-ansible/blob/release-4.2/roles/openshift_node/defaults/main.yml#L13
https://github.com/openshift/openshift-ansible/blob/release-4.2/roles/openshift_node/tasks/install.yml#L50


TASK [openshift_node : Install openshift packages] *****************************
Thursday 10 October 2019  16:25:48 +0800 (0:00:00.084)       0:04:02.661 ****** 
changed: [rhel-1.qe-lxia-upg-share-1010.qe.devcluster.openshift.com] => {"ansible_job_id": "919582869957.1459", "attempts": 1, "changed": true, "changes": {"installed": ["openshift-clients-4.2.0", "openshift-hyperkube-4.2.0"]}, "finished": 1, "msg": "", "rc": 0, "results": ["cri-o-1.13.11-0.10.dev.rhaos4.1.gitbdeb2ca.el7.x86_64 providing cri-o is already installed", "Loaded plugins: product-id, search-disabled-repos, subscription-manager\nThis system is not registered with an entitlement server. You can use subscription-manager to register.\nResolving Dependencies\n--> Running transaction check\n---> Package openshift-clients.x86_64 0:4.1.19-201910070609.git.0.6f9924b.el7 will be updated\n---> Package openshift-clients.x86_64 0:4.2.0-201910041700.git.1.c8c7aaa.el7 will be an update\n---> Package openshift-hyperkube.x86_64 0:4.1.19-201910070609.git.0.6f9924b.el7 will be updated\n---> Package openshift-hyperkube.x86_64 0:4.2.0-201910020731.git.0.463c73f.el7 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package              Arch    Version                                Repository\n                                                                           Size\n================================================================================\nUpdating:\n openshift-clients    x86_64  4.2.0-201910041700.git.1.c8c7aaa.el7   aos   17 M\n openshift-hyperkube  x86_64  4.2.0-201910020731.git.0.463c73f.el7   aos   34 M\n\nTransaction Summary\n================================================================================\nUpgrade  2 Packages\n\nTotal download size: 51 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal                                               12 MB/s |  51 MB  00:04     \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n  Updating   : openshift-hyperkube-4.2.0-201910020731.git.0.463c73f.el7.x   1/4 \n  Updating   : openshift-clients-4.2.0-201910041700.git.1.c8c7aaa.el7.x86   2/4 \n  Cleanup    : openshift-hyperkube-4.1.19-201910070609.git.0.6f9924b.el7.   3/4 \n  Cleanup    : openshift-clients-4.1.19-201910070609.git.0.6f9924b.el7.x8   4/4 \n  Verifying  : openshift-clients-4.2.0-201910041700.git.1.c8c7aaa.el7.x86   1/4 \n  Verifying  : openshift-hyperkube-4.2.0-201910020731.git.0.463c73f.el7.x   2/4 \n  Verifying  : openshift-clients-4.1.19-201910070609.git.0.6f9924b.el7.x8   3/4 \n  Verifying  : openshift-hyperkube-4.1.19-201910070609.git.0.6f9924b.el7.   4/4 \n\nUpdated:\n  openshift-clients.x86_64 0:4.2.0-201910041700.git.1.c8c7aaa.el7               \n  openshift-hyperkube.x86_64 0:4.2.0-201910020731.git.0.463c73f.el7             \n\nComplete!\n"]}



Version-Release number of the following components:
openshift-ansible-4.2.0-201909221318.git.193.0fd88d7.el7

How reproducible:

Steps to Reproduce:
1. Run upgrade.yml playbook to upgrade RHEL worker from 4.1 to 4.2
ansible-playbook -i jenkins_inventory -v /usr/share/ansible/openshift-ansible/playbooks/upgrade.yml

Actual results:
It's still cri-o-1.13.11-0.10.dev.rhaos4.1.gitbdeb2ca.el7.x86_64 on the RHEL node

Expected results:
Should be cri-o-1.14.11-0.17.dev.rhaos4.2.gitc41de67.el7.x86_64.rpm after upgrade to 4.2


Additional info:
Please attach logs from ansible-playbook with the -vvv flag

--- Additional comment from Brenton Leanhardt on 2019-10-11 08:17:02 EDT ---

Definitely seems like a bug and it should be a quick fix.  

Hi Gaoyun, so we understand the urgency, aside from the following bug fixes in cri-o, are there other obviously broken OpenShift features as a result of the cri-o upgrade failure?  If you don't know, that's a fair answer.

https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2019-10214 (med severity, token security)
https://bugzilla.redhat.com/show_bug.cgi?id=1726326 (jenkins)
https://bugzilla.redhat.com/show_bug.cgi?id=1731370 (disable fips)