Bug 1659600

Summary: Updating from 3.10.x to 3.10.72 does not pull in 3.10.72 images unless you specify openshift_image_tag=v3.10.72 in inventory
Product: OpenShift Container Platform Reporter: Candace Sheremeta <cshereme>
Component: InstallerAssignee: Patrick Dillon <padillon>
Installer sub component: openshift-ansible QA Contact: Weihua Meng <wmeng>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: dmoessne, gpei, hpolava, maupadhy, palonsor, sgarciam, wmeng
Version: 3.10.0   
Target Milestone: ---   
Target Release: 3.10.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
This bug fix is a backport of BZ 1658387 Cause: playbooks ran a check to see if image existed on disk with X-version tag, but did not make sure the version on disk was up-to-date to the tagged image in the repo. If the image existed on disk, the image pull was skipped. Consequence: Z-version updates would not work. If trying to upgrade from, e.g. 3.10.x to 3.10.72 the 3.10.x image would satisfy the check and the 3.10.x image would not be pulled. Therefore the cluster would remain at 3.10.x before and after the upgrade. Fix: Remove the on disk check. Image pulls are efficient so there is no need to check whether the disk exists on disk before hand. Result: Upgrades proceed as expected.
Story Points: ---
Clone Of: 1658387 Environment:
Last Closed: 2019-03-14 02:15:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1658387    
Bug Blocks:    

Comment 1 Patrick Dillon 2019-02-07 20:49:48 UTC
3.10 PR: https://github.com/openshift/openshift-ansible/pull/11146

Comment 2 Patrick Dillon 2019-02-08 15:06:56 UTC
This PR fixes a problem in previous PR, where if etcd pre-pull was skipped (e.g. standalone etcd cluster) then the async status check would still run and fail on undefined variable etcd_prepull:
https://github.com/openshift/openshift-ansible/pull/11152

Comment 4 Weihua Meng 2019-02-28 16:13:16 UTC
Not fully fixed.

openshift-ansible-3.10.119-1.git.0.79ca5c5.el7

from v3.10.118 to v3.10.119
after upgrade
some images got updated
registry.reg-aws.openshift.com:443/openshift3/ose-node                      v3.10               03786987873f        5 hours ago         1.29 GB
registry.reg-aws.openshift.com:443/openshift3/ose-control-plane             v3.10               29456c90a356        6 hours ago         794 MB
registry.reg-aws.openshift.com:443/openshift3/ose-pod                       v3.10               b32dafd48ebe        8 hours ago         217 MB
registry.reg-aws.openshift.com:443/openshift3/ose-pod                       v3.10.119           b32dafd48ebe        8 hours ago         217 MB

some images not updated
registry.reg-aws.openshift.com:443/openshift3/ose-service-catalog           v3.10               a5dbb1d5783b        5 days ago          316 MB
registry.reg-aws.openshift.com:443/openshift3/ose-web-console               v3.10               6f24d4f973f8        5 days ago          322 MB
registry.reg-aws.openshift.com:443/openshift3/ose-template-service-broker   v3.10               422278542359        5 days ago          287 MB
registry.reg-aws.openshift.com:443/openshift3/registry-console              v3.10               e7818f259476        5 days ago          235 MB

all related images got updated are expected.

Comment 5 Weihua Meng 2019-02-28 16:14:57 UTC
for fresh install, images are 
# docker images
REPOSITORY                                                                  TAG                 IMAGE ID            CREATED             SIZE
registry.reg-aws.openshift.com:443/openshift3/ose-node                      v3.10               03786987873f        3 hours ago         1.29 GB
registry.reg-aws.openshift.com:443/openshift3/ose-deployer                  v3.10.119           af07da485d5b        3 hours ago         794 MB
registry.reg-aws.openshift.com:443/openshift3/ose-control-plane             v3.10               29456c90a356        3 hours ago         794 MB
registry.reg-aws.openshift.com:443/openshift3/ose-web-console               v3.10               0bc4c5ea2077        3 hours ago         322 MB
registry.reg-aws.openshift.com:443/openshift3/ose-service-catalog           v3.10               308c5b365371        3 hours ago         316 MB
registry.reg-aws.openshift.com:443/openshift3/ose-template-service-broker   v3.10               4182a92f4474        4 hours ago         287 MB
registry.reg-aws.openshift.com:443/openshift3/ose-pod                       v3.10               b32dafd48ebe        5 hours ago         217 MB
registry.reg-aws.openshift.com:443/openshift3/ose-pod                       v3.10.119           b32dafd48ebe        5 hours ago         217 MB
registry.reg-aws.openshift.com:443/openshift3/registry-console              v3.10               fb796dcfa43f        5 hours ago         235 MB

Comment 7 Patrick Dillon 2019-02-28 20:59:54 UTC
Similar to https://bugzilla.redhat.com/show_bug.cgi?id=1658387 

We have changed the scope of this bug so that only the control plane should be updated. We have created another bug for updating the deployments that run on the cluster.

Comment 8 Weihua Meng 2019-03-01 00:41:39 UTC
move to verified according to comment 7

other issue is tracked by https://bugzilla.redhat.com/show_bug.cgi?id=1684315

Comment 10 errata-xmlrpc 2019-03-14 02:15:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0405