Bug 1741247

Summary:	[Upgrades/updates] External update/upgrade run --tags container_image_prepare does not upload the container images in the Undercloud
Product:	Red Hat OpenStack	Reporter:	Jose Luis Franco <jfrancoa>
Component:	python-tripleoclient	Assignee:	Jose Luis Franco <jfrancoa>
Status:	CLOSED ERRATA	QA Contact:	Raviv Bar-Tal <rbartal>
Severity:	high	Docs Contact:
Priority:	high
Version:	15.0 (Stein)	CC:	hbrock, jschluet, jslagle, mburns, morazi, sathlang, scorcora, sgolovat
Target Milestone:	rc	Keywords:	Triaged
Target Release:	15.0 (Stein)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-validations-10.5.1-0.20190830060432.585c119.el8ost python-tripleoclient-11.5.1-0.20190829110437.9b9b5aa.el8ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-09-21 11:24:25 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1746815
Bug Blocks:	1727807, 1727808

Description Jose Luis Franco 2019-08-14 15:10:28 UTC

Description of problem:

Since OSP14 the way to provide the container images to be used during deployment/update/upgrade passed to be specified via the ContainerImagePrepare environment parameter.

When upgrading, it was this command the one that allowed us to point to the new version's container images:

openstack external-upgrade run --tags container_image_prepare

In a similar way,the container images were updating via:

openstack external-update run --tags container_image_prepare

In OSP15 these two commands seemed to be running, however when executing the upgrade/update workflow the container images were not present in the local registry (now a httpd vhost running in the Undercloud named image-serve). As a consequence, as the overcloud images could not be retrieved the update/upgrade process failed.

The tripleo-container-image-prepare command is being executed in the mistral_executor container locally, instead of on the Undercloud's host. Therefore, if we display the content of the /var/lib/image-serve/v2/rhosp15 folder in the mistral_exector, we will see the container images we want to update/upgrade to.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Deploy OSP15
2. Run the undercloud upgrade
3. Run the overcloud update prepare passing the new containers_prepare_parameters.yaml (pointing to the tag we want to update to)
4. Run the "openstack external-update run --tags container_image_prepare" command to upload the container images.
5. Print out the content of the Undercloud's directory /var/lib/image-serve/v2/rhosp15 and see that all the overcloud service container images are missing the new tag which should have been pulled.
6. Display mistral_exector image-serve folder via "sudo podman exec -it mistral_executor ls /var/lib/image-serve/v2/rhosp15"
7. All the images missing in the Undercloud can be found there (or the newest tag in the case of an update).

Actual results:

The container images are being up

Expected results:

Additional info:

Comment 1 Jose Luis Franco 2019-08-14 15:16:18 UTC

A way to workaround this issue is to execute the old openstack command to upload the container images:

sudo openstack tripleo container image prepare \
  -e ~/containers-prepare-parameter.yaml

Comment 2 Sofer Athlan-Guyot 2019-08-28 13:32:42 UTC

*** Bug 1746440 has been marked as a duplicate of this bug. ***

Comment 6 Sofer Athlan-Guyot 2019-08-30 07:34:22 UTC

Everything is merged in stein.

Comment 15 Jose Luis Franco 2019-09-09 07:43:30 UTC

Hello Greg,

In theory this change was meant to fix something which was already part of the upgrade/update process, so no extra doc would be needed besides the one we already had from OSP14: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/14/html/keeping_red_hat_openstack_platform_updated/assembly-updating_the_overcloud#running-the-container-image-preparation

Comment 17 errata-xmlrpc 2019-09-21 11:24:25 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811