Bug 1643055

Summary:	Wrongly rendered namespace in docker images: <class 'jinja2.utils.Namespace'>/centos-binary-ceilometer-notification:current-tripleo"
Product:	Red Hat OpenStack	Reporter:	Jose Luis Franco <jfrancoa>
Component:	openstack-tripleo-common	Assignee:	Jose Luis Franco <jfrancoa>
Status:	CLOSED ERRATA	QA Contact:	Artem Hrechanychenko <ahrechan>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	12.0 (Pike)	CC:	jfrancoa, mburns, slinaber
Target Milestone:	---	Keywords:	Triaged, ZStream
Target Release:	12.0 (Pike)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-tripleoclient-7.3.10-4.el7ost.src.rpm, openstack-tripleo-common-7.6.13-6.el7ost.src.rpm,	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-12-05 18:51:01 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jose Luis Franco 2018-10-25 12:31:15 UTC

Description of problem:

Based on issue found upstream https://bugs.launchpad.net/tripleo/+bug/1796664

In the upgrades job from Ocata to Pike it has been observed an error when rendering the container images during the upgrade step. The overcloud upgrade command fails with the following tasks failing:

...
2018-10-05 09:40:24 | "2018-10-05 09:39:38,539 WARNING: 144091 -- retrying pulling image: <class 'jinja2.utils.Namespace'>/centos-binary-cron:current-tripleo",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,565 WARNING: 144091 -- docker pull failed: Error parsing reference: \"<class 'jinja2.utils.Namespace'>/centos-binary-cron:current-tripleo\" is not a valid repository/tag: invalid reference format",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,565 WARNING: 144091 -- retrying pulling image: <class 'jinja2.utils.Namespace'>/centos-binary-cron:current-tripleo",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,565 ERROR: 144091 -- Failed to pull image: <class 'jinja2.utils.Namespace'>/centos-binary-cron:current-tripleo",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,592 ERROR: 144091 -- Failed running docker-puppet.py for crond",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,592 ERROR: 144091 -- /usr/bin/docker-current: Error parsing reference: \"<class 'jinja2.utils.Namespace'>/centos-binary-cron:current-tripleo\" is not a valid repository/tag: invalid reference format.",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,592 INFO: 144091 -- Finished processing puppet configs",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,697 INFO: 144089 -- Removing container: docker-puppet-neutron",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,756 INFO: 144089 -- Finished processing puppet configs",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring haproxy",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring nova_placement",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring clustercheck",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring mysql",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring rabbitmq",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring iscsid",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring heat",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,757 ERROR: 144088 -- ERROR configuring glance_api",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring keystone",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring nova",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring horizon",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring heat_api",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring heat_api_cfn",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring memcached",
2018-10-05 09:40:24 | "2018-10-05 09:39:51,758 ERROR: 144088 -- ERROR configuring crond"
2018-10-05 09:40:24 | ]
2018-10-05 09:40:24 | }
2018-10-05 09:40:24 | to retry, use: --limit @/var/lib/heat-config/heat-config-ansible/df8a31ae-f608-49a5-b8e7-4fed8b8ff612_playbook.retry
2018-10-05 09:40:24 |
2018-10-05 09:40:24 | PLAY RECAP *********************************************************************
2018-10-05 09:40:24 | localhost : ok=6 changed=2 unreachable=0 failed=1
2018-10-05 09:40:24 |
2018-10-05 09:40:24 | deploy_stderr: |

https://logs.rdoproject.org/21/604121/9/openstack-experimental/legacy-tripleo-ci-centos-7-containers-multinode-upgrades-pike-branch/7923780/logs/undercloud/home/zuul/overcloud_composable_failed_upgrade.log.txt.gz#_2018-10-05_09_40_24

It can be seen that the namespace used to pull the images is unrecognized:
2018-10-05 09:40:24 | "2018-10-05 09:39:41,592 ERROR: 144091 -- Failed running docker-puppet.py for crond",
2018-10-05 09:40:24 | "2018-10-05 09:39:41,592 ERROR: 144091 -- /usr/bin/docker-current: Error parsing reference: \"<class 'jinja2.utils.Namespace'>/centos-binary-cron:current-tripleo\" is not a valid repository/tag: invalid reference format.",

When going to the mistral logs, we see in the result from the container_image_prepare action, that the templates were rendered this way:
https://logs.rdoproject.org/21/604121/9/openstack-experimental/legacy-tripleo-ci-centos-7-containers-multinode-upgrades-pike-branch/7923780/logs/undercloud/var/log/mistral/engine.log.txt.gz#_2018-10-05_09_16_21_609

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 8 errata-xmlrpc 2018-12-05 18:51:01 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3789