DescriptionMichele Baldessari
2020-10-09 09:01:38 UTC
Description of problem:
Our deployment (composable control plane) is failing with:
image: http://rhos-qe-mirror-rdu2.usersys.redhat.com:5002/v2/rh-osbs/rhosp16-openstack-sahara-api/manifests/16.2_20201008.1", "\"\"\"", "", "The above exception was the direct cause of the following exception:", "", "Traceback (most recent call last):", " File \"/usr/lib/python3.6/site-packages/tripleoclient/command.py\", line 32, in run", " super(Command, self).run(parsed_args)", " File \"/usr/lib/python3.6/site-packages/osc_lib/command/command.py\", line 41, in run", " return super(Command, self).run(parsed_args)", " File \"/usr/lib/python3.6/site-packages/cliff/command.py\", line 185, in run", " return_code = self.take_action(parsed_args) or 0", " File \"/usr/lib/python3.6/site-packages/tripleoclient/v1/container_image.py\", line 1055, in take_action", " cleanup=parsed_args.cleanup, lock=lock)", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py\", line 241, in container_images_prepare_multi", " uploader.upload()", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 506, in upload", " uploader.run_tasks()", " File \"/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py\", line 2535, in run_tasks", " for result in p.map(upload_task, self.upload_tasks):", " File \"/usr/lib64/python3.6/concurrent/futures/process.py\", line 366, in _chain_from_iterable_of_lists", " for element in iterable:", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 586, in result_iterator", " yield fs.pop().result()", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 425, in result", " return self.__get_result()", " File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result", " raise self._exception", "tripleo_common.image.exception.ImageNotFoundException: Not found image: http://rhos-qe-mirror-rdu2.usersys.redhat.com:5002/v2/rh-osbs/rhosp16-openstack-sahara-api/manifests/16.2_20201008.1", "Not found image: http://rhos-qe-mirror-rdu2.usersys.redhat.com:5002/v2/rh-osbs/rhosp16-openstack-sahara-api/manifests/16.2_20201008.1"], "stdout": "", "stdout_lines": []}[0m
I don't think it is a mirror problem since the image is not there both on the rdu mirror nor on the brq mirror:
[stack@undercloud-0 ~]$ sudo podman pull rhos-qe-mirror-brq.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1
Trying to pull rhos-qe-mirror-brq.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1...
manifest unknown: manifest unknown
Error: unable to pull rhos-qe-mirror-brq.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1: Error initializing source docker://rhos-qe-mirror-brq.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1: Error reading manifest 16.2_20201008.1 in rhos-qe-mirror-brq.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api: manifest unknown: manifest unknown
[stack@undercloud-0 ~]$ sudo podman pull rhos-qe-mirror-rdu2.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1
Trying to pull rhos-qe-mirror-rdu2.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1...
manifest unknown: manifest unknown
Error: unable to pull rhos-qe-mirror-rdu2.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1: Error initializing source docker://rhos-qe-mirror-rdu2.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api:16.2_20201008.1: Error reading manifest 16.2_2
0201008.1 in rhos-qe-mirror-rdu2.usersys.redhat.com:5002/rh-osbs/rhosp16-openstack-sahara-api: manifest unknown: manifest unknown
Via https://code.engineering.redhat.com/gerrit/#/c/176688/ we dropped them downstream, yet the overcloud deployment (UC succeeds) fails because it is trying to pull them in.
My first hunch is that https://code.engineering.redhat.com/gerrit/#/c/176688/ is not patching container-images/tripleo_containers.yaml.j2
What I don't know yet is why none else is hitting this. I'll keep poking around and assign it to me for the time being
Comment 1Michele Baldessari
2020-10-09 09:05:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Red Hat OpenStack Platform 16.2 containers bug fix advisory), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHEA-2021:3489