Description of problem: Deployments that doesn't use ceph should not interact with the ceph registry Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: vi /var/log/tripleo-container-image-prepare.log ... Image prepare failed: 401 Client Error: Unauthorized for url: https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?service=docker-registry&scope=repository%3Arhceph-beta%2Frhceph-4-rhel8%3Apull Traceback (most recent call last): File "/usr/bin/tripleo-container-image-prepare", line 131, in <module> env, roles_data, cleanup=args.cleanup, dry_run=args.dry_run) File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 213, in container_images_prepare_multi uploader.upload() File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 201, in upload uploader.run_tasks() File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1638, in run_tasks result = uploader.upload_image(first_task) File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 942, in upload_image password=source_password File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f return self.call(f, *args, **kw) File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call do = self.iter(retry_state=retry_state) File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 331, in iter raise retry_exc.reraise() File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 167, in reraise raise self.last_attempt.result() File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result return self.__get_result() File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result raise self._exception File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call result = fn(*args, **kwargs) File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 373, in authenticate rauth.raise_for_status() File "/usr/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?service=docker-registry&scope=repository%3Arhceph-beta%2Frhceph-4-rhel8%3Apull ... (undercloud) [stack@undercloud-0 ~]$ openstack overcloud failures |-> Failures for host: aio-0 |--> Task: Pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1 image |---> _ansible_no_log: false |---> changed: true |---> cmd: "podman pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1" |---> delta: "0:00:00.156931" |---> end: "2019-07-01 15:15:50.749794" |---> invocation: { "module_args": { "_raw_params": "podman pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1", "_uses_shell": true, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "stdin_add_newline": true, "strip_empty_ends": true, "warn": true } } |---> msg: "non-zero return code" |---> rc: 125 |---> start: "2019-07-01 15:15:50.592863" |---> stderr: "Trying to pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1...Failed\nerror pulling image \"192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1\": unable to pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1: unable to pull image: Error determining manifest MIME type for docker://192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1: Error reading manifest 20190625.1 in 192.168.24.1:8787/rhosp15/openstack-cinder-volume: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: \"<!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\">\\n<html><head>\\n<title>404 Not Found</title>\\n</head><body>\\n<h1>Not Found</h1>\\n<p>The requested URL /v2/rhosp15/openstack-cinder-volume/manifests/20190625.1 was not found on this server.</p>\\n</body></html>\\n\"" |---> stderr_lines: [ "Trying to pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1...Failed", "error pulling image \"192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1\": unable to pull 192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1: unable to pull image: Error determining manifest MIME type for docker://192.168.24.1:8787/rhosp15/openstack-cinder-volume:20190625.1: Error reading manifest 20190625.1 in 192.168.24.1:8787/rhosp15/openstack-cinder-volume: error parsing HTTP 404 response body: invalid character '<' looking for beginning of value: \"<!DOCTYPE HTML PUBLIC \\\"-//IETF//DTD HTML 2.0//EN\\\">\\n<html><head>\\n<title>404 Not Found</title>\\n</head><body>\\n<h1>Not Found</h1>\\n<p>The requested URL /v2/rhosp15/openstack-cinder-volume/manifests/20190625.1 was not found on this server.</p>\\n</body></html>\\n\"" ] |---> stdout: "" |---> stdout_lines: [] Expected results: Additional info:
Topology in use was All-In-One : 1 Undercloud node and 1 Overcloud node: (undercloud) [stack@undercloud-0 ~]$ cat overcloud_deploy.sh #!/bin/bash openstack overcloud deploy \ --timeout 100 \ --templates /usr/share/openstack-tripleo-heat-templates \ --libvirt-type kvm \ --stack overcloud \ -r /home/stack/composable_roles/roles/roles_data.yaml \ -e /home/stack/composable_roles/roles/nodes.yaml \ --environment-file /usr/share/openstack-tripleo-heat-templates/environments/standalone/standalone-overcloud.yaml \ -e /home/stack/composable_roles/config_lvm.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \ -e /home/stack/composable_roles/network/network-environment.yaml \ -e /home/stack/composable_roles/enable-tls.yaml \ -e /home/stack/composable_roles/inject-trust-anchor.yaml \ -e /home/stack/composable_roles/public_vip.yaml \ -e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-endpoints-public-ip.yaml \ -e /home/stack/composable_roles/hostnames.yaml \ -e /home/stack/composable_roles/config_heat.yaml \ -e ~/containers-prepare-parameter.yaml \ --log-file overcloud_deployment_15.log But the bug is relevant for any topology that doesn't use ceph storage
The original reported issue was due to a lack of exiting when the pull fails. That has been addressed. The reported issue is a downstream problem due to the usage of two seperate repositories during testing. The openstack one doesn't require authentication, while the testing ceph one does. It's unlikely that we'll address this anytime soon since this is how it has operated since OSP13. For now moving this backed to assigned and we'll investigate this later. Unfortunately due to the nature of the process, we've always assumed availability of ceph for initial installation even if unused. For example the rhos-director-images used to include ceph binaries even if unused.
*** Bug 1812591 has been marked as a duplicate of this bug. ***
I think this upstream change which landed in Ussuri will fix this issue, because it should remove any role mapped to OS::Heat::None from the list of roles during template processing. It would be useful to validate this change by deploying without ceph and confirming the ceph repo credentials are not required.
According to our records, this should be resolved by openstack-tripleo-common-11.4.1-1.20200914165651.el8ost. This build is available now.
The All in One job used when BZ was reported passes. Also, the 401 Client error is not seen in tripleo-container-image-prepare.log.