Bug 1793198

Description of problem:
Openstack Overcloud deployment failed with error 
  File \"/usr/lib64/python3.6/concurrent/futures/_base.py\", line 384, in __get_result", "    raise self._exception", "requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://registry-1.docker.io/v2/prom/grafana/manifests/5.2.4"], "stdout": "", "stdout_lines": []}
Overcloud configuration failed.

Also in /var/log/tripleo-container-image-prepare.log
2020-01-20 21:41:52,033 188450 ERROR root [  ] Image prepare failed: 401 Client Error: Unauthorized for url: https://registry-1.docker.io/v2/prom/grafana/manifests/5.2.4
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib64/python3.6/concurrent/futures/process.py", line 175, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/lib64/python3.6/concurrent/futures/process.py", line 153, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/usr/lib64/python3.6/concurrent/futures/process.py", line 153, in <listcomp>
    return [fn(*args) for args in chunk]
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 2259, in upload_task
    return uploader.upload_image(task)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1321, in upload_image
    t.multi_arch
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1464, in _collect_manifests_layers
    multi_arch=multi_arch
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 292, in wrapped_f
    return self.call(f, *args, **kw)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 358, in call
    do = self.iter(retry_state=retry_state)
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 331, in iter
    raise retry_exc.reraise()
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 167, in reraise
    raise self.last_attempt.result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
    return self.__get_result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/lib/python3.6/site-packages/tenacity/__init__.py", line 361, in call
    result = fn(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 1455, in _fetch_manifest
    cls.check_status(session=session, request=r)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 562, in check_status
    request.raise_for_status()
  File "/usr/lib/python3.6/site-packages/requests/models.py", line 940, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://registry-1.docker.io/v2/prom/grafana/manifests/5.2.4
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/bin/tripleo-container-image-prepare", line 138, in <module>
    lock=lock)
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/kolla_builder.py", line 235, in container_images_prepare_multi
    uploader.upload()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 272, in upload
    uploader.run_tasks()
  File "/usr/lib/python3.6/site-packages/tripleo_common/image/image_uploader.py", line 2200, in run_tasks
    for result in p.map(upload_task, self.upload_tasks):
  File "/usr/lib64/python3.6/concurrent/futures/process.py", line 366, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 586, in result_iterator
    yield fs.pop().result()
  File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 432, in result

[stack@site-undercloud-0 ~]$ cat containers-prepare-parameter.yaml 
# Generated with the following on 2020-01-20T20:16:23.163730
#
#   openstack tripleo container image prepare default --output-env-file /home/stack/containers-prepare-parameter.yaml --local-push-destination
#

parameter_defaults:
  ContainerImagePrepare:
  - push_destination: true
    set:
      ceph_alertmanager_image: alertmanager
      ceph_alertmanager_namespace: docker.io/prom
      ceph_alertmanager_tag: v0.16.2
      ceph_grafana_image: grafana
      ceph_grafana_namespace: docker.io/prom
      ceph_grafana_tag: 5.2.4
      ceph_image: rhceph-4.0-rhel8
      ceph_namespace: docker-registry.upshift.redhat.com/ceph
      ceph_node_exporter_image: node-exporter
      ceph_node_exporter_namespace: docker.io/prom
      ceph_node_exporter_tag: v0.17.0
      ceph_prometheus_image: prometheus
      ceph_prometheus_namespace: docker.io/prom
      ceph_prometheus_tag: v2.7.2
      ceph_tag: latest
      name_prefix: rhosp16-openstack-
      name_suffix: ''
      namespace: registry-proxy.engineering.redhat.com/rh-osbs
      neutron_driver: ovn
      rhel_containers: false
      tag: 20200110.1


Version-Release number of selected component (if applicable):
RHOS_TRUNK-16.0-RHEL-8-20200117.n.1

How reproducible:
Jenkins job
https://rhos-ci-staging-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/edge/job/DFG-edge-deployment-16-rhel-virthost-ipv4-3cont-2comp-2leafs-x-3comp-hci/3/consoleFull

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: