Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1819016

Summary: [OSP16] undercloud install fails even though registry credentials are provided
Product: Red Hat OpenStack Reporter: ctrinh
Component: python-tripleoclientAssignee: Alex Schultz <aschultz>
Status: CLOSED ERRATA QA Contact: David Rosenfeld <drosenfe>
Severity: low Docs Contact:
Priority: low    
Version: 16.0 (Train)CC: amcleod, aschultz, cjeanner, emacchi, hbrock, jschluet, jslagle, kecarter, mburns
Target Milestone: betaKeywords: Triaged, ZStream
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-tripleoclient-12.3.2-0.20200421100214.d2aac8a.el8ost Doc Type: Enhancement
Doc Text:
With this update, the `container_images_file` parameter is now a required option in the `undercloud.conf` file. You must set this parameter before you install the undercloud. + With the recent move to use registry.redhat.io as the container source, you must authenticate when you fetch containers. For the undercloud, the `container_images_file` is the recommended option to provide the credentials when you perform the installation. Before this update, if this parameter was not set, the deployment failed with authentication errors when trying to fetch containers.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2020-07-29 07:51:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description ctrinh 2020-03-31 00:00:52 UTC 
Description of problem:

openstack undercloud install fails with error message "missing registry credentials or the provided container or namespace does not exist".

I have my credentials set in containers-prepare-parameter.yaml. I tested them using 'podman login registry.redhat.io', they work there.

Error message:

2020-03-31 02:44:05.652 23298 ERROR tripleoclient.v1.tripleo_deploy.Deploy [  ] Exception: Unable to authenticate. This may indicate missing registry credentials or the provided container or namespace does not exist. 401 Client Error: Unauthorized for url: https://registry.redhat.io/auth/realms/rhcc/protocol/redhat-docker-v2/auth?service=docker-registry&scope=repository%3Arhosp-rhel8%2Fopenstack-cron%3Apull

My containers-prepare-parameter.yaml:

# Generated with the following on 2020-03-30T01:14:16.087401
#
#   openstack tripleo container image prepare default --local-push-destination --output-env-file containers-prepare-parameter.yaml
#

parameter_defaults:
  ContainerImageRegistryCredentials:
    registry.redhat.io:
      <my RHN ID, redacted>: <my password, redacted>
  ContainerImagePrepare:
  - push_destination: true
    set:
      ceph_alertmanager_image: ose-prometheus-alertmanager
      ceph_alertmanager_namespace: registry.redhat.io/openshift4
      ceph_alertmanager_tag: 4.1
      ceph_grafana_image: rhceph-3-dashboard-rhel7
      ceph_grafana_namespace: registry.redhat.io/rhceph
      ceph_grafana_tag: 3
      ceph_image: rhceph-4-rhel8
      ceph_namespace: registry.redhat.io/rhceph
      ceph_node_exporter_image: ose-prometheus-node-exporter
      ceph_node_exporter_namespace: registry.redhat.io/openshift4
      ceph_node_exporter_tag: v4.1
      ceph_prometheus_image: ose-prometheus
      ceph_prometheus_namespace: registry.redhat.io/openshift4
      ceph_prometheus_tag: 4.1
      ceph_tag: latest
      name_prefix: openstack-
      name_suffix: ''
      namespace: registry.redhat.io/rhosp-rhel8
      neutron_driver: ovn
      rhel_containers: false
      tag: '16.0'
    tag_from_label: '{version}-{release}'
Comment 1 Cédric Jeanneret 2020-03-31 07:32:26 UTC 
Hello there,

Care to follow this doc in order to ensure the access to the registry is working?
https://access.redhat.com/articles/3560571

Thank you!

Cheers,

C.
Comment 2 ctrinh 2020-03-31 19:30:28 UTC 
Hi C,

Thank you for that link. I went through all of the steps. I am able to access it and it is working.

Perhaps it is my other configuration? How do I check if "container or namespace does not exist"?

-Chris
Comment 3 Alex Schultz 2020-03-31 21:22:56 UTC 
Did you configure the container_images_file in the undercloud.conf to point to the file with the credentials in it?
Comment 5 Alex Schultz 2020-03-31 22:33:31 UTC 
It should be noted that there is another issue out there that if fetching container takes too long you'll get a 401. That is being tracked via https://bugzilla.redhat.com/show_bug.cgi?id=1813520
Comment 6 ctrinh 2020-04-01 20:53:01 UTC 
(In reply to Alex Schultz from comment #3)
> Did you configure the container_images_file in the undercloud.conf to point
> to the file with the credentials in it?

It turns out that this was the issue. It was not picking up my credentials because I did not set this variable. Setting it has allowed me to proceed further with the installation.

Can you make the variable "container_images_file" have a default value if it is not set by the user? I followed the walkthrough instructions but must have missed this.

(In reply to Alex Schultz from comment #5)
> It should be noted that there is another issue out there that if fetching
> container takes too long you'll get a 401. That is being tracked via
> https://bugzilla.redhat.com/show_bug.cgi?id=1813520

Thanks for the info, I may be running into this issue now. I will try the workaround mentioned and to debug further issues. My installation attempts have not been able to complete successfully yet.
Comment 7 Alex Schultz 2020-04-01 21:00:07 UTC 
I'm looking into making it actually required for the downstream with improved comments around it in a future version. It's 100% required since OSP15 so anytime someone skips it, the deployment proceeds but errors like this.  https://review.opendev.org/#/c/716441/
Comment 13 errata-xmlrpc 2020-07-29 07:51:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3148