Description of problem: When I use EUS repositories below: - rhel-9-for-x86_64-baseos-eus-rpms - rhel-9-for-x86_64-appstream-eus-rpms - rhel-9-for-x86_64-highavailability-eus-rpms Undercloud installation fails with `Failed to download metadata for repo 'rhel-9-for-x86_64-baseos-eus-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml` message like below. ~~~ $ openstack undercloud install : 2022-08-18 10:56:53.008558 | 525400c5-b16e-5449-ec7b-000000000860 | TASK | Ensure subscription-manager is not detecting container 2022-08-18 10:56:53.201440 | 525400c5-b16e-5449-ec7b-000000000860 | CHANGED | Ensure subscription-manager is not detecting container | undercloud 2022-08-18 10:56:53.202527 | 525400c5-b16e-5449-ec7b-000000000860 | TIMING | Ensure subscription-manager is not detecting container | undercloud | 0:02:15.849040 | 0.19s 2022-08-18 10:56:53.220526 | 525400c5-b16e-5449-ec7b-000000000861 | TASK | Set login facts 2022-08-18 10:56:53.246505 | 525400c5-b16e-5449-ec7b-000000000861 | OK | Set login facts | undercloud 2022-08-18 10:56:53.247310 | 525400c5-b16e-5449-ec7b-000000000861 | TIMING | Set login facts | undercloud | 0:02:15.893826 | 0.03s 2022-08-18 10:56:53.264901 | 525400c5-b16e-5449-ec7b-000000000862 | TASK | Convert logins json to dict 2022-08-18 10:56:53.290466 | 525400c5-b16e-5449-ec7b-000000000862 | SKIPPED | Convert logins json to dict | undercloud 2022-08-18 10:56:53.291388 | 525400c5-b16e-5449-ec7b-000000000862 | TIMING | Convert logins json to dict | undercloud | 0:02:15.937890 | 0.03s 2022-08-18 10:56:53.309507 | 525400c5-b16e-5449-ec7b-000000000863 | TASK | Set registry logins 2022-08-18 10:56:53.334671 | 525400c5-b16e-5449-ec7b-000000000863 | SKIPPED | Set registry logins | undercloud 2022-08-18 10:56:53.335587 | 525400c5-b16e-5449-ec7b-000000000863 | TIMING | Set registry logins | undercloud | 0:02:15.982102 | 0.03s 2022-08-18 10:56:53.353287 | 525400c5-b16e-5449-ec7b-000000000864 | TASK | Run podman install 2022-08-18 10:56:53.378529 | 525400c5-b16e-5449-ec7b-000000000864 | TIMING | Run podman install | undercloud | 0:02:16.025043 | 0.02s 2022-08-18 10:56:53.424012 | 525400c5-b16e-5449-ec7b-000000000b8f | TASK | Gather variables for each operating system 2022-08-18 10:56:53.471172 | 525400c5-b16e-5449-ec7b-000000000b8f | OK | Gather variables for each operating system | undercloud | item=/usr/share/ansible/roles/tripleo_podman/vars/redhat-9.yml 2022-08-18 10:56:53.472884 | 525400c5-b16e-5449-ec7b-000000000b8f | TIMING | tripleo_podman : Gather variables for each operating system | undercloud | 0:02:16.119395 | 0.05s 2022-08-18 10:56:53.482776 | 525400c5-b16e-5449-ec7b-000000000b8f | TIMING | tripleo_podman : Gather variables for each operating system | undercloud | 0:02:16.129299 | 0.06s 2022-08-18 10:56:53.500534 | 525400c5-b16e-5449-ec7b-000000000b91 | TASK | ensure podman and deps are installed 2022-08-18 10:56:56.179281 | 525400c5-b16e-5449-ec7b-000000000b91 | FATAL | ensure podman and deps are installed | undercloud | error={"changed": false, "msg": "Failed to download metadata for repo 'rhel-9-for-x86_64-baseos-eus-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried", "rc": 1, "results": []} 2022-08-18 10:56:56.180285 | 525400c5-b16e-5449-ec7b-000000000b91 | TIMING | tripleo_podman : ensure podman and deps are installed | undercloud | 0:02:18.826804 | 2.68s PLAY RECAP ********************************************************************* localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0 undercloud : ok=186 changed=102 unreachable=0 failed=1 skipped=42 rescued=0 ignored=2 : ~~~ To use EUS repositories, RHEL minor release must be fixed by `subscription-manager release --set=9.0`. However, after the failure of the deployment, I noticed that subscription-manager command doesn't work and cannot obtain release information as below. That's why dnf command couldn't download repomd.xml from rhel-9-for-x86_64-baseos-eus-rpms and the undercloud installation failed. ~~~ [root@undercloud ~]# subscription-manager release subscription-manager is disabled when running inside a container. Please refer to your host system for subscription management. ~~~ The reason subscription-manager doesn't work is that /etc/rhsm-host is created during the installation. ~~~ [root@undercloud ~]# ls -l /etc/|grep rhsm drwxr-xr-x. 6 root root 84 Aug 18 10:39 rhsm lrwxrwxrwx. 1 root root 9 Aug 18 10:56 rhsm-host -> /etc/rhsm ~~~ If /etc/rhsm-host exists, subscription-manager detects container environment. So `subscription-manager` command doesn't work. - https://github.com/candlepin/subscription-manager/blob/385d64843affed7b58c2fb461612cf05f1dac4e5/src/rhsm/config.py#L122 /etc/rhsm-host is created after the following Bugizilla and change. - https://bugzilla.redhat.com/show_bug.cgi?id=2075080 - https://review.opendev.org/c/openstack/tripleo-heat-templates/+/845353 As a workaround, I commented out the ansible task introduced by the above Bugzilla and change, ~~~ [stack@undercloud ~]$ sudo vi /usr/share/openstack-tripleo-heat-templates/deployment/podman/podman-baremetal-ansible.yaml : outputs: role_data: description: Role data for the podman service value: service_name: podman config_settings: {} step_config: '' host_prep_tasks: - name: Install and configure Podman block: &install_and_configure_podman #- name: Ensure subscription-manager is not detecting container # when: # - ansible_facts['distribution'] == 'RedHat' # # Needed in order to solve rhbz#2075080. # # Please refer to rhbz#2095316 for more internal details. # file: # dest: /etc/rhsm-host # src: /etc/rhsm # state: link : ~~~ and removed /etc/rhsm-host, ~~~ [root@undercloud ~]# rm /etc/rhsm-host ~~~ then installation succeeded. ~~~ [stack@undercloud ~]$ openstack undercloud install : ######################################################## Deployment successful! ######################################################## Writing the stack virtual update mark file /var/lib/tripleo-heat-installer/update_mark_undercloud ########################################################## The Undercloud has been successfully installed. Useful files: Password file is at /home/stack/tripleo-undercloud-passwords.yaml The stackrc file is at ~/stackrc Use these files to interact with OpenStack services, and ensure they are secured. ########################################################## ~~~ Version-Release number of selected component (if applicable): RHOSP 17 beta How reproducible: Steps to Reproduce: 1. Follow our Document https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/17.0-beta/html-single/director_installation_and_usage 2. Use EUS repositories : rhel-9-for-x86_64-baseos-eus-rpms and etc. 3. Set minor version by `subscription-manager release --set=9.0` 4. Execute "openstack undercloud install" Actual results: Installation fails. Expected results: Installation succeeds.
I vaguely guess https://review.opendev.org/c/openstack/ansible-role-redhat-subscription/+/852846 is related ? @Cedric I know you've fixed a few subscription-manager issues. Do you have any idea ?
guess a plain revert of the creation of the rhsm-host directory is mandatory. latest subman changes have been a real pain to solve.
Actually, reverting may re-break the satellite/custom repo support..... Wondering if it's not more related to the lack of the env var export[1] than the availability of the directory. Quick question: were containers already running at this point? I mean, podman is supposed to be installed as a dependency of tripleoclient (direct or indirect, I don't remember - still, it's here before the deploy). If so, it may really be the "try to re-deploy" part instead of the /etc/rhsm-host directory availability. Cheers, C. [1] https://review.opendev.org/c/openstack/ansible-role-redhat-subscription/+/848391
＠yatanaka Just to be clear, when did you actually execute `subscription-manager release --set=9.0` ? (1) before you initially attempted undercloud install or (2) after install failed because of missing release What is written in "Steps to Reproduce" sounds like (2) but the whole bug description indicates (1) .
> Quick question: were containers already running at this point? Just after the failure of "openstack undercloud install", podman is installed and I could execute podman command. But I couldn't find any containers by "podman ps" and "sudo podman ps". So there were no running containers. > when did you actually execute `subscription-manager release --set=9.0` I executed `subscription-manager release --set=9.0` before running "openstack undercloud install". Sorry for my bad explanation. I'm detailing steps I executed below. ~~~ # login as root user subscription-manager clean; subscription-manager register subscription-manager attach --pool=XXXXXXXXXXXXXXXXXXXXXXX subscription-manager release --set=9.0 subscription-manager release subscription-manager repos --disable=* subscription-manager repos \ --enable=rhel-9-for-x86_64-baseos-eus-rpms \ --enable=rhel-9-for-x86_64-appstream-eus-rpms \ --enable=rhel-9-for-x86_64-highavailability-eus-rpms \ --enable=openstack-beta-for-rhel-9-x86_64-rpms \ --enable=fast-datapath-for-rhel-9-x86_64-rpms dnf module reset container-tools ===========> I think this command didn't work. dnf module enable -y container-tools:3.0 ===========> Because no modules is provided on RHEL 9 currently. dnf install -y python3-tripleoclient dnf install -y ceph-ansible dnf install -y vim crudini ipmitool dnf update -y timedatectl set-timezone Asia/Tokyo useradd stack echo redhat | passwd --stdin stack echo "stack ALL=(root) NOPASSWD:ALL" | tee -a /etc/sudoers.d/stack chmod 0440 /etc/sudoers.d/stack reboot # login as stack user mkdir ~/images mkdir ~/templates openstack tripleo container image prepare default \ --local-push-destination \ --output-env-file containers-prepare-parameter.yaml cp containers-prepare-parameter.yaml containers-prepare-parameter.yaml.back sed -i 's/rhosp-rhel9/rhosp-beta/g' containers-prepare-parameter.yaml cat << EOF >> containers-prepare-parameter.yaml ContainerImageRegistryCredentials: registry.redhat.io: XXXXX: XXXXX EOF cat << EOF >> /home/stack/templates/custom-undercloud-params.yaml parameter_defaults: Debug: True EOF cp \ /usr/share/python-tripleoclient/undercloud.conf.sample \ ~/undercloud.conf crudini --set undercloud.conf DEFAULT container_images_file /home/stack/containers-prepare-parameter.yaml crudini --set undercloud.conf DEFAULT local_interface eth0 crudini --set undercloud.conf DEFAULT overcloud_domain_name yatanaka.example.com crudini --set undercloud.conf DEFAULT undercloud_nameservers 10.0.0.1 crudini --set undercloud.conf DEFAULT clean_nodes true crudini --set undercloud.conf DEFAULT generate_service_certificate true crudini --set undercloud.conf DEFAULT undercloud_ntp_servers 10.0.0.1 crudini --set undercloud.conf DEFAULT custom_env_files /home/stack/templates/custom-undercloud-params.yaml crudini --set undercloud.conf ctlplane-subnet masquerade true openstack undercloud install ~~~
Notes: - this may become a blocker for 17.0. - 17.0 doesn't have support for satellite yet - 17.1 WILL have satellite support, so we really must take the opportunity to fix things here.
So ... dnf is broken if /etc/rhsm-host or /run/.containerenv exists. We should NOT create /etc/rhsm-host because it breaks subscription-manager at host, but we also need to consider how to workaround this problem after containers are started and .containerenv is created because of /run mounted from host. [root@node-0 ~]# subscription-manager repos \ --enable=rhel-9-for-x86_64-baseos-eus-rpms \ --enable=rhel-9-for-x86_64-appstream-eus-rpms \ --enable=rhel-9-for-x86_64-highavailability-eus-rpms \ --enable=openstack-beta-for-rhel-9-x86_64-rpms \ --enable=fast-datapath-for-rhel-9-x86_64-rpms Repository 'rhel-9-for-x86_64-baseos-eus-rpms' is enabled for this system. Repository 'rhel-9-for-x86_64-appstream-eus-rpms' is enabled for this system. Repository 'rhel-9-for-x86_64-highavailability-eus-rpms' is enabled for this system. Repository 'openstack-beta-for-rhel-9-x86_64-rpms' is enabled for this system. Repository 'fast-datapath-for-rhel-9-x86_64-rpms' is enabled for this system. [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Red Hat Enterprise Linux 9 for x86_64 - BaseOS - Extended Update Support (RPMs) 499 kB/s | 3.6 MB 00:07 Red Hat Enterprise Linux 9 for x86_64 - AppStream - Extended Update Support (RPMs) 1.4 MB/s | 9.2 MB 00:06 Red Hat Enterprise Linux 9 for x86_64 - High Availability - Extended Update Support ( 189 kB/s | 427 kB 00:02 Fast Datapath for RHEL 9 x86_64 (RPMs) 13 kB/s | 22 kB 00:01 Last metadata expiration check: 0:00:01 ago on Thu Aug 18 11:12:40 2022. Dependencies resolved. ====================================================================================================================== Package Architecture Version Repository Size ====================================================================================================================== Installing: podman x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 12 M Installing dependencies: conmon x86_64 2:2.1.2-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 56 k container-selinux noarch 3:2.188.0-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 52 k containers-common x86_64 2:1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 116 k criu x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-eus-rpms 512 k criu-libs x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-eus-rpms 33 k crun x86_64 1.4.5-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 189 k fuse-common x86_64 3.10.2-5.el9 rhel-9-for-x86_64-baseos-eus-rpms 9.3 k fuse-overlayfs x86_64 1.9-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 74 k fuse3 x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-eus-rpms 58 k fuse3-libs x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-eus-rpms 94 k iptables-nft x86_64 1.8.7-28.el9 rhel-9-for-x86_64-baseos-eus-rpms 208 k libnet x86_64 1.2-6.el9 rhel-9-for-x86_64-appstream-eus-rpms 61 k libnftnl x86_64 1.1.9-4.el9 rhel-9-for-x86_64-baseos-eus-rpms 87 k libslirp x86_64 4.4.0-7.el9 rhel-9-for-x86_64-appstream-eus-rpms 72 k netavark x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 2.0 M nftables x86_64 1:0.9.8-12.el9 rhel-9-for-x86_64-baseos-eus-rpms 367 k podman-catatonit x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 354 k shadow-utils-subid x86_64 2:4.9-3.el9 rhel-9-for-x86_64-baseos-eus-rpms 91 k slirp4netns x86_64 1.2.0-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 49 k yajl x86_64 2.1.0-21.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 42 k Installing weak dependencies: aardvark-dns x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 1.0 M Transaction Summary ====================================================================================================================== Install 22 Packages Total download size: 18 M Installed size: 63 M Is this ok [y/N]: ^COperation aborted. [root@node-0 ~]# [root@node-0 ~]# ln -s /etc/rhsm /etc/rhsm-host [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Subscription Manager is operating in container mode. Red Hat Enterprise Linux 9 for x86_64 - BaseOS - Extended Update Support (RPMs) 2.4 B/s | 10 B 00:04 Errors during downloading metadata for repository 'rhel-9-for-x86_64-baseos-eus-rpms': - Status code: 404 for https://cdn.redhat.com/content/eus/rhel9/9/x86_64/baseos/os/repodata/repomd.xml (IP: 23.58.8.251) Error: Failed to download metadata for repo 'rhel-9-for-x86_64-baseos-eus-rpms': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried [root@node-0 ~]# rm /etc/rhsm-host/ rm: cannot remove '/etc/rhsm-host/': Is a directory [root@node-0 ~]# rm /etc/rhsm-host rm: remove symbolic link '/etc/rhsm-host'? y [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Red Hat Enterprise Linux 9 for x86_64 - BaseOS - Extended Update Support (RPMs) 7.2 kB/s | 4.1 kB 00:00 Red Hat Enterprise Linux 9 for x86_64 - AppStream - Extended Update Support (RPMs) 8.0 kB/s | 4.1 kB 00:00 Red Hat OpenStack Platform Beta for RHEL 9 x86_64 (RPMs) 1.7 kB/s | 3.8 kB 00:02 Red Hat Enterprise Linux 9 for x86_64 - High Availability - Extended Update Support (RPMs) 2.5 kB/s | 4.0 kB 00:01 Fast Datapath for RHEL 9 x86_64 (RPMs) 2.4 kB/s | 4.0 kB 00:01 Dependencies resolved. ============================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================== Installing: podman x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 12 M Installing dependencies: conmon x86_64 2:2.1.2-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 56 k container-selinux noarch 3:2.188.0-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 52 k containers-common x86_64 2:1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 116 k criu x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-eus-rpms 512 k criu-libs x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-eus-rpms 33 k crun x86_64 1.4.5-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 189 k fuse-common x86_64 3.10.2-5.el9 rhel-9-for-x86_64-baseos-eus-rpms 9.3 k fuse-overlayfs x86_64 1.9-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 74 k fuse3 x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-eus-rpms 58 k fuse3-libs x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-eus-rpms 94 k iptables-nft x86_64 1.8.7-28.el9 rhel-9-for-x86_64-baseos-eus-rpms 208 k libnet x86_64 1.2-6.el9 rhel-9-for-x86_64-appstream-eus-rpms 61 k libnftnl x86_64 1.1.9-4.el9 rhel-9-for-x86_64-baseos-eus-rpms 87 k libslirp x86_64 4.4.0-7.el9 rhel-9-for-x86_64-appstream-eus-rpms 72 k netavark x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 2.0 M nftables x86_64 1:0.9.8-12.el9 rhel-9-for-x86_64-baseos-eus-rpms 367 k podman-catatonit x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 354 k shadow-utils-subid x86_64 2:4.9-3.el9 rhel-9-for-x86_64-baseos-eus-rpms 91 k slirp4netns x86_64 1.2.0-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 49 k yajl x86_64 2.1.0-21.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 42 k Installing weak dependencies: aardvark-dns x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 1.0 M Transaction Summary ============================================================================================================================================================================================================================================== Install 22 Packages Total download size: 18 M Installed size: 63 M Is this ok [y/N]: ^COperation aborted. [root@node-0 ~]# touch /run/.containerenv [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Subscription Manager is operating in container mode. Red Hat Enterprise Linux 9 for x86_64 - BaseOS - Extended Update Support (RPMs) 0.0 B/s | 0 B 00:00 Errors during downloading metadata for repository 'rhel-9-for-x86_64-baseos-eus-rpms': - Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://cdn.redhat.com/content/eus/rhel9/9/x86_64/baseos/os/repodata/repomd.xml [error setting certificate file: %(ca_cert_dir)sredhat-uep.pem] Error: Failed to download metadata for repo 'rhel-9-for-x86_64-baseos-eus-rpms': Cannot download repomd.xml: Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://cdn.redhat.com/content/eus/rhel9/9/x86_64/baseos/os/repodata/repomd.xml [error setting certificate file: %(ca_cert_dir)sredhat-uep.pem] [root@node-0 ~]# rm /run/.containerenv rm: remove regular empty file '/run/.containerenv'? y [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Red Hat Enterprise Linux 9 for x86_64 - BaseOS - Extended Update Support (RPMs) 8.4 kB/s | 4.1 kB 00:00 Red Hat Enterprise Linux 9 for x86_64 - AppStream - Extended Update Support (RPMs) 8.4 kB/s | 4.1 kB 00:00 Red Hat OpenStack Platform Beta for RHEL 9 x86_64 (RPMs) 7.6 kB/s | 3.8 kB 00:00 Red Hat Enterprise Linux 9 for x86_64 - High Availability - Extended Update Support (RPMs) 7.9 kB/s | 4.0 kB 00:00 Fast Datapath for RHEL 9 x86_64 (RPMs) 8.4 kB/s | 4.0 kB 00:00 Dependencies resolved. ============================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================== Installing: podman x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 12 M Installing dependencies: conmon x86_64 2:2.1.2-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 56 k container-selinux noarch 3:2.188.0-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 52 k containers-common x86_64 2:1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 116 k criu x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-eus-rpms 512 k criu-libs x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-eus-rpms 33 k crun x86_64 1.4.5-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 189 k fuse-common x86_64 3.10.2-5.el9 rhel-9-for-x86_64-baseos-eus-rpms 9.3 k fuse-overlayfs x86_64 1.9-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 74 k fuse3 x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-eus-rpms 58 k fuse3-libs x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-eus-rpms 94 k iptables-nft x86_64 1.8.7-28.el9 rhel-9-for-x86_64-baseos-eus-rpms 208 k libnet x86_64 1.2-6.el9 rhel-9-for-x86_64-appstream-eus-rpms 61 k libnftnl x86_64 1.1.9-4.el9 rhel-9-for-x86_64-baseos-eus-rpms 87 k libslirp x86_64 4.4.0-7.el9 rhel-9-for-x86_64-appstream-eus-rpms 72 k netavark x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 2.0 M nftables x86_64 1:0.9.8-12.el9 rhel-9-for-x86_64-baseos-eus-rpms 367 k podman-catatonit x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 354 k shadow-utils-subid x86_64 2:4.9-3.el9 rhel-9-for-x86_64-baseos-eus-rpms 91 k slirp4netns x86_64 1.2.0-2.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 49 k yajl x86_64 2.1.0-21.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 42 k Installing weak dependencies: aardvark-dns x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-eus-rpms 1.0 M Transaction Summary ============================================================================================================================================================================================================================================== Install 22 Packages Total download size: 18 M Installed size: 63 M Is this ok [y/N]: N Operation aborted.
The problem can be workarounded if I use non-eus repositories, and the issue seems to be related how dnf works with EUS repos. Using non-EUS repos might be a valid and only workaround we have now but as we need to pin base RHEL to RHEL9.0 we need to establish any solution before RHEL9.1 release. [root@node-0 ~]# subscription-manager repos \ --enable=rhel-9-for-x86_64-baseos-rpms \ --enable=rhel-9-for-x86_64-appstream-rpms \ --enable=rhel-9-for-x86_64-highavailability-rpms \ --enable=openstack-beta-for-rhel-9-x86_64-rpms \ --enable=fast-datapath-for-rhel-9-x86_64-rpms Repository 'rhel-9-for-x86_64-baseos-rpms' is enabled for this system. Repository 'rhel-9-for-x86_64-appstream-rpms' is enabled for this system. Repository 'rhel-9-for-x86_64-highavailability-rpms' is enabled for this system. Repository 'openstack-beta-for-rhel-9-x86_64-rpms' is enabled for this system. Repository 'fast-datapath-for-rhel-9-x86_64-rpms' is enabled for this system. [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Red Hat Enterprise Linux 9 for x86_64 - High Availability (RPMs) 252 kB/s | 427 kB 00:01 Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs) 1.8 MB/s | 3.6 MB 00:02 Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs) 3.7 MB/s | 9.2 MB 00:02 Last metadata expiration check: 0:00:01 ago on Thu Aug 18 11:22:52 2022. Dependencies resolved. ============================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================== Installing: podman x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-rpms 12 M Installing dependencies: conmon x86_64 2:2.1.2-2.el9_0 rhel-9-for-x86_64-appstream-rpms 56 k container-selinux noarch 3:2.188.0-1.el9_0 rhel-9-for-x86_64-appstream-rpms 52 k containers-common x86_64 2:1-36.el9_0 rhel-9-for-x86_64-appstream-rpms 116 k criu x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-rpms 512 k criu-libs x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-rpms 33 k crun x86_64 1.4.5-2.el9_0 rhel-9-for-x86_64-appstream-rpms 189 k fuse-common x86_64 3.10.2-5.el9 rhel-9-for-x86_64-baseos-rpms 9.3 k fuse-overlayfs x86_64 1.9-1.el9_0 rhel-9-for-x86_64-appstream-rpms 74 k fuse3 x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-rpms 58 k fuse3-libs x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-rpms 94 k iptables-nft x86_64 1.8.7-28.el9 rhel-9-for-x86_64-baseos-rpms 208 k libnet x86_64 1.2-6.el9 rhel-9-for-x86_64-appstream-rpms 61 k libnftnl x86_64 1.1.9-4.el9 rhel-9-for-x86_64-baseos-rpms 87 k libslirp x86_64 4.4.0-7.el9 rhel-9-for-x86_64-appstream-rpms 72 k netavark x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-rpms 2.0 M nftables x86_64 1:0.9.8-12.el9 rhel-9-for-x86_64-baseos-rpms 367 k podman-catatonit x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-rpms 354 k shadow-utils-subid x86_64 2:4.9-3.el9 rhel-9-for-x86_64-baseos-rpms 91 k slirp4netns x86_64 1.2.0-2.el9_0 rhel-9-for-x86_64-appstream-rpms 49 k yajl x86_64 2.1.0-21.el9_0 rhel-9-for-x86_64-appstream-rpms 42 k Installing weak dependencies: aardvark-dns x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-rpms 1.0 M Transaction Summary ============================================================================================================================================================================================================================================== Install 22 Packages Total download size: 18 M Installed size: 63 M Is this ok [y/N]: n Operation aborted. [root@node-0 ~]# ln -s /etc/rhsm /etc/rhsm-host [root@node-0 ~]# dnf install podman Failed to set locale, defaulting to C.UTF-8 Updating Subscription Management repositories. Subscription Manager is operating in container mode. Red Hat Enterprise Linux 9 for x86_64 - High Availability (RPMs) 218 kB/s | 427 kB 00:01 Red Hat OpenStack Platform Beta for RHEL 9 x86_64 (RPMs) 7.4 kB/s | 3.8 kB 00:00 Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs) 2.1 MB/s | 3.6 MB 00:01 Fast Datapath for RHEL 9 x86_64 (RPMs) 8.5 kB/s | 4.0 kB 00:00 Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs) 4.8 MB/s | 9.2 MB 00:01 Dependencies resolved. ============================================================================================================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================================================================================================== Installing: podman x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-rpms 12 M Installing dependencies: conmon x86_64 2:2.1.2-2.el9_0 rhel-9-for-x86_64-appstream-rpms 56 k container-selinux noarch 3:2.188.0-1.el9_0 rhel-9-for-x86_64-appstream-rpms 52 k containers-common x86_64 2:1-36.el9_0 rhel-9-for-x86_64-appstream-rpms 116 k criu x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-rpms 512 k criu-libs x86_64 3.15-13.el9 rhel-9-for-x86_64-appstream-rpms 33 k crun x86_64 1.4.5-2.el9_0 rhel-9-for-x86_64-appstream-rpms 189 k fuse-common x86_64 3.10.2-5.el9 rhel-9-for-x86_64-baseos-rpms 9.3 k fuse-overlayfs x86_64 1.9-1.el9_0 rhel-9-for-x86_64-appstream-rpms 74 k fuse3 x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-rpms 58 k fuse3-libs x86_64 3.10.2-5.el9 rhel-9-for-x86_64-appstream-rpms 94 k iptables-nft x86_64 1.8.7-28.el9 rhel-9-for-x86_64-baseos-rpms 208 k libnet x86_64 1.2-6.el9 rhel-9-for-x86_64-appstream-rpms 61 k libnftnl x86_64 1.1.9-4.el9 rhel-9-for-x86_64-baseos-rpms 87 k libslirp x86_64 4.4.0-7.el9 rhel-9-for-x86_64-appstream-rpms 72 k netavark x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-rpms 2.0 M nftables x86_64 1:0.9.8-12.el9 rhel-9-for-x86_64-baseos-rpms 367 k podman-catatonit x86_64 2:4.1.1-1.el9_0 rhel-9-for-x86_64-appstream-rpms 354 k shadow-utils-subid x86_64 2:4.9-3.el9 rhel-9-for-x86_64-baseos-rpms 91 k slirp4netns x86_64 1.2.0-2.el9_0 rhel-9-for-x86_64-appstream-rpms 49 k yajl x86_64 2.1.0-21.el9_0 rhel-9-for-x86_64-appstream-rpms 42 k Installing weak dependencies: aardvark-dns x86_64 2:1.0.1-36.el9_0 rhel-9-for-x86_64-appstream-rpms 1.0 M Transaction Summary ============================================================================================================================================================================================================================================== Install 22 Packages Total download size: 18 M Installed size: 63 M Is this ok [y/N]: ^COperation aborted.
(In reply to Takashi Kajinami from comment #8) > The problem can be workarounded if I use non-eus repositories, and the issue > seems to be related how dnf works with EUS repos. So ... this is not a complete workaround. When /etc/rhsm-host exists, dnf command succeeds but I noticed release pin was removed from yum repository settings. ~~~ [rhel-9-for-x86_64-baseos-rpms] name = Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs) baseurl = https://cdn.redhat.com/content/dist/rhel9/$releasever/x86_64/baseos/os ... ~~~ This should look like ~~~ [rhel-9-for-x86_64-baseos-rpms] name = Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs) baseurl = https://cdn.redhat.com/content/dist/rhel9/9.0/x86_64/baseos/os ... ~~~
Interestingly /run/.containerenv is not present in undercloud, though ironic-conductor container bind-mounts /run. On the other hand, the file is created in overcloud nodes and subscription-manager is blocked. ~~~ [root@overcloud-controller-0 ~]# sudo ls /run/.containerenv /run/.containerenv [root@overcloud-controller-0 ~]# sudo ls -d /etc/rhsm* /etc/rhsm [root@overcloud-controller-0 ~]# subscription-manager list subscription-manager is disabled when running inside a container. Please refer to your host system for subscription management. ~~~ ~~~ [root@overcloud-novacompute-0 ~]# sudo ls /run/.containerenv /run/.containerenv [root@overcloud-novacompute-0 ~]# sudo ls -d /etc/rhsm* /etc/rhsm [root@overcloud-novacompute-0 ~]# subscription-manager list subscription-manager is disabled when running inside a container. Please refer to your host system for subscription management. ~~~
OK so it seems the deployment has different version of podman installed in undercloud and overcloud undercloud: podman-4.1.1-1.el9_0.x86_64 overcloud: podman-4.0.2-7.el9_0.x86_64 After updating podman to 4.1.1 in overcloud, subscription-manager in overcloud also works fine. I'll create a separate bz to request bumping podman version in oc image.
*** Bug 2120171 has been marked as a duplicate of this bug. ***
(package hasn't been built yet apparently, sorry for the noise!) trunk build is already here (openstack-tripleo-heat-templates-14.3.1-0.20220823060600.feca772.el9osttrunk) but we're missing the non-trunk
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:6543