Created attachment 1712206 [details] full_cmd_iine_output Description of problem: TASK [Wait for containers to start for step 3 using paunch] Version-Release number of selected component (if applicable): FFU 13z12 16.1.1(latest passed phase2 How reproducible: I am trying to perform an FFU upgrade using https://gitlab.cee.redhat.com/osp16/ffwd2#run-the-undercloud-upgrade Actual results: Please see attachment for full log TASK [Start containers for step 3 using paunch] ******************************************************************************************************************************************************************* Friday 21 August 2020 20:10:58 +0300 (0:00:00.105) 0:05:13.161 ********* changed: [sealusa40] TASK [Wait for containers to start for step 3 using paunch] ******************************************************************************************************************************************************* Friday 21 August 2020 20:10:59 +0300 (0:00:00.364) 0:05:13.526 ********* WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1200 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1199 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1198 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1197 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1196 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1195 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1194 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1193 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1192 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1191 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1190 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1189 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1188 retries left). WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1187 retries left). fatal: [sealusa40]: FAILED! => {"ansible_job_id": "88444727671.805453", "attempts": 15, "changed": false, "finished": 1, "msg": "Paunch failed with config_id tripleo_step3", "rc": 1, "stderr": "Error executing ['podman', 'container', 'exists', 'glance_api_db_sync_stein']: returned 1\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync_stein', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync_stein', '--format', '{{.Names}}']\"\nError executing ['podman', 'run', '--name', 'glance_api_db_sync_stein', '--label', 'config_id=tripleo_step3', '--label', 'container_name=glance_api_db_sync_stein', '--label', 'managed_by=tripleo-Undercloud', '--label', 'config_data={\"command\": [\"/usr/bin/bootstrap_host_exec\", \"glance_api\", \"/container-config-scripts/glance_ffu_db_sync.sh\"], \"detach\": false, \"environment\": {\"KOLLA_BOOTSTRAP\": true, \"KOLLA_CONFIG_STRATEGY\": \"COPY_ALWAYS\", \"TRIPLEO_DEPLOY_IDENTIFIER\": \"1598029257\", \"TRIPLEO_CONFIG_HASH\": \"a6bd1aa8a1c299091fb1c9ce9cb82f31\"}, \"image\": \"sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp15-openstack-glance-api:20200115.1\", \"net\": \"host\", \"privileged\": false, \"start_order\": 0, \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/log/containers/glance:/var/log/glance:z\", \"/var/log/containers/httpd/glance:/var/log/httpd:z\", \"/var/lib/kolla/config_files/glance_api.json:/var/lib/kolla/config_files/config.json\", \"/var/lib/config-data/puppet-generated/glance_api:/var/lib/kolla/config_files/src:ro\", \"/etc/ceph:/var/lib/kolla/config_files/src-ceph:ro\", \"/var/lib/glance:/var/lib/glance:slave\", \"/var/lib/container-config-scripts/:/container-config-scripts/:ro\"]}', '--conmon-pidfile=/var/run/glance_api_db_sync_stein.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/glance_api_db_sync_stein.log', '--env=KOLLA_BOOTSTRAP=True', '--env=KOLLA_CONFIG_STRATEGY=COPY_ALWAYS', '--env=TRIPLEO_CONFIG_HASH=a6bd1aa8a1c299091fb1c9ce9cb82f31', '--env=TRIPLEO_DEPLOY_IDENTIFIER=1598029257', '--net=host', '--privileged=false', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/log/containers/glance:/var/log/glance:z', '--volume=/var/log/containers/httpd/glance:/var/log/httpd:z', '--volume=/var/lib/kolla/config_files/glance_api.json:/var/lib/kolla/config_files/config.json', '--volume=/var/lib/config-data/puppet-generated/glance_api:/var/lib/kolla/config_files/src:ro', '--volume=/etc/ceph:/var/lib/kolla/config_files/src-ceph:ro', '--volume=/var/lib/glance:/var/lib/glance:slave', '--volume=/var/lib/container-config-scripts/:/container-config-scripts/:ro', 'sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp15-openstack-glance-api:20200115.1', '/usr/bin/bootstrap_host_exec', 'glance_api', '/container-config-scripts/glance_ffu_db_sync.sh']: returned 1\nError running ['podman', 'run', '--name', 'glance_api_db_sync_stein', '--label', 'config_id=tripleo_step3', '--label', 'container_name=glance_api_db_sync_stein', '--label', 'managed_by=tripleo-Undercloud', '--label', 'config_data={\"command\": [\"/usr/bin/bootstrap_host_exec\", \"glance_api\", \"/container-config-scripts/glance_ffu_db_sync.sh\"], \"detach\": false, \"environment\": {\"KOLLA_BOOTSTRAP\": true, \"KOLLA_CONFIG_STRATEGY\": \"COPY_ALWAYS\", \"TRIPLEO_DEPLOY_IDENTIFIER\": \"1598029257\", \"TRIPLEO_CONFIG_HASH\": \"a6bd1aa8a1c299091fb1c9ce9cb82f31\"}, \"image\": \"sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp15-openstack-glance-api:20200115.1\", \"net\": \"host\", \"privileged\": false, \"start_order\": 0, \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/log/containers/glance:/var/log/glance:z\", \"/var/log/containers/httpd/glance:/var/log/httpd:z\", \"/var/lib/kolla/config_files/glance_api.json:/var/lib/kolla/config_files/config.json\", \"/var/lib/config-data/puppet-generated/glance_api:/var/lib/kolla/config_files/src:ro\", \"/etc/ceph:/var/lib/kolla/config_files/src-ceph:ro\", \"/var/lib/glance:/var/lib/glance:slave\", \"/var/lib/container-config-scripts/:/container-config-scripts/:ro\"]}', '--conmon-pidfile=/var/run/glance_api_db_sync_stein.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/glance_api_db_sync_stein.log', '--env=KOLLA_BOOTSTRAP=True', '--env=KOLLA_CONFIG_STRATEGY=COPY_ALWAYS', '--env=TRIPLEO_CONFIG_HASH=a6bd1aa8a1c299091fb1c9ce9cb82f31', '--env=TRIPLEO_DEPLOY_IDENTIFIER=1598029257', '--net=host', '--privileged=false', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/log/containers/glance:/var/log/glance:z', '--volume=/var/log/containers/httpd/glance:/var/log/httpd:z', '--volume=/var/lib/kolla/config_files/glance_api.json:/var/lib/kolla/config_files/config.json', '--volume=/var/lib/config-data/puppet-generated/glance_api:/var/lib/kolla/config_files/src:ro', '--volume=/etc/ceph:/var/lib/kolla/config_files/src-ceph:ro', '--volume=/var/lib/glance:/var/lib/glance:slave', '--volume=/var/lib/container-config-scripts/:/container-config-scripts/:ro', 'sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp15-openstack-glance-api:20200115.1', '/usr/bin/bootstrap_host_exec', 'glance_api', '/container-config-scripts/glance_ffu_db_sync.sh']. [1]\n\nstdout: \nstderr: Cannot find config file: /etc/puppet/hiera.yaml\n\nError executing ['podman', 'container', 'exists', 'heat_engine_db_sync']: returned 1\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--format', '{{.Names}}']\"\nError executing ['podman', 'run', '--name', 'heat_engine_db_sync', '--label', 'config_id=tripleo_step3', '--label', 'container_name=heat_engine_db_sync', '--label', 'managed_by=tripleo-Undercloud', '--label', 'config_data={\"command\": \"/usr/bin/bootstrap_host_exec heat_engine su heat -s /bin/bash -c \\'heat-manage db_sync\\'\", \"detach\": false, \"environment\": {\"TRIPLEO_DEPLOY_IDENTIFIER\": \"1598029257\", \"TRIPLEO_CONFIG_HASH\": \"a57c781e7258833d6af4bc0b6f9e05a3-a57c781e7258833d6af4bc0b6f9e05a3\"}, \"image\": \"sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-heat-engine:16.1_20200730.1\", \"net\": \"host\", \"privileged\": false, \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/log/containers/heat:/var/log/heat:z\", \"/var/lib/config-data/heat/etc/my.cnf.d/tripleo.cnf:/etc/my.cnf.d/tripleo.cnf:ro\", \"/var/lib/config-data/heat/etc/heat/:/etc/heat/:ro\"]}', '--conmon-pidfile=/var/run/heat_engine_db_sync.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/heat_engine_db_sync.log', '--env=TRIPLEO_CONFIG_HASH=a57c781e7258833d6af4bc0b6f9e05a3-a57c781e7258833d6af4bc0b6f9e05a3', '--env=TRIPLEO_DEPLOY_IDENTIFIER=1598029257', '--net=host', '--privileged=false', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/log/containers/heat:/var/log/heat:z', '--volume=/var/lib/config-data/heat/etc/my.cnf.d/tripleo.cnf:/etc/my.cnf.d/tripleo.cnf:ro', '--volume=/var/lib/config-data/heat/etc/heat/:/etc/heat/:ro', 'sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-heat-engine:16.1_20200730.1', '/usr/bin/bootstrap_host_exec', 'heat_engine', 'su', 'heat', '-s', '/bin/bash', '-c', \"'heat-manage\", \"db_sync'\"]: returned 1\nError running ['podman', 'run', '--name', 'heat_engine_db_sync', '--label', 'config_id=tripleo_step3', '--label', 'container_name=heat_engine_db_sync', '--label', 'managed_by=tripleo-Undercloud', '--label', 'config_data={\"command\": \"/usr/bin/bootstrap_host_exec heat_engine su heat -s /bin/bash -c \\'heat-manage db_sync\\'\", \"detach\": false, \"environment\": {\"TRIPLEO_DEPLOY_IDENTIFIER\": \"1598029257\", \"TRIPLEO_CONFIG_HASH\": \"a57c781e7258833d6af4bc0b6f9e05a3-a57c781e7258833d6af4bc0b6f9e05a3\"}, \"image\": \"sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-heat-engine:16.1_20200730.1\", \"net\": \"host\", \"privileged\": false, \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/log/containers/heat:/var/log/heat:z\", \"/var/lib/config-data/heat/etc/my.cnf.d/tripleo.cnf:/etc/my.cnf.d/tripleo.cnf:ro\", \"/var/lib/config-data/heat/etc/heat/:/etc/heat/:ro\"]}', '--conmon-pidfile=/var/run/heat_engine_db_sync.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/heat_engine_db_sync.log', '--env=TRIPLEO_CONFIG_HASH=a57c781e7258833d6af4bc0b6f9e05a3-a57c781e7258833d6af4bc0b6f9e05a3', '--env=TRIPLEO_DEPLOY_IDENTIFIER=1598029257', '--net=host', '--privileged=false', '--user=root', '--volume=/etc/hosts:/etc/hosts:ro', '--volume=/etc/localtime:/etc/localtime:ro', '--volume=/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume=/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro', '--volume=/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume=/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume=/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume=/dev/log:/dev/log', '--volume=/etc/puppet:/etc/puppet:ro', '--volume=/var/log/containers/heat:/var/log/heat:z', '--volume=/var/lib/config-data/heat/etc/my.cnf.d/tripleo.cnf:/etc/my.cnf.d/tripleo.cnf:ro', '--volume=/var/lib/config-data/heat/etc/heat/:/etc/heat/:ro', 'sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-heat-engine:16.1_20200730.1', '/usr/bin/bootstrap_host_exec', 'heat_engine', 'su', 'heat', '-s', '/bin/bash', '-c', \"'heat-manage\", \"db_sync'\"]. [1]\n\nstdout: \nstderr: Cannot find config file: /etc/puppet/hiera.yaml\n\nError executing ['podman', 'container', 'exists', 'heat_engine_db_sync_stein']: returned 1\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync_stein', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id\nDid not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync_stein', '--format', '{{.Names}}']\"\nError executing ['podman', 'run', '--name', 'heat_engine_db_sync_stein', '--label', 'config_id=tripleo_step3', '--label', 'container_name=heat_engine_db_sync_stein', '--label', 'managed_by=tripleo-Undercloud', '--label', 'config_data={\"command\": \"/usr/bin/bootstrap_host_exec heat_engine su heat -s /bin/bash -c \\'heat-manage db_sync\\'\", \"detach\": false, \"environment\": {\"TRIPLEO_DEPLOY_IDENTIFIER\": \"1598029257\", \"TRIPLEO_CONFIG_HASH\": \"a57c781e7258833d6af4bc0b6f9e05a3-a57c781e7258833d6af4bc0b6f9e05a3\"}, \"image\": \"sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp15-openstack-heat-engine:20200115.1\", \"net\": \"host\", \"privileged\": false, \"user\": \"root\", \"volumes\": [\"/etc/hosts:/etc/hosts:ro\", \"/etc/localtime:/etc/localtime:ro\", \"/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro\", \"/etc/pki/ca-trust/source/anchors:/etc/pki/ca-trust/source/anchors:ro\", \"/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro\", \"/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro\", \"/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro\", \"/dev/log:/dev/log\", \"/etc/puppet:/etc/puppet:ro\", \"/var/log/containers/heat:/var/log/heat:z\", \"/var/lib/config-data/heat/etc/my.cnf.d/tripleo.cnf:/etc/my.cnf.d/tripleo.cnf:ro\", \"/var/lib/config-data/heat/etc/heat/:/etc/heat/:ro\"]}', '--conmon-pidfile=/var/run/heat_engine_db_sync_stein.pid', '--log-driver', 'k8s-file', '--log-opt', 'path=/var/log/containers/stdouts/heat_engine_db_sync_stein.log', I kicked of the undercloud upgrade last night and it failed, I tried it again this afternoon (undercloud) [stack@sealusa40 ~]$ sudo podman ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 951153cbeb0e sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-keystone:16.1_20200730.1 kolla_start 22 hours ago Up 22 hours ago keystone 5aaaed492d0e sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-iscsid:16.1_20200730.1 kolla_start 22 hours ago Up 22 hours ago iscsid 2e0b811045c0 sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-mariadb:16.1_20200730.1 kolla_start 22 hours ago Up 22 hours ago mysql 32a6d62d9f0a sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-rabbitmq:16.1_20200730.1 kolla_start 22 hours ago Up 22 hours ago rabbitmq 10a80b433236 sealusa40.ctlplane.localdomain:8787/rh-osbs/rhosp16-openstack-memcached:16.1_20200730.1 kolla_start 22 hours ago Up 22 hours ago memcached I also noticed starting at 20:09:15 +0300 TASK [Debug output for task: Run puppet host configuration for step 1] ******************************************************************************************************************************************** Friday 21 August 2020 20:09:15 +0300 (0:00:06.636) 0:03:29.619 ********* changed: [sealusa40] => { "failed_when_result": false, "puppet_host_outputs.stdout_lines | default([]) | union(puppet_host_outputs.stderr_lines | default([]))": [ "<13>Aug 21 20:09:08 puppet-user: Warning: The function 'hiera' is deprecated in favor of using 'lookup'. See https://puppet.com/docs/ Expected results: undercloud successfully upgrades Additional info: baremetal system sealusa40.mobius.lab.eng.rdu2.redhat.com stack\12345678
Closing this BZ as it is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1829609. The workaround to solve this problem is to reinstall containers-selinux RPM and the cause is that the right containers-tools DNF module wasn't enabled before starting the Undercloud Upgrade, as a consequence a bad version of containers-selinux and podman got installed and caused the failure. This should be solved by the following BZ in the future, in the case customers miss to set up the right DNF module we will ensure it is enabled in tripleo-heat-templates: https://bugzilla.redhat.com/show_bug.cgi?id=1866479 *** This bug has been marked as a duplicate of bug 1829609 ***