Bug 1948328

Summary: UC install fails with AttributeError: 'PodmanRunner' object has no attribute 'container_exists'
Product: Red Hat OpenStack Reporter: Pavel Sedlák <psedlak>
Component: python-paunchAssignee: Steve Baker <sbaker>
Status: CLOSED ERRATA QA Contact: nlevinki <nlevinki>
Severity: high Docs Contact:
Priority: high    
Version: 16.1 (Train)CC: aschultz, jhajyahy, mburns, ramishra
Target Milestone: z6Keywords: Triaged
Target Release: 16.1 (Train on RHEL 8.2)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-paunch-5.3.3-1.20210412123423.ed2c015.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-26 13:52:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Pavel Sedlák 2021-04-11 22:15:59 UTC 
UC install fails with AttributeError: 'PodmanRunner' object has no attribute 'container_exists'

Description of problem:
> openstack undercloud install
fails, it's ansible output indicates that it failed during waiting for containers
to start for step3 using paunch ...

although these containers seems exist and already exited (later after the failure).
(At least some I've checked like heat_engine_db_sync, neutron_ovs_bridge,
 ... missing seems mysql_sync_credentials though it was checking for one named mysql instead and that one is there.)

But interesting is that the waiting for containers loop/retries seems aborted/failed too early,
before all the printed-out retries amount was reached.



There seems to be two outstanding interesting Tracebacks in the output too.

First on stdout, maybe it could be indication for the second traceback:
> ValueError: No start of json char found"
though i have no clue what json could be missing where.

Second, and seems possibly important for the failure is:
> AttributeError: 'PodmanRunner' object has no attribute 'container_exists'
as that one maybe is related to why it stops RETRYING THE WAIT before the retries/attempts count reaches the limit.
(Notice the 'attemps' and 'retries left' counts in the output.)




Final failing task in output from undercloud installation:
> 2021-04-09 22:22:40.618 6321 WARNING tripleoclient.v1.tripleo_deploy.Deploy [  ] WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1183 retries left).
> 2021-04-09 22:22:43.812 6321 WARNING tripleoclient.v1.tripleo_deploy.Deploy [  ] fatal: [undercloud-0]: FAILED! =>
> {"ansible_job_id": "160701373548.32264",
> "attempts": 19,
> "changed": false,
> "cmd": "/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py",
> "data": "",
> "finished": 1,
> "msg": "
>   Traceback (most recent call last):
>     File \"/tmp/ansible_async_wrapper_payload_fnqz7765/ansible_async_wrapper_payload.zip/ansible/modules/utilities/logic/async_wrapper.py\", line 166, in _run_module
>     File \"/tmp/ansible_async_wrapper_payload_fnqz7765/ansible_async_wrapper_payload.zip/ansible/modules/utilities/logic/async_wrapper.py\", line 94, in _filter_non_json_lines
>   ValueError: No start of json char found",
> "stderr": "
>   Error executing ['podman', 'container', 'exists', 'heat_engine_db_sync']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'ironic_inspector_init_log']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_log', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_log', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'neutron_ovs_bridge']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_ovs_bridge', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_ovs_bridge', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'nova_statedir_owner']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_statedir_owner', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_statedir_owner', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'swift_copy_rings']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_copy_rings', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_copy_rings', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'zaqar_db_sync']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=zaqar_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=zaqar_db_sync', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'glance_api_db_sync']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'ironic_inspector_init_dnsmasq_dhcp_hostsdir']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_dnsmasq_dhcp_hostsdir', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_dnsmasq_dhcp_hostsdir', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'keystone_db_sync']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone_db_sync', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'mistral_db_sync']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mistral_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mistral_db_sync', '--format', '{{.Names}}']\"
>   Error executing ['podman', 'container', 'exists', 'mysql_sync_credentials']: returned 1
>   Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mysql', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id
>   Traceback (most recent call last):
>     File \"/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py\", line 102, in <module>
>       _ansiballz_main()
>     File \"/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py\", line 94, in _ansiballz_main
>       invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
>     File \"/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py\", line 40, in invoke_module
>       runpy.run_module(mod_name='ansible.modules.paunch', init_globals=None, run_name='__main__', alter_sys=True)
>     File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module
>       return _run_module_code(code, init_globals, run_name, mod_spec)
>     File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code
>       mod_name, mod_spec, pkg_name, script_name)
>     File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code
>       exec(code, run_globals)
>     File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 253, in <module>
>     File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 249, in main
>     File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 180, in __init__
>     File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 204, in paunch_apply
>     File \"/usr/lib/python3.6/site-packages/paunch/__init__.py\", line 84, in apply
>       return builder.apply()
>     File \"/usr/lib/python3.6/site-packages/paunch/builder/base.py\", line 132, in apply
>       if not c_name or not self.runner.container_running(c_name):
>     File \"/usr/lib/python3.6/site-packages/paunch/runner.py\", line 498, in container_running
>       if not self.container_exists(container):
>   AttributeError: 'PodmanRunner' object has no attribute 'container_exists'
> ",
> "stderr_lines": ["Error executing ['podman', 'container', 'exists', 'heat_engine_db_sync']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'ironic_inspector_init_log']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_log', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_log', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'neutron_ovs_bridge']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_ovs_bridge', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_ovs_bridge', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'nova_statedir_owner']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_statedir_owner', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_statedir_owner', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'swift_copy_rings']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_copy_rings', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_copy_rings', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'zaqar_db_sync']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=zaqar_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=zaqar_db_sync', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'glance_api_db_sync']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'ironic_inspector_init_dnsmasq_dhcp_hostsdir']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_dnsmasq_dhcp_hostsdir', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_dnsmasq_dhcp_hostsdir', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'keystone_db_sync']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone_db_sync', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'mistral_db_sync']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mistral_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mistral_db_sync', '--format', '{{.Names}}']\"", "Error executing ['podman', 'container', 'exists', 'mysql_sync_credentials']: returned 1", "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mysql', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id", "Traceback (most recent call last):", "  File \"/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py\", line 102, in <module>", "    _ansiballz_main()", "  File \"/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py\", line 94, in _ansiballz_main", "    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)", "  File \"/root/.ansible/tmp/ansible-tmp-1618021305.0634766-32252-67876202080949/AnsiballZ_paunch.py\", line 40, in invoke_module", "    runpy.run_module(mod_name='ansible.modules.paunch', init_globals=None, run_name='__main__', alter_sys=True)", "  File \"/usr/lib64/python3.6/runpy.py\", line 205, in run_module", "    return _run_module_code(code, init_globals, run_name, mod_spec)", "  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code", "    mod_name, mod_spec, pkg_name, script_name)", "  File \"/usr/lib64/python3.6/runpy.py\", line 85, in _run_code", "    exec(code, run_globals)", "  File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 253, in <module>", "  File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 249, in main", "  File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 180, in __init__", "  File \"/tmp/ansible_paunch_payload_1o7ncm9z/ansible_paunch_payload.zip/ansible/modules/paunch.py\", line 204, in paunch_apply", "  File \"/usr/lib/python3.6/site-packages/paunch/__init__.py\", line 84, in apply", "    return builder.apply()", "  File \"/usr/lib/python3.6/site-packages/paunch/builder/base.py\", line 132, in apply", "    if not c_name or not self.runner.container_running(c_name):", "  File \"/usr/lib/python3.6/site-packages/paunch/runner.py\", line 498, in container_running", "    if not self.container_exists(container):", "AttributeError: 'PodmanRunner' object has no attribute 'container_exists'"]
> }
> 2021-04-09 22:22:43.814 6321 WARNING tripleoclient.v1.tripleo_deploy.Deploy 


List of containers in podman as gathered in post-deployment logs:
> + podman ps --all
> CONTAINER ID  IMAGE                                                                                                  COMMAND               CREATED             STATUS                         PORTS  NAMES
> e8f42e3a918f  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-mistral-api:16.1_20210409.1          /usr/bin/bootstra...  About a minute ago  Exited (0) About a minute ago         mistral_db_sync
> 015917eee8a5  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-keystone:16.1_20210409.1             /usr/bin/bootstra...  About a minute ago  Exited (0) About a minute ago         keystone_db_sync
> 613bcdb4cace  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-ironic-inspector:16.1_20210409.1     /bin/bash -c chow...  About a minute ago  Exited (0) About a minute ago         ironic_inspector_init_dnsmasq_dhcp_hostsdir
> 1806b7646553  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-glance-api:16.1_20210409.1           /usr/bin/bootstra...  About a minute ago  Exited (0) About a minute ago         glance_api_db_sync
> 19b5c7591ef4  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-zaqar-wsgi:16.1_20210409.1           /usr/bin/bootstra...  2 minutes ago       Exited (0) About a minute ago         zaqar_db_sync
> 11707a69528b  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-swift-proxy-server:16.1_20210409.1   /bin/bash -c cp -...  2 minutes ago       Exited (0) 2 minutes ago              swift_copy_rings
> d2cdefad4a7f  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-nova-compute-ironic:16.1_20210409.1  /container-config...  2 minutes ago       Exited (0) 2 minutes ago              nova_statedir_owner
> 2847534ac350  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-neutron-server:16.1_20210409.1       puppet apply --mo...  2 minutes ago       Exited (0) 2 minutes ago              neutron_ovs_bridge
> b7518aa0981e  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-ironic-inspector:16.1_20210409.1     /bin/bash -c chow...  2 minutes ago       Exited (0) 2 minutes ago              ironic_inspector_init_log
> 9a5804495f30  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-heat-engine:16.1_20210409.1          /usr/bin/bootstra...  2 minutes ago       Exited (0) 2 minutes ago              heat_engine_db_sync
> 343e58bf779a  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-mariadb:16.1_20210409.1              /container-config...  3 minutes ago       Exited (0) 3 minutes ago              mysql_neutron_db_rename
> 5a77b7444418  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-mariadb:16.1_20210409.1              kolla_start           3 minutes ago       Up 3 minutes ago                      mysql
> 7151839ba529  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-placement-api:16.1_20210409.1        /bin/bash -c chow...  3 minutes ago       Exited (0) 3 minutes ago              placement_init_log
> b5342bb646ef  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-mariadb:16.1_20210409.1              bash -ec if [ -e ...  3 minutes ago       Exited (0) 3 minutes ago              mysql_bootstrap
> 1642db162a2d  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-keystone:16.1_20210409.1             /bin/bash -c chow...  3 minutes ago       Exited (0) 3 minutes ago              keystone_init_log
> c3723baafa7d  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-neutron-l3-agent:16.1_20210409.1     /container_puppet...  3 minutes ago       Exited (0) 3 minutes ago              create_keepalived_wrapper
> 9efffa4bd34b  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-neutron-dhcp-agent:16.1_20210409.1   /container_puppet...  4 minutes ago       Exited (0) 3 minutes ago              create_dnsmasq_wrapper
> b8ad12e42644  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-zaqar-wsgi:16.1_20210409.1           /bin/bash -c chmo...  4 minutes ago       Exited (0) 4 minutes ago              zaqar_init_log
> d40f6108408f  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-tempest:16.1_20210409.1              /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              tempest_init_logs
> 027395079eb3  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-nova-conductor:16.1_20210409.1       /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              nova_conductor_init_log
> 2276bea63917  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-nova-api:16.1_20210409.1             /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              nova_api_init_logs
> 10a24b23cf33  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-neutron-server:16.1_20210409.1       /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              neutron_init_logs
> 09aef42775c9  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-mistral-api:16.1_20210409.1          /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              mistral_init_logs
> 25ac69f3d166  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-ironic-api:16.1_20210409.1           /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              ironic_init_logs
> 1583923cd4dc  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-heat-engine:16.1_20210409.1          /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              heat_init_log
> 5943be90a231  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-glance-api:16.1_20210409.1           /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              glance_init_logs
> e113ecdeeee1  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-rabbitmq:16.1_20210409.1             kolla_start           4 minutes ago       Up 4 minutes ago                      rabbitmq
> 7777cc7329f5  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-rabbitmq:16.1_20210409.1             kolla_start           4 minutes ago       Exited (0) 4 minutes ago              rabbitmq_bootstrap
> 8f2f98a270d9  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-haproxy:16.1_20210409.1              kolla_start           4 minutes ago       Up 4 minutes ago                      haproxy
> 05e4aa348249  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-rabbitmq:16.1_20210409.1             /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              rabbitmq_init_logs
> 4a91e802c111  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-mariadb:16.1_20210409.1              /bin/bash -c chow...  4 minutes ago       Exited (0) 4 minutes ago              mysql_init_logs
> 5bec963761eb  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-memcached:16.1_20210409.1            kolla_start           4 minutes ago       Up 4 minutes ago                      memcached
> 27e50e1eb55e  undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhosp16-openstack-keepalived:16.1_20210409.1           /usr/local/bin/ko...  4 minutes ago       Up 4 minutes ago                      keepalived

Happens in OSP Phase1 CI run, with compose RHOS-16.1-RHEL-8-20210409.n.2.
Ans it seems to happen repeatedly / can be reproduced.
Comment 10 Pavel Sedlák 2021-04-13 22:36:53 UTC 
Some more details:
> python3-paunch.noarch                         5.3.3-1.20210310104358.ed2c015.el8ost           @rhelosp-16.1
was missing newer patchset of https://review.opendev.org/c/openstack/paunch/+/769186 in which the issue was already fixed.

Verified that with newer version
> python3-paunch.noarch                         5.3.3-1.20210412123423.ed2c015.el8ost           @rhelosp-16.1            
deployment passed and shows no signs of the issue.

Snippet of 'undercloud install' output when it worked:
> ...
> TASK [Wait for containers to start for step 3 using paunch] ********************
> Tuesday 13 April 2021  09:21:43 -0400 (0:00:00.366)       0:09:37.421 *********
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1200 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1199 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1198 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1197 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1196 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1195 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1194 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1193 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1192 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1191 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1190 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1189 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1188 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1187 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1186 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1185 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1184 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1183 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1182 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1181 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1180 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1179 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1178 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1177 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1176 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1175 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1174 retries left).
> WAITING FOR COMPLETION: Wait for containers to start for step 3 using paunch (1173 retries left).
> ok: [undercloud-0]
> 
> TASK [Debug output for task: Start containers for step 3] **********************
> Tuesday 13 April 2021  09:23:13 -0400 (0:01:29.512)       0:11:06.933 *********
> ok: [undercloud-0] => {
>     "failed_when_result": false,
>     "start_containers_outputs.stdout_lines | default([]) | union(start_containers_outputs.stderr_lines | default([]))": [
>         "2021-04-13 13:21:47.484 14 INFO migrate.versioning.api [-] 70 -> 71... \u001b[00m",
>         "2021-04-13 13:21:47.622 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.622 14 INFO migrate.versioning.api [-] 71 -> 72... \u001b[00m",
>         "2021-04-13 13:21:47.646 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.646 14 INFO migrate.versioning.api [-] 72 -> 73... \u001b[00m",
>         "2021-04-13 13:21:47.684 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.684 14 INFO migrate.versioning.api [-] 73 -> 74... \u001b[00m",
>         "2021-04-13 13:21:47.689 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.689 14 INFO migrate.versioning.api [-] 74 -> 75... \u001b[00m",
>         "2021-04-13 13:21:47.694 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.694 14 INFO migrate.versioning.api [-] 75 -> 76... \u001b[00m",
>         "2021-04-13 13:21:47.699 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.699 14 INFO migrate.versioning.api [-] 76 -> 77... \u001b[00m",
>         "2021-04-13 13:21:47.704 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.705 14 INFO migrate.versioning.api [-] 77 -> 78... \u001b[00m",
>         "2021-04-13 13:21:47.709 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.709 14 INFO migrate.versioning.api [-] 78 -> 79... \u001b[00m",
>         "2021-04-13 13:21:47.763 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.763 14 INFO migrate.versioning.api [-] 79 -> 80... \u001b[00m",
>         "2021-04-13 13:21:47.799 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.799 14 INFO migrate.versioning.api [-] 80 -> 81... \u001b[00m",
>         "2021-04-13 13:21:47.803 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.804 14 INFO migrate.versioning.api [-] 81 -> 82... \u001b[00m",
>         "2021-04-13 13:21:47.808 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.808 14 INFO migrate.versioning.api [-] 82 -> 83... \u001b[00m",
>         "2021-04-13 13:21:47.813 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.814 14 INFO migrate.versioning.api [-] 83 -> 84... \u001b[00m",
>         "2021-04-13 13:21:47.818 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.819 14 INFO migrate.versioning.api [-] 84 -> 85... \u001b[00m",
>         "2021-04-13 13:21:47.824 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "2021-04-13 13:21:47.824 14 INFO migrate.versioning.api [-] 85 -> 86... \u001b[00m",
>         "2021-04-13 13:21:47.868 14 INFO migrate.versioning.api [-] done\u001b[00m",
>         "",
>         "b493bba585523d812f8af6f94972af93d64f5775970cadc7183d4502e6e10ab0",
>         "\u001b[0;32mInfo: Loading facts\u001b[0m",
>         "\u001b[mNotice: Compiled catalog for undercloud-0.redhat.local in environment production in 0.60 seconds\u001b[0m",
>         "\u001b[0;32mInfo: Applying configuration version '1618320121'\u001b[0m",
>         "\u001b[0;32mInfo: Creating state file /var/lib/puppet/state/state.yaml\u001b[0m",
>         "\u001b[mNotice: Applied catalog in 0.15 seconds\u001b[0m",
>         "INFO:nova_statedir:Applying nova statedir ownership",
>         "INFO:nova_statedir:Target ownership for /var/lib/nova: 42436:42436",
>         "INFO:nova_statedir:Checking uid: 0 gid: 0 path: /var/lib/nova/",
>         "INFO:nova_statedir:Changing ownership of /var/lib/nova from 0:0 to 42436:42436",
>         "INFO:nova_statedir:Setting selinux context of /var/lib/nova to system_u:object_r:container_file_t:s0",
>         "INFO:nova_statedir:Nova statedir ownership complete",
>         "'/swift_ringbuilder/etc/swift/account.ring.gz' -> '/etc/swift/account.ring.gz'",
>         "'/swift_ringbuilder/etc/swift/container.ring.gz' -> '/etc/swift/container.ring.gz'",
>         "'/swift_ringbuilder/etc/swift/object.ring.gz' -> '/etc/swift/object.ring.gz'",
>         "'/swift_ringbuilder/etc/swift/account.builder' -> '/etc/swift/account.builder'",
>         "'/swift_ringbuilder/etc/swift/container.builder' -> '/etc/swift/container.builder'",
>         "'/swift_ringbuilder/etc/swift/object.builder' -> '/etc/swift/object.builder'",
>         "'/swift_ringbuilder/etc/swift/backups' -> '/etc/swift/backups'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319983.account.builder' -> '/etc/swift/backups/1618319983.account.builder'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319983.object.builder' -> '/etc/swift/backups/1618319983.object.builder'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319984.container.builder' -> '/etc/swift/backups/1618319984.container.builder'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319986.object.builder' -> '/etc/swift/backups/1618319986.object.builder'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319986.object.ring.gz' -> '/etc/swift/backups/1618319986.object.ring.gz'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319987.account.builder' -> '/etc/swift/backups/1618319987.account.builder'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319987.account.ring.gz' -> '/etc/swift/backups/1618319987.account.ring.gz'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319987.container.builder' -> '/etc/swift/backups/1618319987.container.builder'",
>         "'/swift_ringbuilder/etc/swift/backups/1618319987.container.ring.gz' -> '/etc/swift/backups/1618319987.container.ring.gz'",
>         "Upgraded database to: train_expand01, current revision(s): train_expand01",
>         "Database migration is up to date. No migration needed.",
>         "Upgraded database to: train_contract01, current revision(s): train_contract01",
>         "Database is synced successfully.",
>         "3ec9aa268f497340d11b63d4cf57f72b66d15416bb571ff1300f3178a1a0039c",
>         "Running upgrade for neutron ...",
>         "OK",
>         "Running upgrade for networking-l2gw ...",
>         "Running upgrade for networking-sfc ...",
>         "Running upgrade for neutron-dynamic-routing ...",
>         "Running upgrade for vmware-nsx ...",
>         "cf3b1bee53908a6f49a35163178a2aaa5cef98f49d6b80eb766001adf41f66e9",
>         "87f503bcd3299a92881eb7e26fe026537f94d716940448aae7d6b554dfcb29f8",
>         "4072327a417adde903b0ea89f3d3ab5107a96c721e904682dd672546b86bf604",
>         "No data present in nova database - nothing to migrate (new deployment?)",
>         "(cellv2) Creating default cell_v2 cell",
>         "Error executing ['podman', 'container', 'exists', 'heat_engine_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=heat_engine_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'ironic_inspector_init_log']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_log', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_log', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'neutron_ovs_bridge']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_ovs_bridge', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_ovs_bridge', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'nova_statedir_owner']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_statedir_owner', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_statedir_owner', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'swift_copy_rings']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_copy_rings', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_copy_rings', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'zaqar_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=zaqar_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=zaqar_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'glance_api_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=glance_api_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'ironic_inspector_init_dnsmasq_dhcp_hostsdir']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_dnsmasq_dhcp_hostsdir', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_init_dnsmasq_dhcp_hostsdir', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'keystone_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'mistral_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mistral_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mistral_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'mysql_sync_credentials']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=mysql', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Error executing ['podman', 'container', 'exists', 'neutron_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=neutron_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'swift_setup_srv']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_setup_srv', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=swift_setup_srv', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'ironic_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'iscsid']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=iscsid', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=iscsid', '--format', '{{.Names}}']\"",
>         "Created symlink /etc/systemd/system/multi-user.target.wants/tripleo_iscsid.service → /etc/systemd/system/tripleo_iscsid.service.",
>         "Created symlink /etc/systemd/system/timers.target.wants/tripleo_iscsid_healthcheck.timer → /etc/systemd/system/tripleo_iscsid_healthcheck.timer.",
>         "Created symlink /etc/systemd/system/tripleo_iscsid.service.requires/tripleo_iscsid_healthcheck.timer → /etc/systemd/system/tripleo_iscsid_healthcheck.timer.",
>         "Error executing ['podman', 'container', 'exists', 'keystone']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=keystone', '--format', '{{.Names}}']\"",
>         "Created symlink /etc/systemd/system/multi-user.target.wants/tripleo_keystone.service → /etc/systemd/system/tripleo_keystone.service.",
>         "Created symlink /etc/systemd/system/timers.target.wants/tripleo_keystone_healthcheck.timer → /etc/systemd/system/tripleo_keystone_healthcheck.timer.",
>         "Created symlink /etc/systemd/system/tripleo_keystone.service.requires/tripleo_keystone_healthcheck.timer → /etc/systemd/system/tripleo_keystone_healthcheck.timer.",
>         "Error executing ['podman', 'container', 'exists', 'ironic_inspector_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'ironic_inspector_get_ipa']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_get_ipa', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=ironic_inspector_get_ipa', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'keystone_bootstrap']: returned 1",
>         "Error executing ['podman', 'container', 'exists', 'nova_api_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_api_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_api_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'placement_api_db_extract_data_from_nova_api']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=placement_api_db_extract_data_from_nova_api', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=placement_api_db_extract_data_from_nova_api', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'nova_api_map_cell0']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_api_map_cell0', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_api_map_cell0', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'placement_api_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=placement_api_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=placement_api_db_sync', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'nova_api_ensure_default_cell']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_api_ensure_default_cell', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_api_ensure_default_cell', '--format', '{{.Names}}']\"",
>         "Error executing ['podman', 'container', 'exists', 'nova_db_sync']: returned 1",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_db_sync', '--filter', 'label=config_id=tripleo_step3', '--format', '{{.Names}}']\" - retrying without config_id",
>         "Did not find container with \"['podman', 'ps', '-a', '--filter', 'label=container_name=nova_db_sync', '--format', '{{.Names}}']\""
>     ]
> }
> 
> TASK [Manage containers for step 3 with tripleo-ansible] ***********************
> Tuesday 13 April 2021  09:23:13 -0400 (0:00:00.068)       0:11:07.001 *********
> skipping: [undercloud-0]
> 
> TASK [Clean container_puppet_tasks for undercloud-0 step 3] ********************
> Tuesday 13 April 2021  09:23:13 -0400 (0:00:00.066)       0:11:07.067 *********
> ok: [undercloud-0]
> ...
Comment 17 errata-xmlrpc 2021-05-26 13:52:43 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.6 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2097