Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1691408

Summary: Re-deployment of Undercloud failed on stopping tripleo-heat-engine container service
Product: Red Hat OpenStack Reporter: Artem Hrechanychenko <ahrechan>
Component: python-paunchAssignee: Steve Baker <sbaker>
Status: CLOSED ERRATA QA Contact: Sasha Smolyak <ssmolyak>
Severity: high Docs Contact:
Priority: high    
Version: 15.0 (Stein)CC: augol, dbecker, emacchi, mburns, morazi, sbaker, ssmolyak
Target Milestone: rcKeywords: Triaged
Target Release: 15.0 (Stein)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python-paunch-4.3.1-0.20190411151218.61c448d.el8ost Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-09-21 11:20:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Artem Hrechanychenko 2019-03-21 14:44:50 UTC
Description of problem:

After initial installation of Undercloud with disable undercloud_debug I changed undercloud_debug=True and re-run installation

Failed for me:

TASK [Debug output for task: Start containers for step 4] **********************
fatal: [undercloud-0]: FAILED! => {
    "failed_when_result": true,
    "outputs.stdout_lines | default([]) | union(outputs.stderr_lines | default([]))": [
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_inspector_dnsmasq.service.",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_ironic_inspector_dnsmasq_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_inspector.service.",
        "Warning: Stopping tripleo_ironic_inspector_healthcheck.service, but it can still be activated by:",
        "  tripleo_ironic_inspector_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_ironic_inspector_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_pxe_http.service.",
        "No systemd unit file was found for tripleo_ironic_pxe_http_healthcheck.service",
        "No systemd unit file was found for tripleo_ironic_pxe_http_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_pxe_tftp.service.",
        "Warning: Stopping tripleo_ironic_pxe_tftp_healthcheck.service, but it can still be activated by:",
        "  tripleo_ironic_pxe_tftp_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_ironic_pxe_tftp_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_neutron_agent.service.",
        "No systemd unit file was found for tripleo_ironic_neutron_agent_healthcheck.service",
        "No systemd unit file was found for tripleo_ironic_neutron_agent_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_conductor.service.",
        "Warning: Stopping tripleo_ironic_conductor_healthcheck.service, but it can still be activated by:",
        "  tripleo_ironic_conductor_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_ironic_conductor_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_mistral_api.service.",
        "Warning: Stopping tripleo_mistral_api_healthcheck.service, but it can still be activated by:",
        "  tripleo_mistral_api_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_mistral_api_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_neutron_ovs_agent.service.",
        "Warning: Stopping tripleo_neutron_ovs_agent_healthcheck.service, but it can still be activated by:",
        "  tripleo_neutron_ovs_agent_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_neutron_ovs_agent_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_neutron_l3_agent.service.",
        "Warning: Stopping tripleo_neutron_l3_agent_healthcheck.service, but it can still be activated by:",
        "  tripleo_neutron_l3_agent_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_neutron_l3_agent_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_neutron_dhcp.service.",
        "Warning: Stopping tripleo_neutron_dhcp_healthcheck.service, but it can still be activated by:",
        "  tripleo_neutron_dhcp_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_neutron_dhcp_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_ironic_api.service.",
        "Warning: Stopping tripleo_ironic_api_healthcheck.service, but it can still be activated by:",
        "  tripleo_ironic_api_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_ironic_api_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_proxy.service.",
        "Warning: Stopping tripleo_swift_proxy_healthcheck.service, but it can still be activated by:",
        "  tripleo_swift_proxy_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_swift_proxy_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_nova_metadata.service.",
        "Warning: Stopping tripleo_nova_metadata_healthcheck.service, but it can still be activated by:",
        "  tripleo_nova_metadata_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_nova_metadata_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_nova_api.service.",
        "Warning: Stopping tripleo_nova_api_healthcheck.service, but it can still be activated by:",
        "  tripleo_nova_api_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_nova_api_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_glance_api.service.",
        "Warning: Stopping tripleo_glance_api_healthcheck.service, but it can still be activated by:",
        "  tripleo_glance_api_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_glance_api_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_nova_placement.service.",
        "Warning: Stopping tripleo_nova_placement_healthcheck.service, but it can still be activated by:",
        "  tripleo_nova_placement_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_nova_placement_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_zaqar_websocket.service.",
        "No systemd unit file was found for tripleo_zaqar_websocket_healthcheck.service",
        "No systemd unit file was found for tripleo_zaqar_websocket_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_zaqar.service.",
        "No systemd unit file was found for tripleo_zaqar_healthcheck.service",
        "No systemd unit file was found for tripleo_zaqar_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_rsync.service.",
        "No systemd unit file was found for tripleo_swift_rsync_healthcheck.service",
        "No systemd unit file was found for tripleo_swift_rsync_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_object_updater.service.",
        "No systemd unit file was found for tripleo_swift_object_updater_healthcheck.service",
        "No systemd unit file was found for tripleo_swift_object_updater_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_object_server.service.",
        "Warning: Stopping tripleo_swift_object_server_healthcheck.service, but it can still be activated by:",
        "  tripleo_swift_object_server_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_swift_object_server_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_object_expirer.service.",
        "No systemd unit file was found for tripleo_swift_object_expirer_healthcheck.service",
        "No systemd unit file was found for tripleo_swift_object_expirer_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_container_updater.service.",
        "No systemd unit file was found for tripleo_swift_container_updater_healthcheck.service",
        "No systemd unit file was found for tripleo_swift_container_updater_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_container_server.service.",
        "Warning: Stopping tripleo_swift_container_server_healthcheck.service, but it can still be activated by:",
        "  tripleo_swift_container_server_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_swift_container_server_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_account_server.service.",
        "Warning: Stopping tripleo_swift_account_server_healthcheck.service, but it can still be activated by:",
        "  tripleo_swift_account_server_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_swift_account_server_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_swift_account_reaper.service.",
        "No systemd unit file was found for tripleo_swift_account_reaper_healthcheck.service",
        "No systemd unit file was found for tripleo_swift_account_reaper_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_nova_scheduler.service.",
        "Warning: Stopping tripleo_nova_scheduler_healthcheck.service, but it can still be activated by:",
        "  tripleo_nova_scheduler_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_nova_scheduler_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_nova_conductor.service.",
        "Warning: Stopping tripleo_nova_conductor_healthcheck.service, but it can still be activated by:",
        "  tripleo_nova_conductor_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_nova_conductor_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_nova_api_cron.service.",
        "No systemd unit file was found for tripleo_nova_api_cron_healthcheck.service",
        "No systemd unit file was found for tripleo_nova_api_cron_healthcheck.timer",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_neutron_api.service.",
        "Warning: Stopping tripleo_neutron_api_healthcheck.service, but it can still be activated by:",
        "  tripleo_neutron_api_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_neutron_api_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_mistral_executor.service.",
        "Warning: Stopping tripleo_mistral_executor_healthcheck.service, but it can still be activated by:",
        "  tripleo_mistral_executor_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_mistral_executor_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_mistral_event_engine.service.",
        "Warning: Stopping tripleo_mistral_event_engine_healthcheck.service, but it can still be activated by:",
        "  tripleo_mistral_event_engine_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_mistral_event_engine_healthcheck.timer.",
        "Removed /etc/systemd/system/multi-user.target.wants/tripleo_mistral_engine.service.",
        "Warning: Stopping tripleo_mistral_engine_healthcheck.service, but it can still be activated by:",
        "  tripleo_mistral_engine_healthcheck.timer",
        "Removed /etc/systemd/system/timers.target.wants/tripleo_mistral_engine_healthcheck.timer.",
        "Job for tripleo_heat_engine.service canceled.",
        "systemctl failed",
        "Traceback (most recent call last):",
        "  File \"/usr/lib/python3.6/site-packages/paunch/utils/systemd.py\", line 121, in service_delete",
        "    subprocess.check_call(['systemctl', 'stop', sysd_f])",
        "  File \"/usr/lib64/python3.6/subprocess.py\", line 311, in check_call",
        "    raise CalledProcessError(retcode, cmd)",
        "subprocess.CalledProcessError: Command '['systemctl', 'stop', 'tripleo_heat_engine.service']' returned non-zero exit status 1.",
        "Command '['systemctl', 'stop', 'tripleo_heat_engine.service']' returned non-zero exit status 1."

Version-Release number of selected component (if applicable):
OSP15 compose - RHOS_TRUNK-15.0-RHEL-8-20190320.n.1
openstack-tripleo-heat-templates-10.3.1-0.20190318140159.cbe8724.el8ost.noarch
openstack-heat-api-11.1.0-0.20190318122852.acce80a.el8ost.noarch
python-openstackclient-lang-3.18.0-0.20190312140834.6868499.el8ost.noarch
openstack-tripleo-common-containers-10.6.1-0.20190320113112.01b56d0.el8ost.noarch
openstack-tripleo-image-elements-10.3.1-0.20190319120806.1bde610.el8ost.noarch
puppet-openstack_extras-14.3.0-0.20190319213453.b197188.el8ost.noarch
puppet-openstacklib-14.3.1-0.20190320033446.0bce7c1.el8ost.noarch
python3-openstackclient-3.18.0-0.20190312140834.6868499.el8ost.noarch
openstack-heat-common-11.1.0-0.20190318122852.acce80a.el8ost.noarch
openstack-heat-monolith-11.1.0-0.20190318122852.acce80a.el8ost.noarch
openstack-tripleo-validations-10.2.1-0.20190218150113.e6490b3.el8ost.noarch
openstack-tripleo-common-10.6.1-0.20190320113112.01b56d0.el8ost.noarch
python3-openstacksdk-0.26.0-0.20190312204647.feae8da.el8ost.noarch
openstack-selinux-0.8.18-0.20190312135925.c0dd7b6.el8ost.noarch
openstack-tripleo-puppet-elements-10.2.1-0.20190319120806.7903181.el8ost.noarch
openstack-tripleo-heat-templates-10.3.1-0.20190318140159.cbe8724.el8ost.noarch
openstack-heat-engine-11.1.0-0.20190318122852.acce80a.el8ost.noarch
openstack-heat-agents-1.7.1-0.20190212102146.6139799.el8ost.noarch
python3-tripleoclient-heat-installer-11.3.1-0.20190319125100.23e610c.el8ost.noarch
python3-tripleoclient-11.3.1-0.20190319125100.23e610c.el8ost.noarch
python3-tripleo-common-10.6.1-0.20190320113112.01b56d0.el8ost.noarch

How reproducible:
Always

Steps to Reproduce:
1.Deploy Undercloud
2.Change undercloud.conf
3.Re-run undercloud installation

Actual results:
failed on step Start containers for step 4

Expected results:
re-installation passed

Additional info:

Comment 7 Sasha Smolyak 2019-07-21 13:28:15 UTC
Redeployed with undercloud_debug=true, deployment succeeded. Verified

Comment 11 errata-xmlrpc 2019-09-21 11:20:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:2811