rhel-osp-director: failed to update 7.0->7.2 :ControllerNodesPostDeployment: resources.ControllerOvercloudServicesDeployment_Step7: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6 Environment: openstack-tripleo-heat-templates-0.8.6-94.el7ost.noarch instack-undercloud-2.1.2-36.el7ost.noarch Steps to reproduce: 1. Deploy 7.0 with "openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --neutron-network-type vxlan --neutron-tunnel-types vxlan --ntp-server 10.5.26.10 --timeout 90 -e network-environment.yaml" 2. Update the undercloud node and reboot it - verify nothing reports as failed. 3. Follow the procedure to prepare the overcloud nodes for the update and update the included environment files. 4. Run the update command: "yes "" | openstack overcloud update stack overcloud -i --templates -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/updates/update-from-keystone-admin-internal-api.yaml -e network-environment.yaml Result: The update fails: heat resource-show overcloud ControllerNodesPostDeployment +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Property | Value | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | {} | | description | | | links | http://192.0.2.1:8004/v1/99ac7f5e5f124ce5bd59dbdb456beefd/stacks/overcloud/fc3d8f63-a2fc-48c1-a7ad-d72368695ad4/resources/ControllerNodesPostDeployment (self) | | | http://192.0.2.1:8004/v1/99ac7f5e5f124ce5bd59dbdb456beefd/stacks/overcloud/fc3d8f63-a2fc-48c1-a7ad-d72368695ad4 (stack) | | | http://192.0.2.1:8004/v1/99ac7f5e5f124ce5bd59dbdb456beefd/stacks/overcloud-ControllerNodesPostDeployment-yhuym3luxvfe/a805fec5-39f9-484a-8871-77b2c764d180 (nested) | | logical_resource_id | ControllerNodesPostDeployment | | physical_resource_id | a805fec5-39f9-484a-8871-77b2c764d180 | | required_by | CephStorageNodesPostDeployment | | | BlockStorageNodesPostDeployment | | resource_name | ControllerNodesPostDeployment | | resource_status | UPDATE_FAILED | | resource_status_reason | resources.ControllerNodesPostDeployment: resources.ControllerOvercloudServicesDeployment_Step7: Error: resources[1]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6 | | resource_type | OS::TripleO::ControllerPostDeployment | | updated_time | 2015-12-18T18:53:12Z | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ I see the following errors on a controller: Dec 18 14:06:35 overcloud-controller-2 os-collect-config: instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated\u001b[ 0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy: :Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Concat::Setup]): concat::setup is deprecate d as a public API of the concat module and should no longer be directly included in the manifest.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6002]): The default incoming_chmod set to 0644 may yiel d in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6002]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6001]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a late r release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6001]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1 ;31mWarning: Scope(Swift::Storage::Server[6000]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mW Dec 18 14:06:35 overcloud-controller-2 os-collect-config: arning: Scope(Swift::Storage::Server[6000]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later rel ease.\u001b[0m\n", "deploy_status_code": 0} Dec 18 14:06:35 overcloud-controller-2 os-collect-config: #033[1;31mWarning: Scope(Swift::Storage::Server[6002]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.#033[0m Dec 18 14:06:35 overcloud-controller-2 os-collect-config: #033[1;31mWarning: Scope(Swift::Storage::Server[6002]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.#033[0m Dec 18 14:06:35 overcloud-controller-2 os-collect-config: #033[1;31mWarning: Scope(Swift::Storage::Server[6001]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.#033[0m Dec 18 14:06:35 overcloud-controller-2 os-collect-config: #033[1;31mWarning: Scope(Swift::Storage::Server[6001]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.#033[0m Dec 18 14:06:35 overcloud-controller-2 os-collect-config: #033[1;31mWarning: Scope(Swift::Storage::Server[6000]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.#033[0m Dec 18 14:06:35 overcloud-controller-2 os-collect-config: #033[1;31mWarning: Scope(Swift::Storage::Server[6000]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.#033[0m Dec 18 14:09:22 overcloud-controller-2 os-collect-config: \nWarning: nmcli (1.0.6) and NetworkManager (1.0.0) versions don't match. Use --nocheck to suppress the warning.\nWarning: nmcli (1.0.6) and NetworkManager (1.0.0) versions don't match. Use --nocheck to suppress the warning.\nWarning: nmcli (1.0.6) and NetworkManager (1.0.0) versions don't match. Use --nocheck to suppress the warning.\nWarning: nmcli (1.0.6) and NetworkManager (1.0.0) versions don't match. Use --nocheck to suppress the warning.\nWarning: nmcli (1.0.6) and NetworkManager (1.0.0) versions don't match. Use --nocheck to suppress the warning.\n\u001b[1;31mWarning: Scope(Class[Keystone]): Execution of db_sync does not depend on $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Glance::Registry]): Execution of db_sync does not depend on $manage_service or $enabled anymore. Please use sync_db instead.\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_host'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_protocol'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_port'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Nova::Vncproxy::Common]): Could not look up qualified variable '::nova::compute::vncproxy_path'; class ::nova::compute has not been evaluated\u001b[0m\n\u001b[1;31mWarning: Scope(Class[Concat::Setup]): concat::setup is deprecated as a public API of the concat module and should no longer be directly included in the manifest.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6002]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6002]): The def Dec 18 14:09:22 overcloud-controller-2 os-collect-config: ault outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6001]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6001]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6000]): The default incoming_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n\u001b[1;31mWarning: Scope(Swift::Storage::Server[6000]): The default outgoing_chmod set to 0644 may yield in error prone directories and will be changed in a later release.\u001b[0m\n", "deploy_status_code": 0}
Created attachment 1107398 [details] /var/log/messages from controller
can you run heat deployment-show on the failed deployment? is the environment still around for further debugging?
Sasha, can you reproduce this and provide logs?
If this is reproduced with 7.3, please reopen.
Re-open the Bz as ON_QA and with need_info flag to get more logs. Mike correct me if I'm wrong, even if it does not reproduces when updating to 7.3 - still we need to test and to support update to 7.2?
we need to reproduce and add logs .
Verified: Environment: instack-undercloud-2.1.2-39.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-117.el7ost.noarch Able to update successfully 7.0-> 7.3
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0264.html