Bug 1371628
| Summary: | osp-director-9: upgrade from 8.0-> 9.0 fails in [overcloud-ControllerNodesPostDeployment-vfo7hwxgjojk-ControllerOvercloudServicesDeployment_Step4-yv2d6a5ghub3]: CREATE_FAILED | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | rlopez |
| Component: | rhosp-director | Assignee: | Angus Thomas <athomas> |
| Status: | CLOSED DUPLICATE | QA Contact: | Omri Hochman <ohochman> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 9.0 (Mitaka) | CC: | dbecker, jstransk, mburns, mcornea, morazi, rhel-osp-director-maint |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-08-31 16:21:28 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
Adding more info:
$ heat event-list overcloud-ControllerNodesPostDeployment-vfo7hwxgjojk | tac | head -n2
WARNING (shell) "heat event-list" is deprecated, please use "openstack stack event list" instead
+------------------------------------------------------+--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+
| overcloud-ControllerNodesPostDeployment-vfo7hwxgjojk | 46a02f8f-33eb-474c-ad75-ece5b2e0c375 | Resource CREATE failed: Error: resources.ControllerOvercloudServicesDeployment_Step4.resources[1]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6 | CREATE_FAILED | 2016-08-30T03:48:45 |
$ heat stack-show overcloud-ControllerNodesPostDeployment-vfo7hwxgjojk | grep NodeConfigIdentifiers
WARNING (shell) "heat stack-show" is deprecated, please use "openstack stack show" instead
| | "NodeConfigIdentifiers": "{u'deployment_identifier': 1472527744, u'controller_config': {u'1': u'os-apply-config deployment 86fb2737-1dcf-43b1-9421-1a3bf46422bc completed,Root CA cert injection not enabled.,TLS not enabled.,None,', u'0': u'os-apply-config deployment 1ec1e166-073f-4228-997a-64a481c422e0 completed,Root CA cert injection not enabled.,TLS not enabled.,None,', u'2': u'os-apply-config deployment 575d145a-a9e1-4fca-8401-e58bc13e5b05 completed,Root CA cert injection not enabled.,TLS not enabled.,None,'}, u'allnodes_extra': u'none'}" |
From what I can tell, I think the following heat-config-files: vi /var/lib/heat-config/heat-config-script/28ab0b90-07e0-4794-8682-123e055db477 vi /var/lib/heat-config/heat-config-script/28ab0b90-07e0-4794-8682-123e055db477 Lines: 182 and 186, Require the same patch as major_upgrade_pacemaker_migrations.sh: https://code.engineering.redhat.com/gerrit/#/c/82472/1/extraconfig/tasks/major_upgrade_pacemaker_migrations.sh The key info is in the fpaste that Roger posted above Stage[main]/Cinder::Setup_test_volume/Exec[pvcreate /dev/loop2]: pvcreate /dev/loop2 returned 5 instead of one of [0] which suggests that the root cause is bug 1356683. *** This bug has been marked as a duplicate of bug 1356683 *** I'm reposting the fpaste as the previous one is only good for 1 month, and this one is good forever. https://paste.fedoraproject.org/418449/63081147/ |
During the OSP8->OSP9 upgrade, the upgrade fails during the converge process. Environment: ------------------------------------ $ rpm -qa | grep openstack* openstack-zaqar-2.0.1-0.20160621211345.9fdbcfc.el7ost.noarch openstack-cinder-8.0.0-5.el7ost.noarch python-openstackclient-2.2.0-1.el7ost.noarch openstack-aodh-notifier-2.0.3-2.el7ost.noarch openstack-neutron-common-8.1.2-1.el7ost.noarch openstack-ceilometer-notification-6.1.3-2.el7ost.noarch openstack-tripleo-heat-templates-2.0.0-30.el7ost.noarch openstack-heat-api-6.0.0-8.el7ost.noarch openstack-nova-api-13.1.0-6.el7ost.noarch openstack-neutron-8.1.2-1.el7ost.noarch openstack-selinux-0.7.3-3.el7ost.noarch openstack-nova-network-13.1.0-6.el7ost.noarch openstack-nova-cells-13.1.0-6.el7ost.noarch openstack-sahara-api-4.0.1-2.el7ost.noarch openstack-tripleo-image-elements-0.9.9-6.el7ost.noarch openstack-aodh-common-2.0.3-2.el7ost.noarch openstack-nova-common-13.1.0-6.el7ost.noarch openstack-tripleo-common-2.0.0-8.el7ost.noarch openstack-ceilometer-collector-6.1.3-2.el7ost.noarch openstack-ironic-conductor-5.1.2-3.el7ost.noarch openstack-sahara-common-4.0.1-2.el7ost.noarch openstack-tripleo-heat-templates-kilo-0.8.14-13.el7ost.noarch openstack-nova-13.1.0-6.el7ost.noarch openstack-aodh-expirer-2.0.3-2.el7ost.noarch openstack-ironic-common-5.1.2-3.el7ost.noarch openstack-tripleo-puppet-elements-2.0.0-4.el7ost.noarch openstack-swift-plugin-swift3-1.10-1.el7ost.noarch openstack-swift-account-2.7.0-2.el7ost.noarch openstack-glance-12.0.0-1.el7ost.noarch openstack-heat-api-cloudwatch-6.0.0-8.el7ost.noarch openstack-ceilometer-api-6.1.3-2.el7ost.noarch openstack-nova-scheduler-13.1.0-6.el7ost.noarch openstack-neutron-ml2-8.1.2-1.el7ost.noarch openstack-aodh-compat-2.0.3-2.el7ost.noarch openstack-sahara-4.0.1-2.el7ost.noarch openstack-nova-console-13.1.0-6.el7ost.noarch openstack-tripleo-0.0.8-0.2.d81bd6dgit.el7ost.noarch python-openstacksdk-0.8.3-1.el7ost.noarch openstack-aodh-api-2.0.3-2.el7ost.noarch openstack-ceilometer-common-6.1.3-2.el7ost.noarch openstack-heat-common-6.0.0-8.el7ost.noarch openstack-tripleo-heat-templates-liberty-2.0.0-30.el7ost.noarch openstack-heat-templates-0-0.3.96a0b0bgit.el7ost.noarch openstack-swift-container-2.7.0-2.el7ost.noarch openstack-swift-object-2.7.0-2.el7ost.noarch openstack-heat-api-cfn-6.0.0-8.el7ost.noarch openstack-nova-conductor-13.1.0-6.el7ost.noarch openstack-nova-compute-13.1.0-6.el7ost.noarch openstack-neutron-openvswitch-8.1.2-1.el7ost.noarch openstack-ironic-inspector-3.2.2-4.el7ost.noarch openstack-sahara-engine-4.0.1-2.el7ost.noarch openstack-nova-novncproxy-13.1.0-6.el7ost.noarch openstack-tempest-10.0.0-2.b4a056dgit.el7ost.noarch openstack-aodh-evaluator-2.0.3-2.el7ost.noarch openstack-ceilometer-central-6.1.3-2.el7ost.noarch openstack-ironic-api-5.1.2-3.el7ost.noarch openstack-aodh-listener-2.0.3-2.el7ost.noarch openstack-ceilometer-polling-6.1.3-2.el7ost.noarch openstack-utils-2015.2-1.el7ost.noarch openstack-swift-2.7.0-2.el7ost.noarch openstack-swift-proxy-2.7.0-2.el7ost.noarch openstack-heat-engine-6.0.0-8.el7ost.noarch openstack-nova-cert-13.1.0-6.el7ost.noarch openstack-keystone-9.0.2-1.el7ost.noarch openstack-puppet-modules-8.1.7-2.el7ost.noarch Description: ------------------------------------ Upgrade of Baremetal machines, 3 controllers, 3 compute, 3 ceph nodes. Original Deployment Command: ------------------------------------ openstack overcloud deploy --templates ~/templates-osp9 -e ~/templates-osp9/environments/network-isolation.yaml -e ~/templates-osp9/network-environment.yaml -e ~/templates-osp9/environments/storage-environment.yaml -e ~/templates-osp9/environments/major-upgrade-pacemaker-converge.yaml --control-scale 3 --compute-scale 3 --swift-storage-scale 0 --block-storage-scale 0 --ceph-storage-scale 3 --control-flavor control --compute-flavor compute --ceph-storage-flavor ceph-storage --ntp-server 10.16.255.2 --neutron-network-type vxlan --neutron-tunnel-types vxlan $ heat deployment-list | grep -vi complete WARNING (shell) "heat deployment-list" is deprecated, please use "openstack software deployment list" instead +--------------------------------------+--------------------------------------+--------------------------------------+--------+----------+---------------------+---------------------------------------------------------------------+ | id | config_id | server_id | action | status | creation_time | status_reason | +--------------------------------------+--------------------------------------+--------------------------------------+--------+----------+---------------------+---------------------------------------------------------------------+ | 61c94b73-e609-45d0-b347-5d1b42039784 | 041def84-106d-4f3a-9e88-3fe970a0d9ed | 2124db7a-67ce-4b9a-bfdb-2d3fedf8c745 | CREATE | FAILED | 2016-08-30T03:44:03 | deploy_status_code : Deployment exited with non-zero status code: 6 | | 73e60c35-3fef-4ecc-a062-fdebdee49b54 | 34c7f1ad-1ea6-4d04-9048-47ce6a4e351c | f1a1e615-c4f5-4429-a556-c97801acb26a | CREATE | FAILED | 2016-08-30T03:44:05 | deploy_status_code : Deployment exited with non-zero status code: 6 | +--------------------------------------+--------------------------------------+--------------------------------------+--------+----------+---------------------+---------------------------------------------------------------------+ $ heat deployment-show 61c94b73-e609-45d0-b347-5d1b42039784 Output Abbreviated: "deploy_status_code": 6 }, "creation_time": "2016-08-30T03:44:03", "updated_time": "2016-08-30T03:47:31", "input_values": { "step": 3, "update_identifier": { "deployment_identifier": 1472527744, "controller_config": { "1": "os-apply-config deployment 86fb2737-1dcf-43b1-9421-1a3bf46422bc completed,Root CA cert injection not enabled.,TLS not enabled.,None,", "0": "os-apply-config deployment 1ec1e166-073f-4228-997a-64a481c422e0 completed,Root CA cert injection not enabled.,TLS not enabled.,None,", "2": "os-apply-config deployment 575d145a-a9e1-4fca-8401-e58bc13e5b05 completed,Root CA cert injection not enabled.,TLS not enabled.,None," }, "allnodes_extra": "none" } }, "action": "CREATE", "status_reason": "deploy_status_code : Deployment exited with non-zero status code: 6", "id": "61c94b73-e609-45d0-b347-5d1b42039784" } [s FULL OUTPUT: https://paste.fedoraproject.org/417393/25720491/ [root@overcloud-controller-0 ~]# journalctl -u os-collect-config | grep ERROR Aug 29 16:51:39 overcloud-controller-0.localdomain os-collect-config[47027]: [2016-08-29 16:51:39,825] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/dea61713-4202-43c0-ba83-63531d1a8f6b. [1] Aug 29 17:51:02 overcloud-controller-0.localdomain os-collect-config[47027]: [2016-08-29 17:51:02,524] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/28ab0b90-07e0-4794-8682-123e055db477. [1] Aug 29 20:34:31 overcloud-controller-0.localdomain dracut[45643]: -rw-r--r-- 1 root root 191 Sep 29 2014 usr/lib/kbd/consolefonts/ERRORS Aug 29 20:35:44 overcloud-controller-0.localdomain dracut[10185]: -rw-r--r-- 1 root root 191 Sep 29 2014 usr/lib/kbd/consolefonts/ERRORS Aug 29 20:39:33 overcloud-controller-0.localdomain dracut[35875]: -rw-r--r-- 1 root root 191 Sep 29 2014 usr/lib/kbd/consolefonts/ERRORS Script /var/lib/heat-config/heat-config-script/dea61713-4202-43c0-ba83-63531d1a8f6b: https://paste.fedoraproject.org/417410/57302914/ Script /var/lib/heat-config/heat-config-script/28ab0b90-07e0-4794-8682-123e055db477: https://paste.fedoraproject.org/417412/14725731/ Last 100 lines of /var/log/messsages on overcloud-controller-0: ------------------------------------ # tail -n100 /var//log/messages Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:1 (then) Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: error: Internal error: No action found for demote in redis:2 (then) Aug 30 16:07:17 overcloud-controller-0 crmd[10350]: notice: Transition 78 (Complete=0, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-335.bz2): Complete Aug 30 16:07:17 overcloud-controller-0 pengine[10349]: notice: Calculated Transition 78: /var/lib/pacemaker/pengine/pe-input-335.bz2 Aug 30 16:07:17 overcloud-controller-0 crmd[10350]: notice: State transition S_TRANSITION_ENGINE -> S_IDLE [ input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd ] Aug 30 16:07:36 overcloud-controller-0 neutron-server: 2016-08-30 16:07:36.753 22440 WARNING neutron.db.agents_db [req-4a1d94cd-e3b7-4853-a225-164f5f1a77ea - - - - -] Agent healthcheck: found 1 dead agents out of 16: Aug 30 16:07:36 overcloud-controller-0 neutron-server: Type Last heartbeat host Aug 30 16:07:36 overcloud-controller-0 neutron-server: Open vSwitch agent 2016-07-15 17:31:56 overcloud-compute-5.localdomain Aug 30 16:07:38 overcloud-controller-0 os-collect-config: /var/lib/os-collect-config/local-data not found. Skipping Aug 30 16:07:38 overcloud-controller-0 os-collect-config: No local metadata found (['/var/lib/os-collect-config/local-data']) Aug 30 16:08:09 overcloud-controller-0 os-collect-config: /var/lib/os-collect-config/local-data not found. Skipping Aug 30 16:08:09 overcloud-controller-0 os-collect-config: No local metadata found (['/var/lib/os-collect-config/local-data']) Aug 30 16:08:13 overcloud-controller-0 neutron-server: 2016-08-30 16:08:13.758 22440 WARNING neutron.db.agents_db [req-4a1d94cd-e3b7-4853-a225-164f5f1a77ea - - - - -] Agent healthcheck: found 1 dead agents out of 16: Aug 30 16:08:13 overcloud-controller-0 neutron-server: Type Last heartbeat host Aug 30 16:08:13 overcloud-controller-0 neutron-server: Open vSwitch agent 2016-07-15 17:31:56 overcloud-compute-5.localdomain Aug 30 16:08:39 overcloud-controller-0 os-collect-config: /var/lib/os-collect-config/local-data not found. Skipping Aug 30 16:08:39 overcloud-controller-0 os-collect-config: No local metadata found (['/var/lib/os-collect-config/local-data']) Aug 30 16:08:50 overcloud-controller-0 neutron-server: 2016-08-30 16:08:50.752 22440 WARNING neutron.db.agents_db [req-4a1d94cd-e3b7-4853-a225-164f5f1a77ea - - - - -] Agent healthcheck: found 1 dead agents out of 16: Aug 30 16:08:50 overcloud-controller-0 neutron-server: Type Last heartbeat host Aug 30 16:08:50 overcloud-controller-0 neutron-server: Open vSwitch agent 2016-07-15 17:31:56 overcloud-compute-5.localdomain Aug 30 16:08:58 overcloud-controller-0 NetworkManager[1232]: <info> platform-linux: Too many netlink events. Need to resynchronize platform cache Aug 30 16:09:10 overcloud-controller-0 os-collect-config: /var/lib/os-collect-config/local-data not found. Skipping Aug 30 16:09:10 overcloud-controller-0 os-collect-config: No local metadata found (['/var/lib/os-collect-config/local-data']) Last 100 lines of heat-engine.log on undercloud: ------------------------------------ ]$ sudo tail -n100 /var/log/heat/heat-engine.log 2016-08-30 11:56:16.329 17052 INFO heat.engine.service [req-c195410b-4bd0-420e-8163-e68e90773656 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 11:56:16.330 17054 INFO heat.engine.service [req-a89c3a94-d0e5-4f94-b9df-91722182fa33 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 11:56:16.348 17050 INFO heat.engine.service [req-cf013883-8344-4d14-ba52-e13dc7cd55cc - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 11:56:16.352 17051 INFO heat.engine.service [req-59261c05-f394-401e-9437-1c473a55d2fb - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 11:56:16.701 17053 INFO heat.engine.service [req-6a865422-369a-4742-a670-3fb4ca02e167 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 11:56:17.731 17055 INFO heat.engine.service [req-3d59f533-65ef-47ce-b291-d4745190811c - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 11:57:16.328 17054 INFO heat.engine.service [req-085176ad-abe4-416f-8bd1-598a78e09bbd - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 11:57:16.330 17052 INFO heat.engine.service [req-ba91ec0b-25f6-4f73-9860-1261fc82f391 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 11:57:16.349 17050 INFO heat.engine.service [req-85c35acc-6b03-4b57-b513-a7afefc3de4a - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 11:57:16.352 17051 INFO heat.engine.service [req-82b8d946-0ab2-4f86-87c1-d58f2a335507 - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 11:57:16.700 17053 INFO heat.engine.service [req-9f24c8ab-3084-45ab-8f35-450cd4853a22 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 11:57:17.731 17055 INFO heat.engine.service [req-c885b551-9783-4d83-8b7e-3fe07eb00592 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 11:57:25.989 17052 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: 3932a310b93e4810856c8ac095c212cf reply to reply_0cac85c7611148de934b8fd60ebbee67 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201 2016-08-30 11:57:27.003 17052 DEBUG oslo_messaging._drivers.amqpdriver [req-97c8800b-2a65-4166-a6c4-5f511dc6e15b - admin - default default] sending reply msg_id: 3932a310b93e4810856c8ac095c212cf reply queue: reply_0cac85c7611148de934b8fd60ebbee67 time elapsed: 1.01072357618s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:74 2016-08-30 11:57:39.033 17053 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: de4e3c48b15541f4a055a9f64d09215b reply to reply_09b1663337e749c895190fd80417cc59 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201 2016-08-30 11:57:39.057 17053 DEBUG oslo_messaging._drivers.amqpdriver [req-b634847b-791f-4fa0-b591-fdc205595540 - admin - default default] sending reply msg_id: de4e3c48b15541f4a055a9f64d09215b reply queue: reply_09b1663337e749c895190fd80417cc59 time elapsed: 0.0202105680946s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:74 2016-08-30 11:58:16.330 17054 INFO heat.engine.service [req-a35d2787-095d-4ac6-809d-dfe5950a23c5 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 11:58:16.333 17052 INFO heat.engine.service [req-76496db8-fe15-4675-887d-5a59628c4a6d - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 11:58:16.358 17050 INFO heat.engine.service [req-ba95e02b-690a-4ae3-b095-af35df0c2013 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 11:58:16.361 17051 INFO heat.engine.service [req-abe0082f-d2fb-4dd3-8c89-e36bbd79481e - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 11:58:17.173 17053 INFO heat.engine.service [req-c401b073-75fd-4ae4-9c6f-8cc2adadefd7 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 11:58:17.731 17055 INFO heat.engine.service [req-466011e0-ea7a-4ff0-9350-c9b932d5625a - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 11:58:31.498 17050 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: ef05bcc67b6a4032aead60dbc3773108 reply to reply_0c2aa913bb454a188cd525bd5e43ea03 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201 2016-08-30 11:58:31.683 17050 DEBUG oslo_messaging._drivers.amqpdriver [req-b5060d42-42bc-4149-b422-1853d780f9be - admin - default default] sending reply msg_id: ef05bcc67b6a4032aead60dbc3773108 reply queue: reply_0c2aa913bb454a188cd525bd5e43ea03 time elapsed: 0.181179011008s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:74 2016-08-30 11:59:06.498 17051 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: 90a724fc652d4afd95e985d6fc0980c5 reply to reply_0c2aa913bb454a188cd525bd5e43ea03 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201 2016-08-30 11:59:07.614 17051 DEBUG oslo_messaging._drivers.amqpdriver [req-fd8059ac-67b2-43c4-88a4-bc055961fba0 - admin - default default] sending reply msg_id: 90a724fc652d4afd95e985d6fc0980c5 reply queue: reply_0c2aa913bb454a188cd525bd5e43ea03 time elapsed: 1.11223191s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:74 2016-08-30 11:59:16.330 17054 INFO heat.engine.service [req-834af45b-2c26-4d56-83f1-7f9459922d21 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 11:59:16.334 17052 INFO heat.engine.service [req-fd046d4d-0369-49af-8c38-062ecbb2bc98 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 11:59:16.350 17051 INFO heat.engine.service [req-1c62425d-05e9-454c-b157-c5ef2885b54e - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 11:59:16.365 17050 INFO heat.engine.service [req-c5cd9cba-37da-4f6c-9443-ad7169d69814 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 11:59:16.702 17053 INFO heat.engine.service [req-9a4fc87f-0438-4d28-81b3-c68da6764cd4 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 11:59:17.731 17055 INFO heat.engine.service [req-82b0415e-3ece-4b70-9e9f-21fdf795241c - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 11:59:48.934 17054 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: ec4e02a1e818453d9bc5755f0ce637d5 reply to reply_0cac85c7611148de934b8fd60ebbee67 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201 2016-08-30 11:59:48.963 17054 DEBUG oslo_messaging._drivers.amqpdriver [req-4d7f91df-3816-4235-82d3-9f1cfb8de486 - admin - default default] sending reply msg_id: ec4e02a1e818453d9bc5755f0ce637d5 reply queue: reply_0cac85c7611148de934b8fd60ebbee67 time elapsed: 0.0249494090676s _send_reply /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:74 2016-08-30 12:00:16.334 17054 INFO heat.engine.service [req-025b652c-0064-4342-a83b-95c6e8c296b9 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:00:16.339 17052 INFO heat.engine.service [req-5b932a9c-ee12-4123-ac62-5e4f63dea597 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:00:16.346 17050 INFO heat.engine.service [req-4f65d41d-2a44-4281-9b35-9c55d64b42fd - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:00:16.349 17051 INFO heat.engine.service [req-ed20a8d6-6e5c-425d-89f9-0825beb9e227 - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:00:16.702 17053 INFO heat.engine.service [req-a673b13b-c4e1-4780-b797-a2a58a873ea1 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:00:17.730 17055 INFO heat.engine.service [req-fff8bb58-6a26-4b9a-b30e-cff5fd414f76 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:01:16.332 17052 INFO heat.engine.service [req-12a213cb-bb5f-4fb9-b52e-e7a9c9ab0d49 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:01:16.335 17054 INFO heat.engine.service [req-c72d9681-b6f5-45bc-bf0b-29424528071f - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:01:16.346 17050 INFO heat.engine.service [req-fe69e82b-f8ff-4e96-9457-2920a508a750 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:01:16.350 17051 INFO heat.engine.service [req-fc9a4a19-05bf-42e4-b054-778b8aac7cdf - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:01:16.701 17053 INFO heat.engine.service [req-5ec2c9ee-d272-4965-950a-65bd30388892 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:01:17.731 17055 INFO heat.engine.service [req-9d7633cb-2012-4acc-ae63-534bbaab2992 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:02:16.336 17052 INFO heat.engine.service [req-dddd51e0-1f89-4629-bf77-baeb1d4c16f7 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:02:16.338 17054 INFO heat.engine.service [req-8de9270a-41a9-4c56-ba3e-2593866aea8d - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:02:16.347 17050 INFO heat.engine.service [req-f87d2003-ae2a-4ebe-b00d-aeb300127925 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:02:16.349 17051 INFO heat.engine.service [req-c75c207d-ac09-496e-8965-1656601669a0 - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:02:16.705 17053 INFO heat.engine.service [req-969cad9f-9e1f-4668-abb8-d3cda507a5d9 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:02:17.731 17055 INFO heat.engine.service [req-8e6ce071-b73e-4dca-8aeb-82ba311e0479 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:03:16.337 17052 INFO heat.engine.service [req-168f8445-3222-47d2-8f44-be4bc1f40cc3 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:03:16.338 17054 INFO heat.engine.service [req-38ca9f00-d215-4313-a883-0d4a6619581d - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:03:16.351 17051 INFO heat.engine.service [req-aad77b76-0437-46c8-ba0d-bac6514c80ae - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:03:16.362 17050 INFO heat.engine.service [req-ece9b63c-a5bd-4f24-b5b5-64090ead0388 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:03:16.704 17053 INFO heat.engine.service [req-23f167ac-f7fc-44e5-abde-a1fd8ae26892 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:03:17.734 17055 INFO heat.engine.service [req-6cb6fda6-9776-4883-b84e-76bc212ef931 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:04:16.340 17054 INFO heat.engine.service [req-bfc9e79e-3211-4302-84ee-eafc5c5dc5d2 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:04:16.342 17052 INFO heat.engine.service [req-ad2e9bb7-f785-48ce-9da2-a1db49896ab1 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:04:16.345 17050 INFO heat.engine.service [req-c819c2e2-8019-46d0-9d85-bbf2d9d4eff3 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:04:16.354 17051 INFO heat.engine.service [req-6fbce3f9-9d83-4884-9084-781789b943a0 - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:04:16.702 17053 INFO heat.engine.service [req-c88dc083-8e9c-4b74-adbd-453c589539c5 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:04:17.734 17055 INFO heat.engine.service [req-2bdef289-532d-4d13-85b8-dd66fbe1e1b8 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:05:16.333 17052 INFO heat.engine.service [req-000283a6-8ec5-47ed-ad3f-e2f2e24b1b79 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:05:16.340 17054 INFO heat.engine.service [req-de3274e0-a711-4832-973f-c1f003919493 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:05:16.345 17050 INFO heat.engine.service [req-61fcc07c-425f-4c50-9201-28b362a6bb75 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:05:16.353 17051 INFO heat.engine.service [req-31611ce8-660f-496e-b12e-bd8650ab0d5d - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:05:16.702 17053 INFO heat.engine.service [req-e8fbd4e7-60bd-4358-92c1-b60b91baa3c2 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:05:17.733 17055 INFO heat.engine.service [req-983025fd-3fd8-4ef8-bfbf-ad68e55bdc0d - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:06:16.333 17052 INFO heat.engine.service [req-630807b5-5eee-43e8-b785-cedd23a06067 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:06:16.339 17054 INFO heat.engine.service [req-a7f7643b-bcc1-4045-ba85-ecbb40914bd2 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:06:16.344 17050 INFO heat.engine.service [req-88ba45d5-5435-4e68-a54d-58274f256877 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:06:16.352 17051 INFO heat.engine.service [req-8d344f8f-470e-4ee7-a9a0-6d71adeb5b7f - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:06:16.702 17053 INFO heat.engine.service [req-9bb24164-ebbf-44e4-a2e2-b6e2948eadb1 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:06:17.732 17055 INFO heat.engine.service [req-ba9950c9-2310-4b38-ac76-b4beed76842c - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:07:16.332 17052 INFO heat.engine.service [req-9a42d1c7-3fa6-4ede-9a6d-6178ccc67d70 - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:07:16.336 17054 INFO heat.engine.service [req-3b171df1-6e07-44b6-98d2-e983af829ca3 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:07:16.347 17050 INFO heat.engine.service [req-87bd3330-195e-4bcb-a56c-01c467af8193 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:07:16.354 17051 INFO heat.engine.service [req-bdcd8e51-a05f-48a9-8eb5-0b2ba748d6c5 - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:07:16.701 17053 INFO heat.engine.service [req-298de26c-a4e1-4318-b9b6-baba37c837f4 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:07:17.731 17055 INFO heat.engine.service [req-e849b49f-a77f-4eab-b9b4-2cab98497ad3 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:08:16.332 17052 INFO heat.engine.service [req-bc941039-b10b-4bd6-9933-fb5ba74c4aed - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:08:16.337 17054 INFO heat.engine.service [req-d4fc9a99-6772-4445-a274-57b2867204fc - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:08:16.349 17050 INFO heat.engine.service [req-6ea076a7-37e3-4c7c-a765-1cd8656b543d - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:08:16.409 17051 INFO heat.engine.service [req-ec7557f1-ff0f-48a5-88e1-0fe998d894cc - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:08:16.702 17053 INFO heat.engine.service [req-613a74cd-3df6-47a1-bf08-fa00c7233103 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:08:17.731 17055 INFO heat.engine.service [req-05590939-91c7-4304-809e-1b6445f5e754 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:09:16.331 17052 INFO heat.engine.service [req-35a1174c-8f62-4b41-9963-513db860bc1f - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:09:16.337 17054 INFO heat.engine.service [req-23579c5f-8b07-456d-a0f6-6e470697495d - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:09:16.348 17050 INFO heat.engine.service [req-0572e6f5-feea-4dc5-8b39-e97aa2ecca43 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:09:16.349 17051 INFO heat.engine.service [req-8ccc54ad-c308-4219-8959-1e54248661ce - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:09:16.703 17053 INFO heat.engine.service [req-8772f4cb-87f2-45d7-b67d-efb3919edec4 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:09:17.731 17055 INFO heat.engine.service [req-fa583819-91b4-44d8-94b2-941c5b670168 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated 2016-08-30 12:10:16.339 17052 INFO heat.engine.service [req-75187936-f18c-4317-abad-cb497e29d3ee - - - - -] Service 0436e2c4-fca7-4ea3-9c87-f8fc02a28a2d is updated 2016-08-30 12:10:16.339 17054 INFO heat.engine.service [req-1676f2d5-a627-417e-b4ac-fb4bb587e301 - - - - -] Service b6954be5-41d4-47ff-b40a-50dff7633778 is updated 2016-08-30 12:10:16.349 17050 INFO heat.engine.service [req-a44d68c9-f8a9-4102-92a2-a9adbd02f861 - - - - -] Service 93c116ea-d137-409f-a7ac-5a7832d094db is updated 2016-08-30 12:10:16.396 17051 INFO heat.engine.service [req-5ffb4619-fd30-45d1-9e48-7a795431df58 - - - - -] Service 3a71c08f-7ee4-4d37-8770-5b65706fed7d is updated 2016-08-30 12:10:16.703 17053 INFO heat.engine.service [req-2b52f07b-189f-4446-8576-e137dc120018 - - - - -] Service e68ed64f-8541-41c0-9fa5-7e7b8b0bc1f8 is updated 2016-08-30 12:10:17.732 17055 INFO heat.engine.service [req-fd125ed2-ab79-4278-b1c6-0b2089ccff94 - - - - -] Service e51c4a15-cc49-44c1-82b9-d657fb168f66 is updated Things to note: 1) Doing a pcs status on my controller0 shows now all my services are unmanaged. 2) pcs property shows maintenance mode now set to true.