[rhel-osp-director][upgrade to 8.0] Failed to recover from failed overcloud upgrade process, caused by 'undercloud power-outage' during upgrade. Environment (latest ospd8.0 from 4/7/16 ): -------------------------- instack-0.0.8-2.el7ost.noarch instack-undercloud-2.2.7-4.el7ost.noarch openstack-tripleo-heat-templates-0.8.14-7.el7ost.noarch openstack-heat-api-5.0.1-5.el7ost.noarch heat-cfntools-1.2.8-2.el7.noarch python-heatclient-1.0.0-1.el7ost.noarch openstack-heat-engine-5.0.1-5.el7ost.noarch openstack-heat-api-cloudwatch-5.0.1-5.el7ost.noarch openstack-heat-templates-0-0.8.20150605git.el7ost.noarch openstack-tripleo-heat-templates-kilo-0.8.14-7.el7ost.noarch openstack-heat-common-5.0.1-5.el7ost.noarch openstack-heat-api-cfn-5.0.1-5.el7ost.noarch Step: ----- (1) Deploy rhel 7.3GA using director (2) Perform upgrade to ops-d8.0 (openstack overcloud deploy --templates ..) (3) Power down the Undercloud during overcloud Upgrade (4) Power up the Undercloud after few minutes (5) Attempt to resume the upgrade (by re-run the last upgrade command) (stack should: switch from Upgrade_FAILED to Upgrade_COMPLETE) Results: --------- When the undercloud returns from shutdown, the Stack status is Upgrade_FAILED, re-running the last upgrade Command should resume the upgrade process and end with :Upgrade_COMPLETE. But the upgrade command keep failing time after time and the status remains Upgrade_FAILED, Deployment view: ----------------- [stack@instack ~]$ openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/storage-environment.yaml -e /home/stack/ssl-heat-templates/environments/enable-tls.yaml -e /home/stack/ssl-heat-templates/environments/inject-trust-anchor.yaml -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-init.yaml -e /home/stack/rhos-release-8.yaml .. .. ,, 2016-04-11 21:16:04 [overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m]: UPDATE_FAILED Exception: resources.NetIpMap: Resource update already requested 2016-04-11 21:16:04 [NodeExtraConfig]: UPDATE_COMPLETE state changed 2016-04-11 21:16:05 [1]: UPDATE_FAILED resources[1]: Exception: resources.NetIpMap: Resource update already requested 2016-04-11 21:16:06 [overcloud-Controller-xsdpzedowemp]: UPDATE_FAILED resources[0]: Exception: resources.ExternalPort: Resource update already requested 2016-04-11 21:16:08 [Controller]: UPDATE_FAILED resources.Controller: resources[0]: Exception: resources.ExternalPort: Resource update already requested 2016-04-11 21:16:14 [overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd]: UPDATE_COMPLETE Stack UPDATE completed successfully 2016-04-11 21:16:15 [0]: UPDATE_COMPLETE state changed 2016-04-11 21:16:17 [overcloud-Compute-qon76f3lrn4z]: UPDATE_COMPLETE Stack UPDATE completed successfully 2016-04-11 21:16:18 [Compute]: UPDATE_COMPLETE state changed 2016-04-11 21:16:19 [overcloud]: UPDATE_FAILED resources.Controller: resources[0]: Exception: resources.ExternalPort: Resource update already requested Stack overcloud UPDATE_FAILED Heat Stack update failed. [stack@instack ~]$ heat resource-list -n5 overcloud | grep FAILED | Controller | 403694b7-b282-4683-b0df-c65c9f8fa0d1 | OS::Heat::ResourceGroup | UPDATE_FAILED | 2016-04-11T21:14:23 | overcloud | | 0 | dbd43243-7b05-4648-9f04-69d815f7ff34 | OS::TripleO::Controller | UPDATE_FAILED | 2016-04-11T21:14:28 | overcloud-Controller-xsdpzedowemp | | 1 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | OS::TripleO::Controller | UPDATE_FAILED | 2016-04-11T21:14:34 | overcloud-Controller-xsdpzedowemp | | 2 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | OS::TripleO::Controller | UPDATE_FAILED | 2016-04-11T21:14:42 | overcloud-Controller-xsdpzedowemp heat-engine.log (full log attached): ------------------------------------- 2016-04-11 17:16:08.437 4812 DEBUG heat.engine.scheduler [-] Task _run_to_completion from ResourceGroup "Controller" [403694b7-b282-4683-b0df-c65c9f8fa0d1] St ack "overcloud" [f5e2b5b7-dd20-46b3-b283-ccd351f15e57] running step /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:214 2016-04-11 17:16:08.698 4812 INFO heat.engine.resource [-] UPDATE: ResourceGroup "Controller" [403694b7-b282-4683-b0df-c65c9f8fa0d1] Stack "overcloud" [f5e2b5 b7-dd20-46b3-b283-ccd351f15e57] 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource Traceback (most recent call last): 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 638, in _action_recorder 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource yield 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 968, in update 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource prop_diff]) 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 309, in wrapper 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource step = next(subtask) 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 682, in action_handler_task 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource while not check(handler_data): 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 400, in check_update_complete 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource if not checker.step(): 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 217, in step 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource next(self._runner) 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/openstack/heat/resource_group.py", line 388, in _run_to_completion 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource self).check_update_complete(updater): 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 442, in check_ update_complete 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource cookie=cookie) 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 372, in _check _status_complete 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource action=action) 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource ResourceFailure: resources.Controller: resources[0]: Exception: resources.ExternalPort: Resource updat e already requested 2016-04-11 17:16:08.698 4812 ERROR heat.engine.resource 2016-04-11 17:16:08.848 4812 DEBUG heat.engine.scheduler [-] Task _resource_update from Stack "overcloud" [f5e2b5b7-dd20-46b3-b283-ccd351f15e57] Update cancel led cancel /usr/lib/python2.7/site-packages/heat/engine/scheduler.py:239 journalctl -u openstack-heat-engine.service -b -1: (Attached in: /tmp/openstack-heat-engine.service_b_1 ) ------------------------------------------------------------------- Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 INFO heat.engine.resource [-] UPDATE: SoftwareDeploymentGroup "UpgradeIn Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource Traceback (most recent call last): Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource yield Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource prop_diff]) Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource step = next(subtask) Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource while not check(handler_data): Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource if not checker.step(): Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource next(self._runner) Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource self).check_update_complete(updater): Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource cookie=cookie) Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource File "/usr/lib/python2.7/site-packages/heat Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource action=action) Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource ResourceFailure: resources.UpgradeInitControl Apr 11 15:59:53 instack.localdomain heat-engine[28647]: 2016-04-11 15:59:53.543 28741 ERROR heat.engine.resource
[stack@instack ~]$ heat stack-list -n +--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+---------------------+--------------------------------------+ | id | stack_name | stack_status | creation_time | updated_time | parent | +--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+---------------------+--------------------------------------+ | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | overcloud | UPDATE_FAILED | 2016-04-11T17:41:04 | 2016-04-11T21:13:26 | None | | b46438bf-7daf-4013-92e2-ef8655b3cd38 | overcloud-VipConfig-fb4vhnn6ldoh | UPDATE_COMPLETE | 2016-04-11T17:41:05 | 2016-04-11T21:13:42 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 4cb950b7-cf4f-4c59-b0be-131254668ab2 | overcloud-Networks-ngzmq7rtecic-TenantNetwork-m5u2u27cyyno | UPDATE_COMPLETE | 2016-04-11T17:41:06 | 2016-04-11T21:13:48 | 68a2d539-ba92-424f-847c-bedbcab8f3fb | | 68a2d539-ba92-424f-847c-bedbcab8f3fb | overcloud-Networks-ngzmq7rtecic | UPDATE_COMPLETE | 2016-04-11T17:41:06 | 2016-04-11T21:13:41 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 96f7357c-3db6-4167-aa25-1c0c7d3160b4 | overcloud-Networks-ngzmq7rtecic-ExternalNetwork-v6x5uemc24uj | UPDATE_COMPLETE | 2016-04-11T17:41:06 | 2016-04-11T21:13:46 | 68a2d539-ba92-424f-847c-bedbcab8f3fb | | f18ee5fc-8e44-480b-9125-72b5fd2f6add | overcloud-Networks-ngzmq7rtecic-StorageMgmtNetwork-rbj4endgu6cy | UPDATE_COMPLETE | 2016-04-11T17:41:06 | 2016-04-11T21:13:49 | 68a2d539-ba92-424f-847c-bedbcab8f3fb | | 136c21b7-7451-4a0b-975f-c596dabc8ff1 | overcloud-Networks-ngzmq7rtecic-StorageNetwork-japynb5qmtad | UPDATE_COMPLETE | 2016-04-11T17:41:07 | 2016-04-11T21:13:45 | 68a2d539-ba92-424f-847c-bedbcab8f3fb | | a40edcc1-5722-4a5f-9aea-999726e14491 | overcloud-Networks-ngzmq7rtecic-InternalNetwork-mqakefea677c | UPDATE_COMPLETE | 2016-04-11T17:41:07 | 2016-04-11T21:13:44 | 68a2d539-ba92-424f-847c-bedbcab8f3fb | | 3cb95ca6-bedc-4483-8082-f334573d5785 | overcloud-ObjectStorage-hwnxwsnljo7v | UPDATE_COMPLETE | 2016-04-11T17:41:11 | 2016-04-11T21:13:58 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | e760dcad-94c7-4a45-834a-d7469e540fbd | overcloud-CephStorage-7f3ryswrqz4e | UPDATE_COMPLETE | 2016-04-11T17:41:11 | 2016-04-11T21:14:00 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 60597953-9935-4437-9a9a-1466e9fbc87d | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves | UPDATE_COMPLETE | 2016-04-11T17:41:12 | 2016-04-11T21:14:03 | e760dcad-94c7-4a45-834a-d7469e540fbd | | b236f238-1677-4ee6-b23e-430f74dad81b | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NodeUserData-ru7oesgg2di2 | UPDATE_COMPLETE | 2016-04-11T17:41:12 | 2016-04-11T21:14:16 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 42a27558-ed7d-49ae-ace5-30e2b9959bc0 | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-UpdateConfig-vg2qzk566oi7 | UPDATE_COMPLETE | 2016-04-11T17:41:13 | 2016-04-11T21:14:17 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 42df00bb-3a99-47c2-8d8a-1a67f2ee9bb5 | overcloud-InternalApiVirtualIP-paqhsttvv47v | UPDATE_COMPLETE | 2016-04-11T17:41:13 | 2016-04-11T21:14:07 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 46e904e4-2dc3-4ee6-8275-d5cab4d07a8b | overcloud-RedisVirtualIP-dkjvvlfcqzoj | UPDATE_COMPLETE | 2016-04-11T17:41:13 | 2016-04-11T21:14:04 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 02e34597-f3cc-4ba5-8187-393fd474e54e | overcloud-PublicVirtualIP-pa463jyxg6tb | UPDATE_COMPLETE | 2016-04-11T17:41:14 | 2016-04-11T21:14:02 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 44a71257-0f84-42c8-8257-4c4988a447f0 | overcloud-StorageVirtualIP-h63xf3z5epjf | UPDATE_COMPLETE | 2016-04-11T17:41:14 | 2016-04-11T21:14:05 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 6c524422-d68a-4ffc-bafe-3b820e6535c4 | overcloud-StorageMgmtVirtualIP-poebd77x3s7l | UPDATE_COMPLETE | 2016-04-11T17:41:15 | 2016-04-11T21:14:01 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 31c926ac-97c2-419d-bb86-cf4ab4c40b47 | overcloud-VipMap-dmkmj6gqry5t | UPDATE_COMPLETE | 2016-04-11T17:41:17 | 2016-04-11T21:14:15 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | ffe9de56-b574-44b3-b48e-1a96237e5e88 | overcloud-EndpointMap-t6f63wm3dlqr | UPDATE_COMPLETE | 2016-04-11T17:41:18 | 2016-04-11T21:14:21 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | a8079d01-0652-4d5d-8351-51e930bd0201 | overcloud-BlockStorage-4qkkyzfmig3r | UPDATE_COMPLETE | 2016-04-11T17:41:20 | 2016-04-11T21:14:32 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 8e017c99-285c-49a3-81bb-dd2d0f13b7c1 | overcloud-Compute-qon76f3lrn4z | UPDATE_COMPLETE | 2016-04-11T17:41:21 | 2016-04-11T21:14:29 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 403694b7-b282-4683-b0df-c65c9f8fa0d1 | overcloud-Controller-xsdpzedowemp | UPDATE_FAILED | 2016-04-11T17:41:23 | 2016-04-11T21:14:27 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | e412f3bc-7368-4a6f-a515-3e3003f4e307 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd | UPDATE_COMPLETE | 2016-04-11T17:41:23 | 2016-04-11T21:14:35 | 8e017c99-285c-49a3-81bb-dd2d0f13b7c1 | | d36aba58-8cc2-472b-b44f-1feaaad5d432 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-NodeUserData-xzw4j6fwwx3w | UPDATE_COMPLETE | 2016-04-11T17:41:24 | 2016-04-11T21:14:49 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | e4f512c9-c96b-4b4d-a6fc-aada5c097013 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-UpdateConfig-3d45m6cujdqe | UPDATE_COMPLETE | 2016-04-11T17:41:25 | 2016-04-11T21:14:47 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | 5f53e7b5-1d26-4c06-a7ef-440dd04342b4 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-UpdateConfig-5wwv6smewgaq | UPDATE_COMPLETE | 2016-04-11T17:41:27 | 2016-04-11T21:15:15 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa | UPDATE_FAILED | 2016-04-11T17:41:27 | 2016-04-11T21:14:47 | 403694b7-b282-4683-b0df-c65c9f8fa0d1 | | 1ca49884-28c8-40ce-89a9-e7ed84c527db | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NodeUserData-c2pxbxz7we2e | UPDATE_COMPLETE | 2016-04-11T17:41:28 | 2016-04-11T21:15:19 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 1d4841a7-6afa-4d57-892a-bbacb65356ff | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m | UPDATE_FAILED | 2016-04-11T17:41:31 | 2016-04-11T21:14:40 | 403694b7-b282-4683-b0df-c65c9f8fa0d1 | | 0a44484b-6b97-484c-a0f5-e9ea3258729c | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-UpdateConfig-c5pcqg44ij6q | UPDATE_COMPLETE | 2016-04-11T17:41:33 | 2016-04-11T21:15:03 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | d29de9ec-d1e9-4b25-81d2-e39741e3149c | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NodeUserData-2juylph7wfha | UPDATE_COMPLETE | 2016-04-11T17:41:33 | 2016-04-11T21:15:01 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | dbd43243-7b05-4648-9f04-69d815f7ff34 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti | UPDATE_FAILED | 2016-04-11T17:41:41 | 2016-04-11T21:14:33 | 403694b7-b282-4683-b0df-c65c9f8fa0d1 | | 5e1d1105-810a-47bf-965f-486c53c73f0e | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-UpdateConfig-tb2gjimkhsjq | UPDATE_COMPLETE | 2016-04-11T17:41:43 | 2016-04-11T21:14:51 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 0d1edca4-df2a-4512-8361-82c255918082 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NodeUserData-6lj225qqixox | UPDATE_COMPLETE | 2016-04-11T17:41:44 | 2016-04-11T21:14:52 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 837a127d-bc15-40ec-b0b2-06005b91e2ca | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-StoragePort-g7gpkiwclfsn | UPDATE_COMPLETE | 2016-04-11T17:48:50 | 2016-04-11T21:14:32 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 808c83f4-3b28-4fbc-aa53-c4845593055b | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-TenantPort-yf5qcaclpn44 | UPDATE_COMPLETE | 2016-04-11T17:48:51 | 2016-04-11T21:15:02 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | bf56d78e-cb0d-4edf-8123-848e3bb65855 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-ExternalPort-i3sgcxcn7j4y | UPDATE_IN_PROGRESS | 2016-04-11T17:48:53 | 2016-04-11T20:37:24 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 1537b527-f0f4-4683-86cb-f8d419deee40 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-InternalApiPort-o5vdkuwfcxf3 | UPDATE_COMPLETE | 2016-04-11T17:48:54 | 2016-04-11T21:15:11 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | b3a1b175-b241-4765-8b14-1d79198bd3fc | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-StorageMgmtPort-alluhcr2nvfm | UPDATE_COMPLETE | 2016-04-11T17:48:54 | 2016-04-11T21:14:35 | 60597953-9935-4437-9a9a-1466e9fbc87d | | cee843a9-fc7d-4767-a588-e7c9223ce769 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-InternalApiPort-is5jk6mmvd6n | UPDATE_COMPLETE | 2016-04-11T17:48:56 | 2016-04-11T21:11:46 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 2c1e0c37-a95c-4073-ab41-ca18d51a0944 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-TenantPort-e2qm2piyoq6i | UPDATE_COMPLETE | 2016-04-11T17:48:57 | 2016-04-11T21:15:44 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 9cd530a8-b37b-40a0-b85c-b248179c8e94 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-StoragePort-42xg552zagfi | UPDATE_COMPLETE | 2016-04-11T17:48:58 | 2016-04-11T21:15:04 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | ad5bbad4-02b5-4ff8-8287-678e34b316e8 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-StoragePort-7dzszii75fsi | UPDATE_COMPLETE | 2016-04-11T17:49:03 | 2016-04-11T20:37:01 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | a6fe22fb-49de-4515-bc67-57e529633e5f | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NetIpMap-wndwfcvkl3tx | UPDATE_COMPLETE | 2016-04-11T17:49:05 | 2016-04-11T21:14:54 | 60597953-9935-4437-9a9a-1466e9fbc87d | | aa721242-d69c-48fb-8a72-2ce8965e7694 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-InternalApiPort-lrxapduuedsz | UPDATE_COMPLETE | 2016-04-11T17:49:05 | 2016-04-11T21:15:38 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | b0fc205a-6ebe-4ec7-b4e3-ceaf313ea5f2 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-TenantPort-zu3t2c5ag5op | UPDATE_COMPLETE | 2016-04-11T17:49:05 | 2016-04-11T20:37:00 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 61194d0a-277a-4eae-9f02-9d38c5696c99 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-TenantPort-v4kzdt3l3n2p | UPDATE_COMPLETE | 2016-04-11T17:49:06 | 2016-04-11T20:37:16 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 2d83a19d-f33d-4543-bac6-1262f9944f18 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-StoragePort-nq4sjcxw4rkf | UPDATE_COMPLETE | 2016-04-11T17:49:08 | 2016-04-11T20:37:19 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 4381b5c3-8cf9-4d03-a88c-90d5c287f894 | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NetworkConfig-ydxl4fcfyzij | UPDATE_COMPLETE | 2016-04-11T17:49:08 | 2016-04-11T21:14:59 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 78dd53c6-b107-4464-9f88-576e93849ef9 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-ExternalPort-s37ltz6ovrdi | UPDATE_COMPLETE | 2016-04-11T17:49:09 | 2016-04-11T21:15:31 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 3c780e02-35b9-459c-aacb-63e5f82df122 | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NetIpSubnetMap-kmypveenfnb7 | UPDATE_COMPLETE | 2016-04-11T17:49:10 | 2016-04-11T21:15:02 | 60597953-9935-4437-9a9a-1466e9fbc87d | | a25d79b2-dc12-42ae-9d65-7a756030cbcb | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-StorageMgmtPort-rhxma4fp47c2 | UPDATE_COMPLETE | 2016-04-11T17:49:12 | 2016-04-11T20:37:22 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | efed78f7-1b29-479a-885b-faa04685d70f | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-NetworkConfig-zxqeun5k23va | UPDATE_COMPLETE | 2016-04-11T17:49:12 | 2016-04-11T21:15:37 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | 64d161ef-743c-442f-bcac-aa1ed281302a | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-StorageMgmtPort-35j6vlr2q6m5 | UPDATE_IN_PROGRESS | 2016-04-11T17:49:13 | 2016-04-11T20:37:29 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 484be0ab-e548-4d71-8c81-8872c5b838ba | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-NetIpMap-yjkecmsce2cf | UPDATE_COMPLETE | 2016-04-11T17:49:16 | 2016-04-11T21:15:40 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | abf726f0-2bf9-4e5b-ae24-cf4de1e0929e | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-StoragePort-j4yprl7dgmzk | UPDATE_COMPLETE | 2016-04-11T17:49:16 | 2016-04-11T21:15:40 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | b7d79f61-e67b-4df3-a2d6-7ea7f9a62599 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-ExternalPort-dbpafsf7wdkv | UPDATE_COMPLETE | 2016-04-11T17:49:17 | 2016-04-11T20:37:09 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 772d682c-c5ba-4ac3-ad5a-73c7e225def9 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-InternalApiPort-ed4f5rj2vbcm | UPDATE_COMPLETE | 2016-04-11T17:49:18 | 2016-04-11T20:37:07 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 264ff604-f43c-4ec3-8bce-34cc804af7c1 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-StorageMgmtPort-nefcykzwquqj | UPDATE_COMPLETE | 2016-04-11T17:49:20 | 2016-04-11T21:15:36 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 659d762d-0e41-4a29-82d5-f2b6798f0144 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NetIpMap-jnolqi6eugup | UPDATE_COMPLETE | 2016-04-11T17:49:27 | 2016-04-11T20:11:09 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 8392e8b4-4609-42e5-90c2-2bc69e77bc82 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NetIpSubnetMap-3dl2baaoqb4t | UPDATE_COMPLETE | 2016-04-11T17:49:30 | 2016-04-11T20:11:01 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 8838aab1-895d-4440-9b01-84f82f114793 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NetworkConfig-nle6ehdkmtg4 | UPDATE_COMPLETE | 2016-04-11T17:49:32 | 2016-04-11T20:11:05 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 9ab25d12-5045-4a0a-8481-da4f98d6cb92 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NetIpSubnetMap-az32zugplw3e | UPDATE_COMPLETE | 2016-04-11T17:49:32 | 2016-04-11T20:11:06 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | c05ba464-6034-421b-8207-dc94605c0c53 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NetIpMap-7uljunx3ehvq | UPDATE_COMPLETE | 2016-04-11T17:49:33 | 2016-04-11T20:11:22 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | f1d56ec1-cc5b-4b82-a4f1-e0ebb0affaf8 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NetworkConfig-miqqiabzrcev | UPDATE_COMPLETE | 2016-04-11T17:49:35 | 2016-04-11T21:15:58 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | c360e490-a118-4acf-b3f9-719eccc3f362 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NetworkConfig-2e2oftdy5mbw | UPDATE_COMPLETE | 2016-04-11T17:49:37 | 2016-04-11T20:11:02 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 0a50861a-7a8b-4094-96f8-93e580a46e13 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NetIpMap-nzbzjdqv3eeq | UPDATE_COMPLETE | 2016-04-11T17:49:38 | 2016-04-11T20:10:58 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | d7f03820-3593-4acb-8afc-b4a0db57a7b2 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NetIpSubnetMap-bguxfaag24s2 | UPDATE_COMPLETE | 2016-04-11T17:49:41 | 2016-04-11T21:12:33 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 109fdf63-660d-476d-9ef2-a07a5b25d06b | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NodeTLSCAData-5h6ubwyu5i67 | UPDATE_COMPLETE | 2016-04-11T17:50:27 | 2016-04-11T21:15:13 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 76100f14-44b4-4348-ae76-b16fee663b4b | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NodeTLSCAData-snyyg7v4zf57 | UPDATE_COMPLETE | 2016-04-11T17:50:27 | 2016-04-11T20:11:20 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | f623a6d5-9b85-4b4f-831c-41a8ebb96048 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NodeTLSCAData-xp5jwnjlvp5n | UPDATE_COMPLETE | 2016-04-11T17:50:29 | 2016-04-11T20:11:14 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | b094a905-1c5d-465a-9a26-60787c888b6d | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NodeTLSCAData-dpafvhpiaiuh | UPDATE_COMPLETE | 2016-04-11T17:50:33 | 2016-04-11T20:11:32 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 32fa58fa-85f6-4764-bd09-e74882276a13 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-NodeTLSCAData-6qo2cunn2rtl | UPDATE_COMPLETE | 2016-04-11T17:51:04 | 2016-04-11T21:15:52 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | d5a8ee8c-635b-4938-b2cb-4f4703c2b11a | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-ComputeExtraConfigPre-e4bhfn4sru5r | UPDATE_COMPLETE | 2016-04-11T17:51:05 | 2016-04-11T21:15:50 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | c9a11080-78fa-4b62-ba90-6f7740a4e594 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NodeTLSData-iboo6zet64ll | UPDATE_COMPLETE | 2016-04-11T17:51:10 | 2016-04-11T20:11:30 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 66fb6c3d-11fc-4fb0-977e-9a68166e3dae | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NodeTLSData-spvrhkajdwbz | UPDATE_COMPLETE | 2016-04-11T17:51:12 | 2016-04-11T20:11:28 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 3f48e41e-af58-4ff7-9865-75a131e4ee8d | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NodeExtraConfig-cwku5pjmswvo | UPDATE_COMPLETE | 2016-04-11T17:51:13 | 2016-04-11T21:15:23 | 60597953-9935-4437-9a9a-1466e9fbc87d | | ee2f67dc-cff3-492b-98ed-10db9953921d | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NodeTLSData-ioirmycvtef5 | UPDATE_COMPLETE | 2016-04-11T17:51:17 | 2016-04-11T20:11:40 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 9d983bd0-ede7-4d77-9320-80cbe306f367 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-NodeExtraConfig-bo64giif7fs5 | UPDATE_COMPLETE | 2016-04-11T17:51:49 | 2016-04-11T21:16:01 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | e4a12dd3-cf01-44aa-a74c-c775b9b369ee | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-ControllerExtraConfigPre-mbwpwhz44car | UPDATE_COMPLETE | 2016-04-11T17:52:42 | 2016-04-11T20:13:25 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | 3bf3392d-6315-431a-84ed-bc301fdd36e8 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-ControllerExtraConfigPre-v5jhvee5wsrv | UPDATE_COMPLETE | 2016-04-11T17:52:44 | 2016-04-11T20:13:12 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 8975fa60-d3e6-4448-8fdc-c2160b5c057a | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NodeExtraConfig-evkcasdhyzz6 | UPDATE_COMPLETE | 2016-04-11T17:52:44 | 2016-04-11T20:13:32 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | f6ef6915-aa3c-4c70-b911-ede5583bd495 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NodeExtraConfig-axpk7bfcnebn | UPDATE_COMPLETE | 2016-04-11T17:52:46 | 2016-04-11T20:13:22 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 7f48ad8a-2f48-4f54-aa8a-b56dd39291cf | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-ControllerExtraConfigPre-3djcxewzltix | UPDATE_COMPLETE | 2016-04-11T17:52:49 | 2016-04-11T20:13:30 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | ea8cbbf9-edea-4465-b80a-85cb8a6c191b | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NodeExtraConfig-b3gym6ejqljx | UPDATE_COMPLETE | 2016-04-11T17:52:51 | 2016-04-11T20:13:37 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 02691308-a8e4-4812-8ca9-6d3de16b822d | overcloud-VipDeployment-n2evwsj37rib | UPDATE_COMPLETE | 2016-04-11T17:52:54 | 2016-04-11T20:14:01 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | e7c95c72-6bda-4992-a14e-a83d6b6dca27 | overcloud-AllNodesValidationConfig-ayfmqhbotdp2 | UPDATE_COMPLETE | 2016-04-11T17:52:55 | 2016-04-11T20:14:16 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 83fbfaea-5a82-4616-8568-9073c880a4a3 | overcloud-ControllerBootstrapNodeConfig-es7zyuzk7ttt | UPDATE_COMPLETE | 2016-04-11T17:52:57 | 2016-04-11T20:14:00 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | f97c806c-0844-4952-87f7-7439c2a77c81 | overcloud-SwiftDevicesAndProxyConfig-q5q4dvr5rkzd | UPDATE_COMPLETE | 2016-04-11T17:52:57 | 2016-04-11T20:14:08 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 7a127dee-018a-411e-8a09-923845e33ebe | overcloud-ControllerIpListMap-ns4rnho235kg | UPDATE_COMPLETE | 2016-04-11T17:52:58 | 2016-04-11T20:14:14 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 8541bd66-30ec-4fa9-b8da-2778e8cd15be | overcloud-AllNodesExtraConfig-f2kv23iuubcm | UPDATE_COMPLETE | 2016-04-11T17:52:59 | 2016-04-11T20:16:49 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 62bba8d2-009d-4ddd-a58c-9710e0bc0386 | overcloud-ControllerSwiftDeployment-bw52cj5uzy56 | UPDATE_COMPLETE | 2016-04-11T17:53:01 | 2016-04-11T20:14:27 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 438905d8-8a18-4fc0-883b-5e3bebc7cbf9 | overcloud-CephClusterConfig-6etbnlljdspu | UPDATE_COMPLETE | 2016-04-11T17:53:02 | 2016-04-11T20:14:33 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 3201e552-9085-4273-a9cf-44041d6fd56a | overcloud-ControllerBootstrapNodeDeployment-gxuqohbtugl6 | UPDATE_COMPLETE | 2016-04-11T17:53:03 | 2016-04-11T20:14:25 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 3c1f7e02-9086-411d-8903-22c001e82b39 | overcloud-ObjectStorageSwiftDeployment-jsznqfm2evep | UPDATE_COMPLETE | 2016-04-11T17:53:04 | 2016-04-11T20:14:31 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 606cbe62-fbf3-4d9d-aff7-a837c96ded6c | overcloud-allNodesConfig-6wp7d7h2lw5l | UPDATE_COMPLETE | 2016-04-11T17:53:05 | 2016-04-11T20:14:36 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 0a30a6b4-9383-4d80-9216-ae9a486f33f2 | overcloud-BlockStorageAllNodesDeployment-qabw2ulnnbd6 | UPDATE_COMPLETE | 2016-04-11T17:53:09 | 2016-04-11T20:14:59 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 7d4a6a56-bc93-4e4e-9148-bba64f12c364 | overcloud-CephStorageCephDeployment-uvcy6khnnzly | UPDATE_COMPLETE | 2016-04-11T17:53:11 | 2016-04-11T20:14:48 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | aa6e409d-0b4f-4e1c-a556-52aaa00697ab | overcloud-ControllerAllNodesDeployment-3cubo6mgkwhj | UPDATE_COMPLETE | 2016-04-11T17:53:11 | 2016-04-11T20:15:04 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 0867a764-a205-4723-8c1d-46d558ac4cb8 | overcloud-ComputeAllNodesDeployment-gvstuyxxf4gr | UPDATE_COMPLETE | 2016-04-11T17:53:13 | 2016-04-11T20:15:01 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 019d9bec-0a0d-467d-a00a-f688b6f95b18 | overcloud-ControllerCephDeployment-ymbw3pm2lhdo | UPDATE_COMPLETE | 2016-04-11T17:53:14 | 2016-04-11T20:14:46 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 3caf4dda-f369-477e-890e-8db2c8504e92 | overcloud-ObjectStorageAllNodesDeployment-6tgrjw6h7rwy | UPDATE_COMPLETE | 2016-04-11T17:53:15 | 2016-04-11T20:15:07 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 0977e627-e04a-4438-b2a7-89e93ca49678 | overcloud-ComputeCephDeployment-pqpqfkdx4rzf | UPDATE_COMPLETE | 2016-04-11T17:53:17 | 2016-04-11T20:14:51 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 73967b83-ad33-4f5a-b800-3ae1ee044ef8 | overcloud-CephStorageAllNodesDeployment-immvkdmezihu | UPDATE_COMPLETE | 2016-04-11T17:53:19 | 2016-04-11T20:15:09 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 8887cc1b-c7eb-4d74-bd31-456e82e7f714 | overcloud-ObjectStorageNodesPostDeployment-xjbzrwloabu7 | CREATE_COMPLETE | 2016-04-11T17:53:23 | None | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | ccff38a2-2cbf-4d52-b71b-8b272d340ea7 | overcloud-ObjectStorageAllNodesValidationDeployment-4voclwwudafq | UPDATE_COMPLETE | 2016-04-11T17:53:25 | 2016-04-11T20:15:21 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 0321d3d7-a6b2-4f69-8b73-3ebf654933e7 | overcloud-ObjectStorageNodesPostDeployment-xjbzrwloabu7-StorageDeployment_Step1-hckwgn3lfb3q | CREATE_COMPLETE | 2016-04-11T17:53:26 | None | 8887cc1b-c7eb-4d74-bd31-456e82e7f714 | | 071c2f65-e8f4-4310-a84f-da11fde0fbfc | overcloud-BlockStorageAllNodesValidationDeployment-d74agbqyqoc7 | UPDATE_COMPLETE | 2016-04-11T17:53:27 | 2016-04-11T20:15:19 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 0f236107-9997-4df9-82b7-c14641706665 | overcloud-ObjectStorageNodesPostDeployment-xjbzrwloabu7-StorageRingbuilderDeployment_Step2-r54wbcdii7mq | CREATE_COMPLETE | 2016-04-11T17:53:28 | None | 8887cc1b-c7eb-4d74-bd31-456e82e7f714 | | 58ceddf9-9818-466f-a3fa-14ea057bdf7f | overcloud-ObjectStorageNodesPostDeployment-xjbzrwloabu7-StoragePostPuppetDeployment-asyn7dfboydc | CREATE_COMPLETE | 2016-04-11T17:53:31 | None | 8887cc1b-c7eb-4d74-bd31-456e82e7f714 | | 9e529b85-7bb5-4ae3-9c4a-779763d95622 | overcloud-ObjectStorageNodesPostDeployment-xjbzrwloabu7-StoragePostPuppetDeployment-asyn7dfboydc-SwiftPostPuppetYumUpdateDeployment-25bir2evpti3 | CREATE_COMPLETE | 2016-04-11T17:53:32 | None | 58ceddf9-9818-466f-a3fa-14ea057bdf7f | | 8ca46dcf-2516-408f-b17a-17697f8d2119 | overcloud-ObjectStorageNodesPostDeployment-xjbzrwloabu7-ExtraConfig-nsqednx5e3zy | CREATE_COMPLETE | 2016-04-11T17:53:35 | None | 8887cc1b-c7eb-4d74-bd31-456e82e7f714 | | 6b85b689-f3ff-4c7a-ba50-9dccbcc3e326 | overcloud-ComputeAllNodesValidationDeployment-577mx7ib3rvu | UPDATE_COMPLETE | 2016-04-11T17:53:45 | 2016-04-11T20:15:42 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 92480d0c-6681-4de5-b595-ae88b57aa902 | overcloud-ComputeNodesPostDeployment-2755xfozxrgf | CREATE_COMPLETE | 2016-04-11T17:53:49 | None | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 96a4a6d0-6bd3-4d95-8889-8a876f3005a8 | overcloud-CephStorageAllNodesValidationDeployment-ak4v26j3ldrb | UPDATE_COMPLETE | 2016-04-11T17:53:53 | 2016-04-11T20:16:00 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | f86705f1-b709-4751-9cde-030b3bc40a98 | overcloud-ComputeNodesPostDeployment-2755xfozxrgf-ComputePuppetDeployment-26us64sm3y2n | CREATE_COMPLETE | 2016-04-11T17:53:54 | None | 92480d0c-6681-4de5-b595-ae88b57aa902 | | 1b21d408-4d86-4753-bbbb-0d5fc87ec9da | overcloud-ControllerAllNodesValidationDeployment-vzfkh5bqxepb | UPDATE_COMPLETE | 2016-04-11T17:54:01 | 2016-04-11T20:16:42 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | ca60c130-2c85-4449-bb97-e67a4280a86d | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr | CREATE_COMPLETE | 2016-04-11T17:54:36 | None | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 3e577e12-2385-458d-bb5f-52266ad6a4e9 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerPuppetConfig-ahcpzp6zuzdf | CREATE_COMPLETE | 2016-04-11T17:54:37 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | cb94e92b-9249-421c-bf1a-59099bfaf181 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerPrePuppet-zt5yw2tmwqmx | CREATE_COMPLETE | 2016-04-11T17:54:38 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | ff577846-f06f-4ad2-89be-cff05b8ceb3b | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerPrePuppet-zt5yw2tmwqmx-ControllerPrePuppetMaintenanceModeDeployment-w4jlc7y2rect | CREATE_COMPLETE | 2016-04-11T17:54:40 | None | cb94e92b-9249-421c-bf1a-59099bfaf181 | | 5197cb88-cf12-40a4-a0e6-4b47e63b4426 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerLoadBalancerDeployment_Step1-ftv2mtp3we7y | CREATE_COMPLETE | 2016-04-11T17:55:53 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 9f7d3317-e94c-4ff1-a58e-e5c98345a7ab | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerServicesBaseDeployment_Step2-rio7xz4cpm36 | CREATE_COMPLETE | 2016-04-11T17:58:06 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 6fa12118-c2d0-4eac-b99b-6884b10bfb4e | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerRingbuilderDeployment_Step3-a5kttu7etnkl | CREATE_COMPLETE | 2016-04-11T18:00:43 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | fd7af9c0-4ce2-4b46-85ed-84f28c860315 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerOvercloudServicesDeployment_Step4-pybxctofjcls | CREATE_COMPLETE | 2016-04-11T18:01:54 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 2deec7df-95ff-4697-a9e9-30c3ddf429d2 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerOvercloudServicesDeployment_Step5-umhq2hf6sojf | CREATE_COMPLETE | 2016-04-11T18:04:51 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 1be7479c-0fdb-454d-98e0-c12a3354cd5b | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerOvercloudServicesDeployment_Step6-5sdmrntrtf2l | CREATE_COMPLETE | 2016-04-11T18:07:56 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 9fd4d18e-11d1-4065-9911-ce980d63ecef | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerOvercloudServicesDeployment_Step7-re5eaoezueow | CREATE_COMPLETE | 2016-04-11T18:10:28 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 48cf4880-9e86-43c2-8416-be90afac0e5e | overcloud-ComputeNodesPostDeployment-2755xfozxrgf-ComputePostPuppetDeployment-kkrcdw3ifcp5 | CREATE_COMPLETE | 2016-04-11T18:11:25 | None | 92480d0c-6681-4de5-b595-ae88b57aa902 | | 01a0dce1-085f-41f7-9426-2d233b4c3091 | overcloud-ComputeNodesPostDeployment-2755xfozxrgf-ComputePostPuppetDeployment-kkrcdw3ifcp5-ComputePostPuppetYumUpdateDeployment-q55qcqjodtkx | CREATE_COMPLETE | 2016-04-11T18:11:27 | None | 48cf4880-9e86-43c2-8416-be90afac0e5e | | 35d5bb83-ef36-4b7c-9342-92739f4e4643 | overcloud-ComputeNodesPostDeployment-2755xfozxrgf-ExtraConfig-xhamhvuhi3ik | CREATE_COMPLETE | 2016-04-11T18:12:19 | None | 92480d0c-6681-4de5-b595-ae88b57aa902 | | 1ed03194-d484-4b22-966b-ba8ceff73608 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerPostPuppet-xk4wfwbclg33 | CREATE_COMPLETE | 2016-04-11T18:12:59 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 2d107dd6-da76-4cb6-bfcf-e3f0acd4ebe3 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerPostPuppet-xk4wfwbclg33-ControllerPostPuppetMaintenanceModeDeployment-ytmi3esuhqrw | CREATE_COMPLETE | 2016-04-11T18:13:02 | None | 1ed03194-d484-4b22-966b-ba8ceff73608 | | 4cf86ddf-7216-43a8-8be3-e5e8d0742331 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ControllerPostPuppet-xk4wfwbclg33-ControllerPostPuppetRestartDeployment-qe4kecrlnpev | CREATE_COMPLETE | 2016-04-11T18:14:12 | None | 1ed03194-d484-4b22-966b-ba8ceff73608 | | 9454ac73-bb94-4056-b053-c96a16d9c741 | overcloud-ControllerNodesPostDeployment-xstjyqs6m5dr-ExtraConfig-coambwfte2lw | CREATE_COMPLETE | 2016-04-11T18:20:04 | None | ca60c130-2c85-4449-bb97-e67a4280a86d | | 9660daf1-ff6f-4477-839e-77a276c63d6b | overcloud-BlockStorageNodesPostDeployment-vlddqk6zs6ft | CREATE_COMPLETE | 2016-04-11T18:20:09 | None | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 8c19abda-d9a9-4f8d-8438-76393d05bf2c | overcloud-BlockStorageNodesPostDeployment-vlddqk6zs6ft-VolumeDeployment_Step1-nw7sbtw5zgh2 | CREATE_COMPLETE | 2016-04-11T18:20:10 | None | 9660daf1-ff6f-4477-839e-77a276c63d6b | | efa5cafe-d1a1-443e-958f-2bba8f7888d4 | overcloud-BlockStorageNodesPostDeployment-vlddqk6zs6ft-VolumePostPuppetDeployment-rl72lwx3ytyn | CREATE_COMPLETE | 2016-04-11T18:20:12 | None | 9660daf1-ff6f-4477-839e-77a276c63d6b | | fb57b631-0ff0-46e0-b930-bee2de8bfb83 | overcloud-CephStorageNodesPostDeployment-lmpcd6qitejc | CREATE_COMPLETE | 2016-04-11T18:20:12 | None | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | b977d36c-1508-458d-9492-f5c9df17f92f | overcloud-BlockStorageNodesPostDeployment-vlddqk6zs6ft-VolumePostPuppetDeployment-rl72lwx3ytyn-CinderPostPuppetYumUpdateDeployment-53ngvl76mfzd | CREATE_COMPLETE | 2016-04-11T18:20:13 | None | efa5cafe-d1a1-443e-958f-2bba8f7888d4 | | baaa0c96-a527-4b24-a184-07f34b1f2bdd | overcloud-CephStorageNodesPostDeployment-lmpcd6qitejc-CephStorageDeployment_Step1-o346ldzuccbo | CREATE_COMPLETE | 2016-04-11T18:20:14 | None | fb57b631-0ff0-46e0-b930-bee2de8bfb83 | | d09e9013-6635-4672-8eb9-84ec07523ef9 | overcloud-CephStorageNodesPostDeployment-lmpcd6qitejc-CephStoragePostPuppetDeployment-hbgaf2h4hmfp | CREATE_COMPLETE | 2016-04-11T18:21:02 | None | fb57b631-0ff0-46e0-b930-bee2de8bfb83 | | 3e441902-5220-48c5-b0db-5111f2e3b874 | overcloud-CephStorageNodesPostDeployment-lmpcd6qitejc-CephStoragePostPuppetDeployment-hbgaf2h4hmfp-CephPostPuppetYumUpdateDeployment-2ipayjbf4jbo | CREATE_COMPLETE | 2016-04-11T18:21:03 | None | d09e9013-6635-4672-8eb9-84ec07523ef9 | | c3a3075d-1127-4d51-8b5c-589d69768f7e | overcloud-CephStorageNodesPostDeployment-lmpcd6qitejc-ExtraConfig-xkutsewoeukr | CREATE_COMPLETE | 2016-04-11T18:21:51 | None | fb57b631-0ff0-46e0-b930-bee2de8bfb83 | | f8bf85b4-811a-4bd1-939a-7b8ac3ec3fa4 | overcloud-Networks-ngzmq7rtecic-ManagementNetwork-ffz3abnbcrtx | UPDATE_COMPLETE | 2016-04-11T18:54:47 | 2016-04-11T21:13:47 | 68a2d539-ba92-424f-847c-bedbcab8f3fb | | dbe32cdd-640c-4f80-ab12-48079decc26b | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-NodeAdminUserData-rlahn7c3vlik | UPDATE_COMPLETE | 2016-04-11T18:55:11 | 2016-04-11T21:14:19 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 1b84740b-31ab-457c-a919-1512492a4f99 | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-ExternalPort-lfw3ghwo34il | UPDATE_COMPLETE | 2016-04-11T18:55:29 | 2016-04-11T21:14:37 | 60597953-9935-4437-9a9a-1466e9fbc87d | | f1d86df3-0505-444a-9f7a-4df5e10e05b6 | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-InternalApiPort-smejltxjduex | UPDATE_COMPLETE | 2016-04-11T18:55:31 | 2016-04-11T21:14:29 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 116948d2-cdd3-4fd7-bab7-33609c9706cb | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-TenantPort-vnewcqxet4bo | UPDATE_COMPLETE | 2016-04-11T18:55:37 | 2016-04-11T21:14:43 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 58b4ee30-58b1-4911-8017-af87e58d8419 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-NodeAdminUserData-pxcniicbynkx | UPDATE_COMPLETE | 2016-04-11T18:55:42 | 2016-04-11T21:14:46 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | b9b37c86-25c3-461e-b409-3fe16e9f8b2c | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-ManagementPort-abepcr2jrsuq | UPDATE_COMPLETE | 2016-04-11T18:55:44 | 2016-04-11T21:14:40 | 60597953-9935-4437-9a9a-1466e9fbc87d | | 238155c3-2cca-4c83-a4c1-deac3893887e | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-NodeAdminUserData-wipjcfrprorm | UPDATE_COMPLETE | 2016-04-11T18:55:45 | 2016-04-11T21:15:07 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 186b1138-04ee-48f8-9b87-41e12f02d2df | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-NodeAdminUserData-de4osqw4sr26 | UPDATE_COMPLETE | 2016-04-11T18:55:54 | 2016-04-11T21:15:07 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | 6fadaf72-2f08-4192-90c7-a01d89fde881 | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-NodeAdminUserData-7feaz5enwwxy | UPDATE_COMPLETE | 2016-04-11T18:56:02 | 2016-04-11T21:14:49 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | a0f164c7-9ce0-472e-a662-1cf7af067282 | overcloud-Controller-xsdpzedowemp-1-4otht4yypv3m-ManagementPort-rhaped7msc5m | UPDATE_COMPLETE | 2016-04-11T18:56:05 | 2016-04-11T21:15:33 | 1d4841a7-6afa-4d57-892a-bbacb65356ff | | 73f04a8a-79e7-4dff-8664-25ccdccf69c8 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-StorageMgmtPort-4lkkw4ls2io7 | UPDATE_COMPLETE | 2016-04-11T18:56:17 | 2016-04-11T21:15:15 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | 2990f22f-66f2-402e-a1e4-1241ee8b728b | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-ManagementPort-aanzylygwksi | UPDATE_COMPLETE | 2016-04-11T18:56:20 | 2016-04-11T21:15:08 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | 44af90d4-557d-471d-be92-7977305961e0 | overcloud-Compute-qon76f3lrn4z-0-vufnthn5j7bd-ExternalPort-h4rsllx2ob4h | UPDATE_COMPLETE | 2016-04-11T18:56:25 | 2016-04-11T21:15:19 | e412f3bc-7368-4a6f-a515-3e3003f4e307 | | 4d1cd883-f381-42e1-bacd-28e63417db86 | overcloud-Controller-xsdpzedowemp-2-e66yd6kjrdoa-ManagementPort-juwj7q6gqa7q | UPDATE_COMPLETE | 2016-04-11T18:56:30 | 2016-04-11T20:37:12 | 8d106a0d-e7b1-43ca-a85b-bed5b2ff5f63 | | af95774f-633b-47de-b09c-2b00664175de | overcloud-Controller-xsdpzedowemp-0-kptf2n6og6ti-ManagementPort-g4ro7tmmrk4g | UPDATE_COMPLETE | 2016-04-11T18:56:36 | 2016-04-11T20:36:55 | dbd43243-7b05-4648-9f04-69d815f7ff34 | | f334edee-a5d3-4552-b6b1-2f99937db008 | overcloud-CephStorage-7f3ryswrqz4e-0-u2rtrhpchves-CephStorageExtraConfigPre-sblpjehbbwho | UPDATE_COMPLETE | 2016-04-11T19:38:13 | 2016-04-11T21:15:11 | 60597953-9935-4437-9a9a-1466e9fbc87d | | a0119733-9d0f-4449-a1cd-33cfbd93093a | overcloud-UpdateWorkflow-bo7drt3wqk7z | UPDATE_COMPLETE | 2016-04-11T19:39:32 | 2016-04-11T20:14:06 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 999795d0-4a43-4bc0-b50b-5eb7c78228d9 | overcloud-UpdateWorkflow-bo7drt3wqk7z-ComputeDeliverUpgradeScriptDeployment-kp7a5yowdghf | UPDATE_COMPLETE | 2016-04-11T19:39:35 | 2016-04-11T20:14:22 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | fad15f82-4b87-4328-b950-ae2e8c60418c | overcloud-UpdateWorkflow-bo7drt3wqk7z-ObjectStorageDeliverUpgradeScriptDeployment-qcoonknps2x6 | UPDATE_COMPLETE | 2016-04-11T19:39:38 | 2016-04-11T20:14:11 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | e6542367-ae48-49ce-8a14-ceb78861e846 | overcloud-UpdateWorkflow-bo7drt3wqk7z-CephStorageDeliverUpgradeScriptDeployment-5phldfw2bj3y | UPDATE_COMPLETE | 2016-04-11T19:39:43 | 2016-04-11T20:14:10 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | de122452-5c35-414a-9a42-f1eeae99ad23 | overcloud-UpdateWorkflow-bo7drt3wqk7z-UpgradeInitComputeDeployment-o4sgegynqexc | UPDATE_COMPLETE | 2016-04-11T19:39:48 | 2016-04-11T20:14:14 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | ca19ce6c-32a1-4b00-988f-3ec97b679869 | overcloud-UpdateWorkflow-bo7drt3wqk7z-UpgradeInitObjectStorageDeployment-4zvr53f7best | UPDATE_COMPLETE | 2016-04-11T19:39:52 | 2016-04-11T20:14:17 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | 77e33cce-21c7-4cd8-9b12-37c1a9c76d7f | overcloud-UpdateWorkflow-bo7drt3wqk7z-UpgradeInitBlockStorageDeployment-oy6wst7zh7s2 | UPDATE_COMPLETE | 2016-04-11T19:39:57 | 2016-04-11T20:14:20 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | 99b82750-04bf-4c3d-baf0-f83e59f02b7e | overcloud-UpdateWorkflow-bo7drt3wqk7z-UpgradeInitControllerDeployment-znldg2eazelc | UPDATE_COMPLETE | 2016-04-11T19:40:02 | 2016-04-11T20:14:13 | a0119733-9d0f-4449-a1cd-33cfbd93093a | | cdb1a9be-7868-41d3-b245-b7cb22603e32 | overcloud-ControllerClusterDeployment-uvom2qp3xtcn | UPDATE_COMPLETE | 2016-04-11T19:40:04 | 2016-04-11T20:14:04 | f5e2b5b7-dd20-46b3-b283-ccd351f15e57 | | 36db6d95-3871-447f-9183-99f46bf7a992 | overcloud-UpdateWorkflow-bo7drt3wqk7z-UpgradeInitCephStorageDeployment-olnqoqajwh6n | UPDATE_COMPLETE | 2016-04-11T19:40:06 | 2016-04-11T20:14:18 | a0119733-9d0f-4449-a1cd-33cfbd93093a | +--------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+---------------------+---------------------+--------------------------------------+
Created attachment 1146149 [details] logs Adding Full heat-engine.log and openstack-heat-engine.service -b -1
It looks like other manifestation of https://bugzilla.redhat.com/show_bug.cgi?id=1320771 I believe. Some resources seem stuck IN_PROGRESS, and the subsequent update doesn't manage to fix them.
Marking as a duplicate so we can keep track of this in one place. *** This bug has been marked as a duplicate of bug 1320771 ***