Created attachment 1119232 [details] Undercloud description of error OSP + RHEV + CFME deployment stops in the middle w/ ControllerNodePostDeployment failure. Current workaround is to undeploy from Undercloud UI and resume the OSP deployment task in Satellite UI. Logging from heat-api.log regarding the error: 2016-01-27 19:31:59.174 18689 INFO eventlet.wsgi.server [req-40210273-fab2-4e9a-aa7e-29b601069be8 admin admin] 192.0.2.1 - - [27/Jan/2016 19:31:59] "GET /v1/369b087f32454cfc890a36a604b6bdd2/stacks HTTP/1.1" 200 916 0.143380 2016-01-28 08:34:48.045 18689 INFO eventlet.wsgi.server [-] (18689) accepted ('192.0.2.254', 60252) 2016-01-28 08:34:48.047 18689 DEBUG heat.api.middleware.version_negotiation [-] Processing request: GET /v1/369b087f32454cfc890a36a604b6bdd2/stacks Accept: application/json process_request /usr/lib/python2.7/site-packages/heat/api/middleware/version_negotiation.py:50 2016-01-28 08:34:48.047 18689 DEBUG heat.api.middleware.version_negotiation [-] Matched versioned URI. Version: 1.0 process_request /usr/lib/python2.7/site-packages/heat/api/middleware/version_negotiation.py:65 2016-01-28 08:34:48.048 18689 DEBUG keystoneclient.auth.identity.v2 [-] Making authentication request to http://192.0.2.254:35357/v2.0/tokens get_auth_ref /usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v2.py:76 2016-01-28 08:34:48.143 18689 DEBUG keystoneclient.session [-] REQ: curl -g -i -X GET http://192.0.2.254:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}17dae1b61b7814e6a1e53340cbe803e24253c85f" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}d47835b90aa754e0b0ab2ef7ec6141685d64de11" _http_log_request /usr/lib/python2.7/site-packages/keystoneclient/session.py:195 2016-01-28 08:34:48.185 18689 DEBUG keystoneclient.session [-] RESP: [200] content-length: 6435 x-subject-token: {SHA1}17dae1b61b7814e6a1e53340cbe803e24253c85f vary: X-Auth-Token connection: keep-alive date: Thu, 28 Jan 2016 13:34:48 GMT content-type: application/json x-openstack-request-id: req-f0561ed2-8b03-4809-90b9-6e370042b065 RESP BODY: {"token": {"methods": ["password", "token"], "roles": [{"id": "0472321ad9e244ef837e75c9af853888", "name": "admin"}, {"id": "9fe2ff9ee4384b1894a90878d3e92bab", "name": "_member_"}], "expires_at": "2016-01-28T17:34:45.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "369b087f32454cfc890a36a604b6bdd2", "name": "admin"}, "catalog": "<removed>", "extras": {}, "user": {"domain": {"id": "default", "name": "Default"}, "id": "1660416dcad649e5b908789bde0c084c", "name": "admin"}, "audit_ids": ["e4UWZ5e6THiVHuRSdqT88w", "W0lKQintSF6IxIo7U7goag"], "issued_at": "2016-01-28T13:34:45.174348"}} _http_log_response /usr/lib/python2.7/site-packages/keystoneclient/session.py:224 2016-01-28 08:34:48.188 18689 DEBUG heat.openstack.common.policy [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] Rules successfully reloaded _load_policy_file /usr/lib/python2.7/site-packages/heat/openstack/common/policy.py:295 2016-01-28 08:34:48.188 18689 INFO heat.openstack.common.policy [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] Can not find policy directory: policy.d 2016-01-28 08:34:48.189 18689 DEBUG heat.common.wsgi [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] Calling <heat.api.openstack.v1.stacks.StackController object at 0x30f36d0> : index __call__ /usr/lib/python2.7/site-packages/heat/common/wsgi.py:647 2016-01-28 08:34:48.189 18689 INFO heat.openstack.common.policy [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] Can not find policy directory: policy.d 2016-01-28 08:34:48.190 18689 DEBUG oslo_messaging._drivers.amqpdriver [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] MSG_ID is a330fde8de244e19b5658bdfb0ac56ab _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:311 2016-01-28 08:34:48.191 18689 DEBUG oslo_messaging._drivers.amqp [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] UNIQUE_ID is a2ad677fd5cf46748e245c7e74d09da7. _add_unique_id /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258 2016-01-28 08:34:48.251 18689 DEBUG heat.common.serializers [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] JSON response : {"stacks": [{"parent": null, "description": "No description", "links": [{"href": "http://192.0.2.254:8004/v1/369b087f32454cfc890a36a604b6bdd2/stacks/overcloud/da4ea309-d27b-46cf-9126-aed91b1fac99", "rel": "self"}], "stack_status_reason": "Resource CREATE failed: Error: resources.ControllerNodesPostDeployment.resources.ControllerServicesBaseDeployment_Step2.resources[0]: Deployment to server failed: deploy_status_code: Deployment exited with non-zero status code: 6", "stack_name": "overcloud", "stack_user_project_id": "618ccf0bc8974e73b91e451e3a915602", "creation_time": "2016-01-28T00:12:43Z", "updated_time": null, "stack_owner": "admin", "stack_status": "CREATE_FAILED", "id": "da4ea309-d27b-46cf-9126-aed91b1fac99"}]} to_json /usr/lib/python2.7/site-packages/heat/common/serializers.py:42 2016-01-28 08:34:48.252 18689 INFO eventlet.wsgi.server [req-37124495-f1ba-468b-b81a-56462b00caa3 - admin] 192.0.2.254 - - [28/Jan/2016 08:34:48] "GET /v1/369b087f32454cfc890a36a604b6bdd2/stacks?sort_dir=desc&sort_key=created_at&limit=1000 HTTP/1.1" 200 940 0.206061 2016-01-28 08:34:48.255 18689 INFO eventlet.wsgi.server [-] (18689) accepted ('192.0.2.254', 60255)
This is a puppet error, please debug it using the API. Some hints here: http://hardysteven.blogspot.co.uk/2015/04/debugging-tripleo-heat-templates.html
We can certainly do that. Is there a reason that when these errors occur and simply undeploying and restarting the deployment is successful though?
This bug did not make the OSP 8.0 release. It is being deferred to OSP 10.
This is still awaiting needinfo, I don't think this will make it for 10. Additionally as this is a puppet error, I'm not sure it belongs in DFG:UI?
I don't think this BZ is related to Portfolio Integration, but don't know who contact with QCI. Is this BZ still an issue?
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days