Bug 1262124

Summary:	Deployment failure - resources.ControllerOvercloudServicesDeployment_Step5
Product:	Red Hat OpenStack	Reporter:	Ola Pavlenko <opavlenk>
Component:	openstack-tuskar-ui	Assignee:	Florian Fuchs <flfuchs>
Status:	CLOSED DUPLICATE	QA Contact:	yeylon <yeylon>
Severity:	urgent	Docs Contact:
Priority:	high
Version:	unspecified	CC:	akrivoka, calfonso, mburns, opavlenk, rhel-osp-director-maint, srevivo, vincent
Target Milestone:	y2	Keywords:	Triaged
Target Release:	7.0 (Kilo)	Flags:	opavlenk: needinfo+
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-09-23 16:09:07 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1250250

Description Ola Pavlenko 2015-09-10 20:49:16 UTC

Description of problem:
-----------------------------------
Tried to deploy HA+ceph via UI and failed with 2015-09-10 10:39:00.825 29559 INFO heat.engine.resource [-] CREATE: TemplateResource "ControllerNodesPostDeployment" [2ee05625-572f-4cc4-bc0c-59586555b482] Stack "overcloud" [09061553-c323-4151-acb4-68bf5086d749]
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource Traceback (most recent call last):
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 508, in _action_recorder
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource     yield
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 578, in _do_action
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 313, in wrapper
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource     step = next(subtask)
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 552, in action_handler_task
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource     while not check(handler_data):
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 288, in check_create_complete
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource     return self._check_status_complete(resource.Resource.CREATE)
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 329, in _check_status_complete
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource     action=action)
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource ResourceFailure: resources.ControllerOvercloudServicesDeployment_Step5: resources.ControllerNodesPostDeployment.Property error: resources[1].properties.server: Expecting to find username or userId in passwordCredentials - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400)
2015-09-10 10:39:00.825 29559 TRACE heat.engine.resource 
2015-09-10 10:39:00.884 29559 DEBUG oslo_messaging._drivers.amqp [req-d87f6243-ba09-43e7-baf0-de2d03c1864c - admin] UNIQUE_ID is 54c295dd8ccb449ab48b703cde9172ba. _add_unique_id /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258
2015-09-10 10:39:00.886 29559 DEBUG oslo_messaging._drivers.amqp [req-3dc32cf1-17f9-4740-acea-2e819ebec26e admin admin] UNIQUE_ID is 7a91195a2fe449ab986592861a9c05ee. _add_unique_id /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqp.py:258


Version-Release number of selected component (if applicable):
-----------------------------------
poodle from Sep 08
openstack-tuskar-ui-0.4.0-3.el7ost.noarch
python-tuskarclient-0.1.18-4.el7ost.noarch
openstack-tuskar-0.4.18-4.el7ost.noarch
openstack-tuskar-ui-extras-0.0.4-1.el7ost.noarch


How reproducible:
-----------------------------------

Steps to Reproduce:
-----------------------------------
1.Access Tuskar UI and change Project to admin.
2. change snmp password to 'password'
3.check Service net map value for correctness in all resources(to be in json format)
4.attempt to deploy 3 controllers, 1 compute and 1 ceph via UI

Actual results:
-----------------------------------
failed with 
Expecting to find username or userId in passwordCredentials - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400)

Expected results:
-----------------------------------
successfully deployed overcloud

Additional info:
-----------------------------------
$ heat resource-list -n 5 overcloud  | grep CREATE_FAILED
| ComputeNodesPostDeployment                  | 97f9e821-da0c-4fb8-8466-9b6cdeaf0839          | OS::TripleO::ComputePostDeployment                | CREATE_FAILED   | 2015-09-10T13:45:09Z |                                             |
| ControllerNodesPostDeployment               | 2ee05625-572f-4cc4-bc0c-59586555b482          | OS::TripleO::ControllerPostDeployment             | CREATE_FAILED   | 2015-09-10T13:45:09Z |                                             |
| ControllerOvercloudServicesDeployment_Step5 |                                               | OS::Heat::StructuredDeployments                   | CREATE_FAILED   | 2015-09-10T14:21:08Z | ControllerNodesPostDeployment               |
| ComputePuppetDeployment                     | 54418c08-42c5-4afe-92ff-34f5052a7c8d          | OS::Heat::StructuredDeployments                   | CREATE_FAILED   | 2015-09-10T14:22:37Z | ComputeNodesPostDeployment                  |
| 0                                           | 041656aa-129c-4252-b570-4bef378e6295          | OS::Heat::StructuredDeployment                    | CREATE_FAILED   | 2015-09-10T14:23:35Z | ComputePuppetDeployment

Comment 4 chris alfonso 2015-09-14 16:19:46 UTC

Is the result the same with a cli based tuskar deployment?

Comment 6 Ola Pavlenko 2015-09-22 15:26:48 UTC

deployment via CLI with tuskar plan succeed with 3 controllers + 1 compute + 1 ceph.

Comment 7 Ola Pavlenko 2015-09-22 15:59:24 UTC

Happened to me just now but for a different resource ControllerServicesBaseDeployment_Step2

var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource Traceback (most recent call last):
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 508, in _action_recorder
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource     yield
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 578, in _do_action
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource     yield self.action_handler_task(action, args=handler_args)
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/scheduler.py", line 313, in wrapper
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource     step = next(subtask)
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 552, in action_handler_task
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource     while not check(handler_data):
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 288, in check_create_complete
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource     return self._check_status_complete(resource.Resource.CREATE)
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource   File "/usr/lib/python2.7/site-packages/heat/engine/resources/stack_resource.py", line 329, in _check_status_complete
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource     action=action)
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource ResourceFailure: resources.ControllerServicesBaseDeployment_Step2: resources.ControllerNodesPostDeployment.Property error: resources[0].properties.server: Expecting to find username or userId in passwordCredentials - the server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. (HTTP 400)
/var/log/heat/heat-engine.log:2015-09-22 11:20:23.012 3683 TRACE heat.engine.resource

Comment 9 Ola Pavlenko 2015-09-22 16:15:53 UTC

python-tuskarclient-0.1.18-4.el7ost.noarch
openstack-tuskar-0.4.18-4.el7ost.noarch
openstack-tuskar-ui-extras-0.0.4-1.el7ost.noarch
openstack-tuskar-ui-0.4.0-3.el7ost.noarch

Comment 11 Ana Krivokapic 2015-09-23 16:09:07 UTC


*** This bug has been marked as a duplicate of bug 1263651 ***