Bug 1227050 - rhel-osp-director: The overcloud deployment with COMPUTESCALE=2 reported failure.
Summary: rhel-osp-director: The overcloud deployment with COMPUTESCALE=2 reported fail...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director
Version: unspecified
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ga
: Director
Assignee: Mike Burns
QA Contact: Alexander Chuzhoy
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-01 20:11 UTC by Alexander Chuzhoy
Modified: 2015-08-05 13:52 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-05 13:52:21 UTC
Target Upstream Version:


Attachments (Terms of Use)
heat logs from the undercloud machine. (3.20 MB, application/x-gzip)
2015-06-01 20:18 UTC, Alexander Chuzhoy
no flags Details
heat event-list (19.57 KB, text/plain)
2015-06-01 20:22 UTC, Alexander Chuzhoy
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2015:1549 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform director Release 2015-08-05 17:49:10 UTC

Description Alexander Chuzhoy 2015-06-01 20:11:39 UTC
rhel-osp-director: The overcloud deployment with COMPUTESCALE=2 reported failure.

Environment:
instack-0.0.6-5.el7ost.noarch
instack-undercloud-2.1.0-3.el7ost.noarch

Steps to reproduce:
Set the COMPUTESCALE=2 and run instack-deploy-overcloud --tuskar

Result:

 heat stack-list
+--------------------------------------+------------+---------------+----------------------+
| id                                   | stack_name | stack_status  | creation_time        |
+--------------------------------------+------------+---------------+----------------------+
| 1d6b1d27-5c1f-4f57-9966-be2a8063e8b3 | overcloud  | CREATE_FAILED | 2015-06-01T18:45:19Z |
+--------------------------------------+------------+---------------+----------------------+



Checking the undercloud hosts:
[stack@rhos-compute-node-13 ~]$ nova list
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+-----------------------+
| ID                                   | Name                                                  | Status | Task State | Power State | Networks              |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+-----------------------+
| 4151f7ae-b1bf-480c-9654-6f5bc5e21057 | ov-4rin4xylpy-0-dhibswpvmd6p-NovaCompute-ajei3xglcz5a | ACTIVE | -          | Running     | ctlplane=192.168.0.10 |
| 287745fe-c09c-49f9-adac-27a4b464c1d9 | ov-4rin4xylpy-1-zvus5eq5qyyl-NovaCompute-6mn56heamvsy | BUILD  | spawning   | NOSTATE     | ctlplane=192.168.0.11 |
| 0236351f-b2ae-456a-9b79-4b2ea8d43f1a | ov-4tpox3bi6nr-0-zlpntppioqne-Controller-v6rg4gvzgk5n | ACTIVE | -          | Running     | ctlplane=192.168.0.9  |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+-----------------------+


The host with "NOSTATE" is actually up and I was able to ssh into it with heat-admin.

expected result:
the overcloud deployment should complete successfully.

Comment 3 Alexander Chuzhoy 2015-06-01 20:18:45 UTC
Created attachment 1033481 [details]
heat logs from the undercloud machine.

Comment 4 Alexander Chuzhoy 2015-06-01 20:19:44 UTC
heat event-list overcloud
+-----------------------------------+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+----------------------+                                                
| resource_name                     | id                                   | resource_status_reason                                                                                                                                                                                                                                          | resource_status    | event_time           |                                                
+-----------------------------------+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+----------------------+                                                
| PublicVirtualIP                   | e6e5469a-5158-4a8e-be86-762d2c9dde93 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:20Z |                                                
| Swift-Storage                     | d2d1a3ff-cbcc-497b-a803-a79877d9653d | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:20Z |                                                
| PcsdPassword                      | 51b06864-0e52-4bd4-969e-ff8ccad68568 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:21Z |                                                
| Ceph-Storage                      | 9c9dd5c3-60bd-450f-bcb4-7ea6741849af | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:21Z |                                                
| MysqlClusterUniquePart            | 713c9026-17be-48c4-b407-c8860d834240 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:21Z |                                                
| MysqlRootPassword                 | e7b4d99d-b171-4a2e-90b0-aac3bd1058b5 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:21Z |                                                
| ControlVirtualIP                  | bad00fbc-3327-4d3b-b5d0-6f5145efe661 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:21Z |                                                
| RabbitCookie                      | 17ff2041-24e5-49be-b806-70e9f5a86b3d | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:22Z |                                                
| HeatAuthEncryptionKey             | 2d933c8d-affe-4d1e-84fb-cb632b1b107d | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:23Z |                                                
| PublicVirtualIP                   | 6fc9fe68-f969-4ab8-b70d-c1aa80f249c0 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| Swift-Storage                     | dcb6bc25-fec3-4286-96fc-1ccf6380f7a7 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| ControlVirtualIP                  | d9777b85-4992-4475-9b04-2504c05acdf7 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| MysqlClusterUniquePart            | 3be566ee-c457-499b-a7c1-de57f2f54051 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| Ceph-Storage                      | 42d92dda-b3c7-4da2-baf4-349cbef15fa3 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| PcsdPassword                      | 8b210f1d-cea6-4a72-8e16-f57c6b18e033 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| MysqlRootPassword                 | 09c2e978-3d2f-4d7c-a6a4-b800a8b8270b | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| RabbitCookie                      | d19667d8-fcec-455f-924e-f99b8b1f8176 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| HeatAuthEncryptionKey             | 057ae629-5a42-4f5c-8b4d-a9307b01571a | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:23Z |                                                
| Cinder-Storage                    | 0222a9b9-0af2-47d8-9aa6-e07d376536cc | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:24Z |                                                
| Compute                           | 916ca792-73e4-4a4f-bdff-8869c27c8642 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:24Z |                                                
| Controller                        | 37d9a9f8-60ee-4cd6-8b74-39eff06e3d7a | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:45:26Z |                                                
| Cinder-Storage                    | e856993b-2cb1-4ce1-b899-dab4649c03d7 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:45:30Z |                                                
| Controller                        | 14fbab1e-f46f-46a3-b317-b2fab481be99 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:12Z |                                                
| CephClusterConfig                 | f1eb26e0-783b-43c6-9399-222c67dc86e5 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:12Z |                                                
| SwiftDevicesAndProxyConfig        | 86f3366f-6b1b-45ba-a197-24105418b30f | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:13Z |                                                
| ControllerBootstrapNodeConfig     | c7f1f8e2-4506-43b8-add1-03dfdee60d61 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:14Z |                                                
| ControllerClusterConfig           | 0114ff4f-a4fb-46e7-a953-104a12c6aa73 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:14Z |                                                
| CephClusterConfig                 | 47bfba21-7f58-474a-b12d-3bd71feae3d6 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:15Z |                                                
| ControllerClusterConfig           | 4c4207ee-2dd0-42ac-a323-7e963ee88ff3 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:15Z |                                                
| SwiftDevicesAndProxyConfig        | 8d3fb577-56d7-4217-b5bb-215d9d382c34 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:15Z |                                                
| ControllerBootstrapNodeConfig     | 62edd43c-15d7-4b79-adfe-3cafa8e3549e | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:16Z |                                                
| ObjectStorageSwiftDeployment      | 724000f2-6d4d-4112-8bf8-6856d6122715 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:16Z |                                                
| ControllerBootstrapNodeDeployment | c99bceca-4a21-4a0c-9f2a-5d9944746732 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:17Z |
| ControllerClusterDeployment       | 64df6773-36be-4508-9959-526140010dda | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:17Z |
| ControllerSwiftDeployment         | 7aa34c89-f443-4342-8661-76bdee762b95 | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:18Z |
| CephStorageCephDeployment         | 728d30f8-9669-4538-89c4-1943cd0a704e | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:19Z |
| ControllerCephDeployment          | 81b0b90a-e9a9-4422-9c29-223df30bd2cc | state changed                                                                                                                                                                                                                                                   | CREATE_IN_PROGRESS | 2015-06-01T18:50:19Z |
| ObjectStorageSwiftDeployment      | b5a8500a-da0e-42f7-937e-241ba7702771 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:21Z |
| ControllerSwiftDeployment         | 72c9dd9b-6ae3-4686-bbf8-8b4058b8480b | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:21Z |
| ControllerClusterDeployment       | e2c477a2-104d-44e0-a285-92d8ead74abb | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:21Z |
| ControllerBootstrapNodeDeployment | b18d5d8b-9704-4b27-b424-4483d41ce44a | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:21Z |
| CephStorageCephDeployment         | d962e52a-b14a-460e-8b26-191641b9fa6d | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:50:22Z |
| ControllerCephDeployment          | 3252650d-9e83-477f-9ee4-9bb1fa6be424 | state changed                                                                                                                                                                                                                                                   | CREATE_COMPLETE    | 2015-06-01T18:53:44Z |
| Compute                           | cb9b9836-ba99-43ac-bacb-e3a9f55afbd7 | ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: BadRequest: Expecting to find username or userId in passwordCrede | CREATE_FAILED      | 2015-06-01T19:45:18Z |
+-----------------------------------+--------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------+----------------------+

Comment 5 Alexander Chuzhoy 2015-06-01 20:22:39 UTC
Created attachment 1033482 [details]
heat event-list

Comment 6 Alexander Chuzhoy 2015-06-01 21:37:34 UTC
The issue reproduced with COMPUTESCALE=1.

Comment 7 chris alfonso 2015-06-02 17:46:27 UTC
Poodle is updated with backport patches that should resolve this. Please retest when you get the chance.

Comment 10 Alexander Chuzhoy 2015-06-19 15:27:25 UTC
Verified:

Environment:
instack-undercloud-2.1.2-1.el7ost.noarch



The right command now is:
openstack overcloud deploy --plan-uuid <UUID>  --compute-scale 2



output from nova list:
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
| ID                                   | Name                   | Status | Task State | Power State | Networks              |
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+
| 7c031c81-a307-4bad-bd11-5c41fec04908 | overcloud-compute-0    | ACTIVE | -          | Running     | ctlplane=192.168.0.21 |
| a5d4cd8b-2eb5-46f0-8e25-e8724a3be802 | overcloud-compute-1    | ACTIVE | -          | Running     | ctlplane=192.168.0.19 |
| d7c8f548-e8a4-46d5-b3b0-a2850204622c | overcloud-controller-0 | ACTIVE | -          | Running     | ctlplane=192.168.0.20 |
| 7253cce3-5ece-437e-9615-a771fa5e724a | overcloud-controller-1 | ACTIVE | -          | Running     | ctlplane=192.168.0.18 |
| be91bce9-56c3-47cd-95d9-53b998a81823 | overcloud-controller-2 | ACTIVE | -          | Running     | ctlplane=192.168.0.22 |
+--------------------------------------+------------------------+--------+------------+-------------+-----------------------+

Comment 12 errata-xmlrpc 2015-08-05 13:52:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2015:1549


Note You need to log in before you can comment on or make changes to this bug.