Bug 1227421
| Summary: | Overcloud deployment failed - "Resource CREATE failed - status code: 1 - Error running /var/lib/heat-config/heat-config-puppet | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Ola Pavlenko <opavlenk> | ||||||||||
| Component: | rhosp-director | Assignee: | Giulio Fidente <gfidente> | ||||||||||
| Status: | CLOSED ERRATA | QA Contact: | Ola Pavlenko <opavlenk> | ||||||||||
| Severity: | urgent | Docs Contact: | |||||||||||
| Priority: | low | ||||||||||||
| Version: | 7.0 (Kilo) | CC: | dmacpher, gfidente, mandreou, mburns, rhel-osp-director-maint, rrosa, yeylon | ||||||||||
| Target Milestone: | ga | Keywords: | TestOnly, Triaged | ||||||||||
| Target Release: | Director | ||||||||||||
| Hardware: | Unspecified | ||||||||||||
| OS: | Linux | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
| Doc Text: |
Upstream Nova defaults to appending "novalocal" to host names. This causes problems with our tooling. The resulting nova host names don't correspond to the names expected by Puppet. This caused CREATE_FAILED on ControllerNodesPostDeployment in the Heat stack during Overcloud creation. This fix overrides the upstream default and does not set the "novalocal" suffix. Now post-deployment Puppet configuration reports a CREATE_COMPLETE on the Overcloud Heat stack.
|
Story Points: | --- | ||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2015-08-05 13:52:25 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Attachments: |
|
||||||||||||
I've reinstalled my env with new puddle (from June 2nd) and it reproduces.
env: root.lab.tlv.redhat.com
instack VM: 192.168.122.236
(Leaving the env as is until tomorrow in case anyone would like to take a look)
Versions:
$ rpm -qa | grep rhos
rhos-release-0.62-1.noarch
$ rpm -qa | grep triple
openstack-tripleo-0.0.6-dev1717.el7.centos.noarch
tripleo-common-0.0.1-dev5.el7.centos.noarch
openstack-tripleo-heat-templates-0.8.6-dev15.el7.centos.noarch
openstack-tripleo-image-elements-0.9.6-dev4.el7.centos.noarch
openstack-tripleo-puppet-elements-0.0.1.dev55-2.el7ost.noarch
$ rpm -qa | grep rdo
rdo-release-kilo-1.noarch
rubygem-rdoc-4.0.0-24.el7.noarch
python-rdomanager-oscplugin-0.0.7-dev45.el7.centos.noarch
$ rpm -qa | grep puppet
openstack-puppet-modules-2015.1.2-1.el7ost.noarch
openstack-tripleo-puppet-elements-0.0.1.dev55-2.el7ost.noarch
puppet-3.6.2-3.el7.noarch
$ rpm -qa | grep heat
openstack-heat-engine-2015.1.1-dev11.el7.centos.noarch
openstack-heat-api-cfn-2015.1.1-dev11.el7.centos.noarch
openstack-tripleo-heat-templates-0.8.6-dev15.el7.centos.noarch
openstack-heat-api-2015.1.1-dev11.el7.centos.noarch
heat-cfntools-1.2.8-2.el7.noarch
openstack-heat-common-2015.1.1-dev11.el7.centos.noarch
openstack-heat-templates-0.0.0-post350.el7.centos.noarch
python-heatclient-0.5.0-1.el7ost.noarch
openstack-heat-api-cloudwatch-2015.1.1-dev11.el7.centos.noarch
[stack@instack ~]$ openstack overcloud deploy --plan-uuid aa1102b1-899a-47ef-9b2e-c2db0e6b2067 --debug &> deploy.log
please see attached deploy.log for the output of this command.
[stack@instack ~]$ heat stack-list
+--------------------------------------+------------+---------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+---------------+----------------------+
| ffc5245e-96ce-4464-a763-fd215645ad11 | overcloud | CREATE_FAILED | 2015-06-03T16:14:15Z |
+--------------------------------------+------------+---------------+----------------------+
[stack@instack ~]$ heat resource-list overcloud
+-----------------------------------+--------------------------------------------------+---------------------------------------------------+-----------------+----------------------+
| resource_name | physical_resource_id | resource_type | resource_status | updated_time |
+-----------------------------------+--------------------------------------------------+---------------------------------------------------+-----------------+----------------------+
| BlockStorageAllNodesDeployment | ab1b8e85-917b-4097-8189-2ee34e227ced | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| BlockStorageNodesPostDeployment | | OS::TripleO::BlockStoragePostDeployment | INIT_COMPLETE | 2015-06-03T16:14:15Z |
| Ceph-Storage | 95eb67c0-b7d4-4de6-aa14-7de3249a42a6 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| CephClusterConfig | 89d68382-61e1-4fcb-b830-acd8203845d2 | OS::TripleO::CephClusterConfig::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| CephStorageAllNodesDeployment | f8214e8e-db4a-45c1-8349-e8ad561de8b1 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| CephStorageCephDeployment | 4c07364c-556f-4338-8f0e-60b8f4316874 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| CephStorageNodesPostDeployment | | OS::TripleO::CephStoragePostDeployment | INIT_COMPLETE | 2015-06-03T16:14:15Z |
| Cinder-Storage | f33e3e2d-333c-4510-8d08-37f8dbd49794 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| Compute | 6d4e7117-75a7-4386-ac52-cd96f70da85b | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ComputeAllNodesDeployment | 77f50677-c973-423c-b98b-31d5eeebb671 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ComputeCephDeployment | 479454ea-f663-4892-809f-4da9d6305a5a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ComputeNodesPostDeployment | d82cc4d7-998a-4bb3-bac9-2ae145f29bbc | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2015-06-03T16:14:15Z |
| ControlVirtualIP | 0601e644-eea2-4158-af31-5f78a8e5c244 | OS::Neutron::Port | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| Controller | d8f637b8-6c11-4f6b-8603-351ea4690d75 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerAllNodesDeployment | 75ff4376-16ba-4f2a-b28a-1239cd79c388 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerBootstrapNodeConfig | 3113766c-5898-4bc5-b4ae-ef13b777af54 | OS::TripleO::BootstrapNode::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerBootstrapNodeDeployment | edf8b092-8173-4260-b8f5-935aba56847b | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerCephDeployment | bd0c1550-b60f-4c1f-b58f-0c7fc9cc49f8 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerClusterConfig | 7890ba7c-30f3-46a4-bbbf-07ac1cea1212 | OS::Heat::StructuredConfig | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerClusterDeployment | dc629bed-d0a5-483b-8980-d94c62af1f61 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ControllerNodesPostDeployment | b179f9ff-fc30-41ab-aca4-6347478e2d86 | OS::TripleO::ControllerPostDeployment | CREATE_FAILED | 2015-06-03T16:14:15Z |
| ControllerSwiftDeployment | 1622999d-0903-41ba-885b-40328b4983ef | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| HeatAuthEncryptionKey | overcloud-HeatAuthEncryptionKey-coede6t72kyt | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| HorizonSecret | overcloud-HorizonSecret-45ixallq4ujg | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| MysqlClusterUniquePart | overcloud-MysqlClusterUniquePart-fefkvsaggdzt | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| MysqlClustercheckPassword | overcloud-MysqlClustercheckPassword-kntgyt7xmgph | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| MysqlRootPassword | overcloud-MysqlRootPassword-qn22slimc6dv | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ObjectStorageAllNodesDeployment | 451a174c-a65a-4af0-84da-8a193ad0ee46 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ObjectStorageNodesPostDeployment | dd0c999e-4c2a-4695-b8a0-57aef5a82cad | OS::TripleO::ObjectStoragePostDeployment | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| ObjectStorageSwiftDeployment | 9a6b2a7a-48bf-4ecb-ac9d-b7451a672cd9 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| PcsdPassword | overcloud-PcsdPassword-nqwjff5crppb | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| PublicVirtualIP | d522b729-802e-4160-a9bf-40aa27739ff7 | OS::Neutron::Port | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| RabbitCookie | overcloud-RabbitCookie-am6i4h5krmf4 | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| Swift-Storage | fb9ba73d-9926-46a7-a99e-92227d7de346 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| SwiftDevicesAndProxyConfig | 21a41c14-2aeb-4ebb-a448-94978faff0f7 | OS::TripleO::SwiftDevicesAndProxy::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
| allNodesConfig | 948a52dc-8a84-4543-8f87-20aafbc5895f | OS::TripleO::AllNodes::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:14:15Z |
+-----------------------------------+--------------------------------------------------+---------------------------------------------------+-----------------+----------------------+
[stack@instack ~]$ heat resource-show overcloud ComputeNodesPostDeployment
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes | {} |
| description | |
| links | http://192.0.2.1:8004/v1/90d8558659544443bb48882548428f73/stacks/overcloud/ffc5245e-96ce-4464-a763-fd215645ad11/resources/ComputeNodesPostDeployment (self) |
| | http://192.0.2.1:8004/v1/90d8558659544443bb48882548428f73/stacks/overcloud/ffc5245e-96ce-4464-a763-fd215645ad11 (stack) |
| | http://192.0.2.1:8004/v1/90d8558659544443bb48882548428f73/stacks/overcloud-ComputeNodesPostDeployment-puxhkne3t7lu/d82cc4d7-998a-4bb3-bac9-2ae145f29bbc (nested) |
| logical_resource_id | ComputeNodesPostDeployment |
| physical_resource_id | d82cc4d7-998a-4bb3-bac9-2ae145f29bbc |
| required_by | |
| resource_name | ComputeNodesPostDeployment |
| resource_status | CREATE_FAILED |
| resource_status_reason | ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: Error: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1"" |
| resource_type | OS::TripleO::ComputePostDeployment |
| updated_time | 2015-06-03T16:14:15Z |
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[stack@instack ~]$ nova list
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
| 45259403-2791-4f0e-98e5-4dafafbd90bf | ov-mwlmrezfff-0-bjbogr7d2lzk-NovaCompute-fqwa6wunhveh | ACTIVE | - | Running | ctlplane=192.0.2.13 |
| fa229586-5f7e-4c6a-a5fa-202bf89142ff | ov-ofjbpdc6ygf-0-jqb2csqzdo2z-Controller-zjkisvgzhmos | ACTIVE | - | Running | ctlplane=192.0.2.12 |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
[stack@instack ~]$ ssh heat-admin.2.13
The authenticity of host '192.0.2.13 (192.0.2.13)' can't be established.
ECDSA key fingerprint is 16:e2:ce:a9:59:5a:0f:68:a2:5a:16:74:0d:f1:ca:85.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.0.2.13' (ECDSA) to the list of known hosts.
[heat-admin@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]$ sudo -i
[root@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]# less /var/log/messages
[root@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]# date
Wed Jun 3 12:23:11 EDT 2015
[root@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]# less /var/log/messages
[root@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]# grep ERROR /var/log/messages
Jun 3 12:17:13 localhost os-collect-config: [2015-06-03 12:17:13,899] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/88dc8a53-1c08-4fb2-a44d-1e4b42f27674.pp. [1]
[root@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]# exit
logout
[heat-admin@ov-mwlmrezfff-0-bjbogr7d2lzk-novacompute-fqwa6wunhveh ~]$ exit
logout
Connection to 192.0.2.13 closed.
[stack@instack ~]$ heat stack-delete overcloud
+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| ffc5245e-96ce-4464-a763-fd215645ad11 | overcloud | DELETE_IN_PROGRESS | 2015-06-03T16:14:15Z |
+--------------------------------------+------------+--------------------+----------------------+
[stack@instack ~]$ heat stack-list
+--------------------------------------+------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+--------------------+----------------------+
| ffc5245e-96ce-4464-a763-fd215645ad11 | overcloud | DELETE_IN_PROGRESS | 2015-06-03T16:14:15Z |
+--------------------------------------+------------+--------------------+----------------------+
[stack@instack ~]$ sudo vi /etc/sysconfig/iptables
[stack@instack ~]$ heat stack-list
+----+------------+--------------+---------------+
| id | stack_name | stack_status | creation_time |
+----+------------+--------------+---------------+
+----+------------+--------------+---------------+
[stack@instack ~]$ openstack overcloud deploy --plan-uuid aa1102b1-899a-47ef-9b2e-c2db0e6b2067 --debug &> deploy.log
[stack@instack ~]$ heat stack-list
+--------------------------------------+------------+---------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+------------+---------------+----------------------+
| a7ec7ffb-18dd-4c9d-8152-2a482add31ad | overcloud | CREATE_FAILED | 2015-06-03T16:26:55Z |
+--------------------------------------+------------+---------------+----------------------+
[stack@instack ~]$ heat resource-list overcloud
+-----------------------------------+--------------------------------------------------+---------------------------------------------------+-----------------+----------------------+
| resource_name | physical_resource_id | resource_type | resource_status | updated_time |
+-----------------------------------+--------------------------------------------------+---------------------------------------------------+-----------------+----------------------+
| BlockStorageAllNodesDeployment | 49023b2d-19f9-44ed-8d74-8a3c9af61882 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| BlockStorageNodesPostDeployment | | OS::TripleO::BlockStoragePostDeployment | INIT_COMPLETE | 2015-06-03T16:26:55Z |
| Ceph-Storage | 5713ee9e-72cb-4f83-bcd1-132c27caf505 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| CephClusterConfig | f06e0133-7636-488f-a85e-30bd980cd722 | OS::TripleO::CephClusterConfig::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| CephStorageAllNodesDeployment | e720698f-7b85-4663-9e68-d413d3310e2b | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| CephStorageCephDeployment | 43ec507e-8c97-4de1-966e-f8a00318c273 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| CephStorageNodesPostDeployment | | OS::TripleO::CephStoragePostDeployment | INIT_COMPLETE | 2015-06-03T16:26:55Z |
| Cinder-Storage | e2edf784-88ca-45c5-8a5a-48d486bdd2c4 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| Compute | 3b965d6e-4746-4381-827b-ca55aad16776 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ComputeAllNodesDeployment | 845ffa9b-f525-4cb2-a791-cdaab41a5aec | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ComputeCephDeployment | 6437085f-ffb5-49ca-a369-2ec57e968f4a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ComputeNodesPostDeployment | e8afb777-c8e6-44bc-864e-8e5f2d2b36f2 | OS::TripleO::ComputePostDeployment | CREATE_FAILED | 2015-06-03T16:26:55Z |
| ControlVirtualIP | 8593a9d7-a876-42e7-b14e-0c5b6353f2d9 | OS::Neutron::Port | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| Controller | 2daf6c93-690b-4716-b663-9c30d65a8422 | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerAllNodesDeployment | ead221c0-7c8b-43ab-8c14-a9575e122c5a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerBootstrapNodeConfig | 9978ee63-001f-4bce-a51c-004413b7e4b3 | OS::TripleO::BootstrapNode::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerBootstrapNodeDeployment | 4b304d45-5be8-4eef-a577-0fe7256f45b3 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerCephDeployment | fd2db166-03fe-4567-b9e1-1d3044ebc426 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerClusterConfig | 04dfb51e-c370-45a9-b1cc-63a5b17a793d | OS::Heat::StructuredConfig | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerClusterDeployment | d68c5956-0d1f-46ca-bea3-f0b535d63f5a | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ControllerNodesPostDeployment | 3b94204d-98d2-40a6-be8e-88ae4026389f | OS::TripleO::ControllerPostDeployment | CREATE_FAILED | 2015-06-03T16:26:55Z |
| ControllerSwiftDeployment | fd693945-0594-4cf5-ba89-cf5464c07419 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| HeatAuthEncryptionKey | overcloud-HeatAuthEncryptionKey-tyg36x7s2ald | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| HorizonSecret | overcloud-HorizonSecret-4mkspk35fnld | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| MysqlClusterUniquePart | overcloud-MysqlClusterUniquePart-3eikkgixv7hz | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| MysqlClustercheckPassword | overcloud-MysqlClustercheckPassword-v6sbdodmm5zx | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| MysqlRootPassword | overcloud-MysqlRootPassword-hmsoymzpkihg | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ObjectStorageAllNodesDeployment | 13f5677f-6459-4d49-b3cf-23e21a1c45c6 | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ObjectStorageNodesPostDeployment | bb0dc7c7-74fb-4833-a157-bb735e5b2ab7 | OS::TripleO::ObjectStoragePostDeployment | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| ObjectStorageSwiftDeployment | 7b30028b-08db-4e9f-90f0-1404f2c9f09f | OS::Heat::StructuredDeployments | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| PcsdPassword | overcloud-PcsdPassword-ujnxiwp53c5n | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| PublicVirtualIP | b1564768-1550-4d02-bebc-96f6fe9c009f | OS::Neutron::Port | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| RabbitCookie | overcloud-RabbitCookie-zzamuibhrz6l | OS::Heat::RandomString | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| Swift-Storage | 2eb72681-5c1d-486b-9344-d6573256441e | OS::Heat::ResourceGroup | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| SwiftDevicesAndProxyConfig | c96326e8-34f3-4cbc-aca2-fa98356e2098 | OS::TripleO::SwiftDevicesAndProxy::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
| allNodesConfig | f48c0f10-55b0-4da4-ae7b-67bef067e8a3 | OS::TripleO::AllNodes::SoftwareConfig | CREATE_COMPLETE | 2015-06-03T16:26:55Z |
+-----------------------------------+--------------------------------------------------+---------------------------------------------------+-----------------+----------------------+
[stack@instack ~]$ heat resource-show overcloud ComputeNodesPostDeployment
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Property | Value |
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| attributes | {} |
| description | |
| links | http://192.0.2.1:8004/v1/90d8558659544443bb48882548428f73/stacks/overcloud/a7ec7ffb-18dd-4c9d-8152-2a482add31ad/resources/ComputeNodesPostDeployment (self) |
| | http://192.0.2.1:8004/v1/90d8558659544443bb48882548428f73/stacks/overcloud/a7ec7ffb-18dd-4c9d-8152-2a482add31ad (stack) |
| | http://192.0.2.1:8004/v1/90d8558659544443bb48882548428f73/stacks/overcloud-ComputeNodesPostDeployment-m54udzg5bj3y/e8afb777-c8e6-44bc-864e-8e5f2d2b36f2 (nested) |
| logical_resource_id | ComputeNodesPostDeployment |
| physical_resource_id | e8afb777-c8e6-44bc-864e-8e5f2d2b36f2 |
| required_by | |
| resource_name | ComputeNodesPostDeployment |
| resource_status | CREATE_FAILED |
| resource_status_reason | ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: Error: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1"" |
| resource_type | OS::TripleO::ComputePostDeployment |
| updated_time | 2015-06-03T16:26:55Z |
+------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
[stack@instack ~]$ nova list
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
| 6e90957a-e677-459b-bb7a-a7cdc93d6c27 | ov-7djrh43hma3-0-eqypolxayezy-Controller-cam4hpvy5ptg | ACTIVE | - | Running | ctlplane=192.0.2.16 |
| 3a1ecb67-4d9a-4e7f-af15-5f36479e276e | ov-ypxvgvmpml-0-4vs2gu3jciuw-NovaCompute-jk2u6hpmdxpn | ACTIVE | - | Running | ctlplane=192.0.2.17 |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
[stack@instack ~]$ ssh heat-admin.2.16
The authenticity of host '192.0.2.16 (192.0.2.16)' can't be established.
ECDSA key fingerprint is db:26:84:e3:c9:44:07:27:f8:32:73:9d:30:0b:e6:49.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.0.2.16' (ECDSA) to the list of known hosts.
[heat-admin@ov-7djrh43hma3-0-eqypolxayezy-controller-cam4hpvy5ptg ~]$ sudo -i
[root@ov-7djrh43hma3-0-eqypolxayezy-controller-cam4hpvy5ptg ~]# grep ERROR /var/log/messages
Jun 3 12:30:07 localhost os-collect-config: [2015-06-03 12:30:07,356] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/225756ee-1bd6-42c1-bc5b-a9c6fa6f0db1.pp. [1]
And same for compute node: $ nova list
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
| 6e90957a-e677-459b-bb7a-a7cdc93d6c27 | ov-7djrh43hma3-0-eqypolxayezy-Controller-cam4hpvy5ptg | ACTIVE | - | Running | ctlplane=192.0.2.16 |
| 3a1ecb67-4d9a-4e7f-af15-5f36479e276e | ov-ypxvgvmpml-0-4vs2gu3jciuw-NovaCompute-jk2u6hpmdxpn | ACTIVE | - | Running | ctlplane=192.0.2.17 |
+--------------------------------------+-------------------------------------------------------+--------+------------+-------------+---------------------+
[stack@instack ~]$ ssh heat-admin.2.17
The authenticity of host '192.0.2.17 (192.0.2.17)' can't be established.
ECDSA key fingerprint is e4:bf:4a:f4:96:ed:a8:32:d1:ba:46:0e:38:9f:5e:91.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.0.2.17' (ECDSA) to the list of known hosts.
[heat-admin@ov-ypxvgvmpml-0-4vs2gu3jciuw-novacompute-jk2u6hpmdxpn ~]$ sudo -i
[root@ov-ypxvgvmpml-0-4vs2gu3jciuw-novacompute-jk2u6hpmdxpn ~]# grep ERROR /var/log/messages
Jun 3 12:29:53 localhost os-collect-config: [2015-06-03 12:29:53,805] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-puppet/499be8d5-0410-4f45-9c48-8c9aa397ef66.pp. [1]
/var/log/messages from both nodes are attached
Created attachment 1034411 [details]
compute_messages.log
Created attachment 1034412 [details]
control_messages.log
Created attachment 1034413 [details]
deploy.log
Used https://repos.fedorapeople.org/repos/openstack-m/docs/internal/master/environments/virtual.html but with openstack client commands (e.g unified cli) from controller logs I see: Could not retrieve facts for ov-7djrh43hma3-0-eqypolxayezy-controller-cam4hpvy5ptg.novalocal which seems correct, .novalocal suffix should not be there because of: https://review.openstack.org/#/c/178108/ https://review.openstack.org/#/c/178110/ https://review.openstack.org/#/c/178111/ During the installation of the undercloud I'v enabled RDO, epel, and RDO-Manager... could be the reason? Should be fixed by:https://review.gerrithub.io/#/c/235162 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2015:1549 |
Created attachment 1033860 [details] deploy_new.log Description of problem: prepared a plan for deployment using unified cli (openstack client) tried to deploy the overcloud using the openstack overcloud deploy --plan-uuid $UUID command and got the following error: Resource failed - Unknown status FAILED due to "Resource CREATE failed: ResourceUnknownStatus: Resource failed - Unknown status FAILED due to "Resource CREATE failed: Error: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1"" Version-Release number of selected component (if applicable): $ rpm -qa | grep rdo rdo-release-kilo-1.noarch rubygem-rdoc-4.0.0-24.el7.noarch python-rdomanager-oscplugin-0.0.7-dev45.el7.centos.noarch [stack@instack ~]$ rpm -qa | grep openstack| grep heat openstack-heat-engine-2015.1.1-dev11.el7.centos.noarch openstack-heat-api-cfn-2015.1.1-dev11.el7.centos.noarch openstack-tripleo-heat-templates-0.8.6-dev15.el7.centos.noarch openstack-heat-api-2015.1.1-dev11.el7.centos.noarch openstack-heat-common-2015.1.1-dev11.el7.centos.noarch openstack-heat-templates-0.0.0-post350.el7.centos.noarch openstack-heat-api-cloudwatch-2015.1.1-dev11.el7.centos.noarch How reproducible: 100% Steps to Reproduce: 1.prepare a plan using unified cli 2.deploy a plan using unified cli Actual results: deployment failed Expected results: overcloud was successfully deployed Additional info: please see attached deploy_new.log with all details.