Bug 1261192 - heat stack-update failed when attempting to deploy three additional compute nodes [NEEDINFO]
heat stack-update failed when attempting to deploy three additional compute n...
Status: CLOSED INSUFFICIENT_DATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
unspecified Severity medium
: ---
: 7.0 (Kilo)
Assigned To: chris alfonso
yeylon@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-08 16:34 EDT by bigswitch
Modified: 2016-04-18 02:54 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-30 12:52:06 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
calfonso: needinfo? (rhosp-bugs-internal)


Attachments (Terms of Use)

  None (edit)
Description bigswitch 2015-09-08 16:34:58 EDT
Description of problem:
After successfully deploying three controller and two compute nodes, I attempt to install three more compute nodes. heat stack update failed with message 

"ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"Error: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1\"\"",

ironic shows the nodes are deployed, and nova list shows deployed node as active


[stack@rhosp7-undercloud ~]$ heat stack-list
+--------------------------------------+------------+---------------+----------------------+
| id                                   | stack_name | stack_status  | creation_time        |
+--------------------------------------+------------+---------------+----------------------+
| a3106bce-f5d4-409e-b2a6-1a38016587ef | overcloud  | UPDATE_FAILED | 2015-09-08T19:25:50Z |
+--------------------------------------+------------+---------------+----------------------+
[stack@rhosp7-undercloud ~]$ nova list
+--------------------------------------+------------------------+--------+------------+-------------+----------------------+
| ID                                   | Name                   | Status | Task State | Power State | Networks             |
+--------------------------------------+------------------------+--------+------------+-------------+----------------------+
| 0df1cf5c-61df-4679-b791-c1b4062fc483 | overcloud-compute-0    | ACTIVE | -          | Running     | ctlplane=192.0.2.156 |
| 41e4d6d1-05a5-4971-a394-41eb48833eb0 | overcloud-compute-1    | ACTIVE | -          | Running     | ctlplane=192.0.2.158 |
| d7113ec2-e1dc-4e85-a29c-652f631dd928 | overcloud-compute-2    | ACTIVE | -          | Running     | ctlplane=192.0.2.161 |
| 9d4e31dd-6134-464e-b247-3e6c1da72a32 | overcloud-compute-3    | ACTIVE | -          | Running     | ctlplane=192.0.2.162 |
| 98b7fde0-7f0c-4e6f-aca8-117e1cad0ff6 | overcloud-compute-4    | ACTIVE | -          | Running     | ctlplane=192.0.2.160 |
| 38dc490f-184d-4e7a-a113-4c0b96729ee5 | overcloud-controller-0 | ACTIVE | -          | Running     | ctlplane=192.0.2.157 |
| a191001c-31ae-40cf-877e-30a6c4292d62 | overcloud-controller-1 | ACTIVE | -          | Running     | ctlplane=192.0.2.155 |
| 48d9d33d-265d-4483-a576-d656b335993e | overcloud-controller-2 | ACTIVE | -          | Running     | ctlplane=192.0.2.159 |
+--------------------------------------+------------------------+--------+------------+-------------+----------------------+
[stack@rhosp7-undercloud ~]$ ironic node-list
+--------------------------------------+------+--------------------------------------+-------------+-----------------+-------------+
| UUID                                 | Name | Instance UUID                        | Power State | Provision State | Maintenance |
+--------------------------------------+------+--------------------------------------+-------------+-----------------+-------------+
| 510b4ab5-32a6-4d6f-b591-c65fbc17d62c | None | 48d9d33d-265d-4483-a576-d656b335993e | power on    | active          | False       |
| bea1652a-0be8-4282-8ac1-76f01ea87856 | None | a191001c-31ae-40cf-877e-30a6c4292d62 | power on    | active          | False       |
| 9a1dde2f-994f-40ca-8460-74b98a8bcfee | None | 38dc490f-184d-4e7a-a113-4c0b96729ee5 | power on    | active          | False       |
| 791665f4-e610-4453-8cdd-cb2fb6def37a | None | None                                 | power off   | available       | False       |
| 9cb73c6e-0a91-475c-87e9-f9a6174506db | None | None                                 | power off   | available       | False       |
| e72db0ca-a1fb-4506-a561-058286b85a22 | None | None                                 | power off   | available       | False       |
| 9ea548a0-3d95-401e-a571-198f527de340 | None | None                                 | power off   | available       | False       |
| 575cad27-74e8-45da-8510-48d44b927997 | None | None                                 | power off   | available       | False       |
| 04241fc2-84d5-4e81-87a4-e78189afc5a0 | None | None                                 | power off   | available       | False       |
| 85594939-c3e7-48c4-bcf2-63b39d0333c1 | None | None                                 | power off   | available       | False       |
| a10d98c1-c916-4761-83e1-426441002d18 | None | None                                 | power off   | available       | False       |
| f14bf44c-a5db-40ac-ac12-99d5c825866e | None | None                                 | power off   | available       | False       |
| ffcaa4fd-1f6c-4842-8f9d-510f5aada750 | None | 98b7fde0-7f0c-4e6f-aca8-117e1cad0ff6 | power on    | active          | False       |
| 5ad905c2-94c2-4c8f-8ac6-78cbaa292c58 | None | None                                 | power off   | available       | False       |
| 20df4168-7196-49d8-9f75-afd9d5715911 | None | 9d4e31dd-6134-464e-b247-3e6c1da72a32 | power on    | active          | False       |
| a8c647dc-5e59-46ec-aede-072eb7d9038a | None | None                                 | power off   | available       | False       |
| 6b2b4529-fe04-451b-b0b9-7ebbd1ee0164 | None | None                                 | power off   | available       | False       |
| b7c696b8-df7f-48ff-abbb-4c8d599a4ef1 | None | d7113ec2-e1dc-4e85-a29c-652f631dd928 | power on    | active          | False       |
| de8c34ec-d916-41aa-89a9-bbfae0714e30 | None | None                                 | power off   | available       | False       |
| 1dac9659-260d-4536-9eef-683747d024d6 | None | None                                 | power off   | available       | False       |
| 0e31520f-7701-411f-9056-54ca4f0392d7 | None | None                                 | power off   | available       | False       |
| 7b787a96-a1f5-49de-927b-d060f2079fdf | None | None                                 | power off   | available       | False       |
| 88b7134f-9566-48e2-82f6-e31cc8ea9191 | None | None                                 | power off   | available       | False       |
| 5fc6892e-73c0-4443-89f4-c66bb0dc1600 | None | None                                 | power off   | available       | False       |
| 7d930c2e-b426-45eb-a685-d1b4bf8120fd | None | None                                 | power off   | available       | False       |
| 81b15a3b-fbac-4117-a5a6-6f3720bb25d9 | None | None                                 | power off   | available       | False       |
| 8b9d9cfe-c435-4bca-9eb5-ac77ae90007c | None | 0df1cf5c-61df-4679-b791-c1b4062fc483 | power on    | active          | False       |
| 54fc6c98-13bc-4ce6-a865-9fa092eacae8 | None | None                                 | power off   | available       | False       |
| 74d28dbe-e733-4d78-b721-193665b43230 | None | None                                 | power off   | available       | False       |
| f18bbc32-3350-4c3e-a81b-95773eb2096f | None | None                                 | power off   | available       | False       |
| 6e388e0c-60fc-4e1b-b2b6-22c70887927a | None | None                                 | power off   | available       | False       |
| 89098be7-486c-46a2-a489-51a246c74b7d | None | 41e4d6d1-05a5-4971-a394-41eb48833eb0 | power on    | active          | False       |
| 9d07697a-0221-4f1a-a1e7-808f9c0b8a46 | None | None                                 | power off   | available       | False       |
| 7402b4a8-0410-42fb-a1c0-5ea0cb50feef | None | None                                 | power off   | available       | False       |
| 681c83a4-250f-42b5-b662-3787d7a26270 | None | None                                 | power off   | available       | False       |
| aec9147a-cdd2-4dc1-8a0f-5672f438e5c6 | None | None                                 | power off   | available       | False       |
| 9083ae94-ce0b-4e03-8271-dc757178f5a7 | None | None                                 | power off   | available       | False       |
+--------------------------------------+------+--------------------------------------+-------------+-----------------+-------------+



The resource was found at http://192.0.2.1:8004/v1/2adf80a11b894d9b8da5dbdc36d56ae4/stacks/overcloud/a3106bce-f5d4-409e-b2a6-1a38016587ef; you should be redirected automatically.

DEBUG: heatclient.common.http curl -g -i -X GET -H 'User-Agent: python-heatclient' -H 'Content-Type: application/json' -H 'X-Auth-Url: http://192.0.2.1:5000/v2.0' -H 'Accept: application/json' -H 'X-Auth-Token: {SHA1}60a9e9bbb7c42bbb8de1b9784583c4339ad82df9' http://192.0.2.1:8004/v1/2adf80a11b894d9b8da5dbdc36d56ae4/stacks/overcloud/a3106bce-f5d4-409e-b2a6-1a38016587ef
DEBUG: heatclient.common.http
HTTP/1.1 200 OK
date: Tue, 08 Sep 2015 20:22:15 GMT
connection: keep-alive
content-type: application/json; charset=UTF-8
content-length: 10520
x-openstack-request-id: req-18f719ae-4577-4e75-9d4b-2f9e7a7ce7b6

{"stack": {"parent": null, "disable_rollback": true, "description": "Nova API,Keystone,Heat Engine and API,Glance,Neutron,Dedicated MySQL server,Dedicated RabbitMQ Server,Group of Nova Computes\n", "links": [{"href": "http://192.0.2.1:8004/v1/2adf80a11b894d9b8da5dbdc36d56ae4/stacks/overcloud/a3106bce-f5d4-409e-b2a6-1a38016587ef", "rel": "self"}], "stack_status_reason": "ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"Error: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 1\"\"", "stack_name": "overcloud", "stack_user_project_id": "0d17f8cba077495091c68e614790b7cc", "stack_owner": "admin", "creation_time": "2015-09-08T19:25:50Z", "capabilities": [], "notification_topics": [], "updated_time": "2015-09-08T20:07:44Z", "timeout_mins": 240, "stack_status": "UPDATE_FAILED", "parameters": {"NeutronPublicInterfaceRawDevice": "", "KeystoneSigningKey": "******", "NtpServer": "0.rhel.pool.ntp.org", "NeutronDhcpAgentsPerNetwork": "3", "NeutronFlatNetworks": "datacentre", "NovaEnableRbdBackend": "False", "controllerImage": "overcloud-full", "GlancePassword": "******", "NeutronTunnelIdRanges": "1:1000", "CinderEnableRbdBackend": "False", "SwiftReplicas": "3", "NeutronPublicInterfaceTag": "", "ControllerHostnameFormat": "%stackname%-controller-%index%", "KeystoneSigningCertificate": "-----BEGIN CERTIFICATE-----\nMIIDJDCCAgygAwIBAgIBAjANBgkqhkiG9w0BAQUFADBTMQswCQYDVQQGEwJYWDEO\nMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEU\nMBIGA1UEAxMLS2V5c3RvbmUgQ0EwHhcNMTUwOTA4MTkyNTQ2WhcNMjUwOTA1MTky\nNTQ2WjBYMQswCQYDVQQGEwJYWDEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVu\nc2V0MQ4wDAYDVQQKEwVVbnNldDEZMBcGA1UEAxMQS2V5c3RvbmUgU2lnbmluZzCC\nASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMe2VS/SsV+aSalFabbEfaZe\nlXB7iigQNoJ6IB4GROV80bPrXTCTb6tcfw0SXSyVQOmtdxlj9HHLf+PmXymPFI7J\nncna9KCgnEVIRxxy/Fnbh9ZyC+y5M92EmyU3LDs34S5fEDlLaynk5wnNVxq6SqR6\nR3JNy45qVMCPt3Bx6tNu/ckrihaEYmiUcilyDelHWDjEo1LTrw6AWd4xEbHfO5F5\nvttxKSFo71s7lK5hxMC6obPi6Wu001/UxaeMyCU9zeIYyW2zzVtN8dmnx5s7r5cB\nOAu61BRyNf1XIm8g5UmEtcAWgEW6XXfcDbaKfsx0L2LZVEpWZWNLf2+puwM72LMC\nAwEAATANBgkqhkiG9w0BAQUFAAOCAQEADTNwaOYIfvQj5kIjFQuTq5D2YLy5aswo\nJRDVzfrSUfk7eaJlFRtFOjk2zQFFkMQ5UfoWA7OBIuSPdqqlSzvkhozdlSTCesf8\nkGWiQ16vHpMlGqycIkJfin3FKxT32ety1UBFjosRg9tjXpZRLkfC2ved9X1h5QoK\nx2GIH1843PDayxKkZoXHr+cQeo2eX/HNCphO2na/U+59KqN/WOz+kNGu5aE6mkJA\ncu0K7S1iwXehgtLWXPBdU7rUx2W1HlTf+PB/FUS+CAFitByEldE4XGtu6ccxq9Ux\nuC8NEnzSIhLWBVbgbsLRvdv4qq6owSGIqLzmjG3Zb9UATAJ177wwQQ==\n-----END CERTIFICATE-----\n", "CephStorageCount": "0", "ServiceNetMap": "{\"NovaVncProxyNetwork\": \"internal_api\", \"CinderApiNetwork\": \"internal_api\", \"NovaApiNetwork\": \"internal_api\", \"CeilometerApiNetwork\": \"internal_api\", \"CephStorageHostnameResolveNetwork\": \"storage\", \"SwiftMgmtNetwork\": \"storage_mgmt\", \"MemcachedNetwork\": \"internal_api\", \"RabbitMqNetwork\": \"internal_api\", \"KeystoneAdminApiNetwork\": \"internal_api\", \"SwiftProxyNetwork\": \"storage\", \"NeutronTenantNetwork\": \"tenant\", \"CephClusterNetwork\": \"storage_mgmt\", \"NovaMetadataNetwork\": \"internal_api\", \"ControllerHostnameResolveNetwork\": \"internal_api\", \"NeutronApiNetwork\": \"internal_api\", \"GlanceApiNetwork\": \"storage\", \"ObjectStorageHostnameResolveNetwork\": \"internal_api\", \"KeystonePublicApiNetwork\": \"internal_api\", \"HeatApiNetwork\": \"internal_api\", \"GlanceRegistryNetwork\": \"internal_api\", \"RedisNetwork\": \"internal_api\", \"MysqlNetwork\": \"internal_api\", \"BlockStorageHostnameResolveNetwork\": \"internal_api\", \"ComputeHostnameResolveNetwork\": \"internal_api\", \"CephPublicNetwork\": \"storage\", \"MongoDbNetwork\": \"internal_api\", \"HorizonNetwork\": \"internal_api\", \"CinderIscsiNetwork\": \"storage\"}", "SnmpdReadonlyUserName": "ro_snmp_user", "KeyName": "default", "AdminToken": "******", "CinderISCSIHelper": "lioadm", "BlockStorageImage": "overcloud-full", "OvercloudControlFlavor": "control", "CloudName": "overcloud", "RabbitPassword": "******", "OS::stack_id": "a3106bce-f5d4-409e-b2a6-1a38016587ef", "PublicVirtualInterface": "br-ex", "Debug": "True", "NeutronPublicInterface": "nic1", "GlanceFilePcmkDevice": "", "OvercloudComputeFlavor": "compute", "CinderEnableIscsiBackend": "True", "NeutronMetadataProxySharedSecret": "unset", "CephStorageRemovalPolicies": "[]", "GlancePort": "9292", "SwiftPartPower": "10", "NeutronPublicInterfaceDefaultRoute": "", "HeatPassword": "******", "SwiftMountCheck": "False", "ControlVirtualInterface": "br-ex", "HeatStackDomainAdminPassword": "******", "OS::stack_name": "overcloud", "OvercloudSwiftStorageFlavor": "baremetal", "CephAdminKey": "", "SwiftPassword": "******", "ComputeHostnameFormat": "%stackname%-compute-%index%", "controllerExtraConfig": "{\"neutron_ovs_use_veth\": true, \"neutron::agents::dhcp::enable_metadata_network\": true, \"neutron::agents::dhcp::enable_isolated_metadata\": true}", "SSLCertificate": "******", "ControllerEnableCephStorage": "False", "ControllerRemovalPolicies": "[]", "GlanceBackend": "swift", "RabbitClientUseSSL": "False", "NeutronAllowL3AgentFailover": "False", "GlanceFilePcmkOptions": "", "RabbitUserName": "guest", "GlanceLogFile": "", "NeutronExternalNetworkBridge": "''", "BlockStorageHostnameFormat": "%stackname%-blockstorage-%index%", "SwiftStorageImage": "overcloud-full", "RabbitCookieSalt": "unset", "AdminPassword": "******", "NeutronDVR": "False", "CephClusterFSID": "", "CephStorageHostnameFormat": "%stackname%-cephstorage-%index%", "NeutronAgentMode": "dvr_snat", "NeutronEnableTunnelling": "False", "CephStorageImage": "overcloud-full", "NeutronDnsmasqOptions": "dhcp-option-force=26,1400", "MysqlMaxConnections": "1024", "ControllerCount": "3", "NeutronTunnelTypes": "gre", "SSLCACertificate": "", "GlanceFilePcmkManage": "False", "PublicVirtualFixedIPs": "[]", "NeutronPassword": "******", "NeutronBridgeMappings": "datacentre:br-ex", "NeutronVniRanges": "1:1000", "PublicVirtualNetwork": "ctlplane", "NovaImage": "overcloud-full", "SSLKey": "******", "GlanceFilePcmkFstype": "nfs", "OvercloudCephStorageFlavor": "baremetal", "CinderPassword": "******", "HypervisorNeutronPublicInterface": "nic1", "ObjectStorageCount": "0", "CinderLVMLoopDeviceSize": "5000", "CeilometerBackend": "mongodb", "CinderNfsServers": "", "ImageUpdatePolicy": "REBUILD_PRESERVE_EPHEMERAL", "MysqlInnodbBufferPoolSize": "0", "BlockStorageCount": "0", "ControlFixedIPs": "[]", "GlanceProtocol": "http", "SwiftHashSuffix": "******", "SwiftMinPartHours": "1", "KeystoneSSLCertificate": "", "NeutronL3HA": "True", "NovaComputeLibvirtType": "qemu", "EnableGalera": "True", "CephMonKey": "", "ComputeRemovalPolicies": "[]", "HypervisorNeutronPhysicalBridge": "br-ex", "CeilometerMeteringSecret": "******", "CinderEnableNfsBackend": "False", "OS::project_id": "2adf80a11b894d9b8da5dbdc36d56ae4", "NovaComputeDriver": "libvirt.LibvirtDriver", "ExtraConfig": "{}", "SnmpdReadonlyUserPassword": "******", "OvercloudBlockStorageFlavor": "baremetal", "BlockStorageRemovalPolicies": "[]", "RabbitClientPort": "5672", "NovaComputeExtraConfig": "{}", "ComputeCount": "2", "KeystoneSSLCertificateKey": "******", "NeutronControlPlaneID": "23e23f02-947c-455f-ad13-2309d36ad583", "EnablePacemaker": "False", "NeutronNetworkType": "vlan", "CeilometerPassword": "******", "ObjectStorageRemovalPolicies": "[]", "NeutronMechanismDrivers": "openvswitch,bsn_ml2", "ControllerEnableSwiftStorage": "True", "GlanceNotifierStrategy": "noop", "KeystoneCACertificate": "-----BEGIN CERTIFICATE-----\nMIIDNzCCAh+gAwIBAgIBATANBgkqhkiG9w0BAQUFADBTMQswCQYDVQQGEwJYWDEO\nMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEU\nMBIGA1UEAxMLS2V5c3RvbmUgQ0EwHhcNMTUwOTA4MTkyNTQ2WhcNMjUwOTA1MTky\nNTQ2WjBTMQswCQYDVQQGEwJYWDEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVu\nc2V0MQ4wDAYDVQQKEwVVbnNldDEUMBIGA1UEAxMLS2V5c3RvbmUgQ0EwggEiMA0G\nCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQClN71o2AKh2zY6hMP4yLaOhguigOoF\n1T7iotweb/59IIGbQG8aXc98iZfhICtLBUeyJw05xvSt30ISnwagHlDrJfm/1fJj\nquY9T1bPE6D8uplmS0ZVrAKln0YK6gcKXU18cFwtK8X1MWXOZuqT/8AHwTbjjUHD\nax66E2t92oJnuSU20U5LmnYX5XMkh6uzzP0ro1RAgJvGKCZv6bgAVgSg+Nz04cdv\nd3FZvLcb4sCvgnkxXvti2GurR3Q9jCNt/VMQy2i8DpJsT8dFpK+K8l2VXjcvb7wp\nygskaqF9zJwWcT5//ORm3C2UnauxcMHXVJE5TNmox8Fg3JkBiAqpkatBAgMBAAGj\nFjAUMBIGA1UdEwEB/wQIMAYBAf8CAQAwDQYJKoZIhvcNAQEFBQADggEBAIus9OrQ\nW9m6+vJhcjMBb4HDoGui9G94bAXiv/psjH2/zRZVHJyd0gJzaR/yug3JAETlAIk1\nGoV4/PwfylF/ZJocoPs33/m9ri2XDW7lOv8Xvrjk/fW6K+FF3VE54L9ldk7SjUuL\ntqc6amz4CacvCOEyvGaOg4fd4UCVfMe91fB0cb7m+JVfzdqV2aOYrwcfV8tK90uW\nsgQ4GlXsy560MbriLYNDRA/pMVYP36yUE00DjPKmzOK1nn9h3rpL0rXvnlDNFzrh\ns9/tuGsjsp+uQ9dJnSAXvSz/jwbz70leQcyjDa5HcKOrpeUNHVuA9ZIxrYL4pgfH\ntmLuZT1GnmQ3gYk=\n-----END CERTIFICATE-----\n", "NovaPassword": "******", "CinderNfsMountOptions": "", "NeutronPublicInterfaceIP": "", "ObjectStorageHostnameFormat": "%stackname%-objectstorage-%index%", "CeilometerComputeAgent": "", "UpdateIdentifier": "", "NeutronComputeAgentMode": "dvr", "NeutronNetworkVLANRanges": "datacentre:1001:2000"}, "id": "a3106bce-f5d4-409e-b2a6-1a38016587ef", "outputs": [{"output_value": "172.17.0.11", "description": "VIP for Ceilometer API internal endpoint", "output_key": "CeilometerInternalVip"}, {"output_value": "172.18.0.10", "description": "VIP for Glance API internal endpoint", "output_key": "GlanceInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Heat API internal endpoint", "output_key": "HeatInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Keystone API internal endpoint", "output_key": "KeystoneInternalVip"}, {"output_value": "10.17.66.11", "description": "Controller VIP for public API endpoints", "output_key": "PublicVip"}, {"output_value": "http://10.17.66.11:5000/v2.0/", "description": "URL for the Overcloud Keystone service", "output_key": "KeystoneURL"}, {"output_value": "172.17.0.11", "description": "VIP for Neutron API internal endpoint", "output_key": "NeutronInternalVip"}, {"output_value": "172.18.0.10", "description": "VIP for Swift Proxy internal endpoint", "output_key": "SwiftInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Cinder API internal endpoint", "output_key": "CinderInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Nova API internal endpoint", "output_key": "NovaInternalVip"}], "template_description": "Nova API,Keystone,Heat Engine and API,Glance,Neutron,Dedicated MySQL server,Dedicated RabbitMQ Server,Group of Nova Computes\n"}}

ERROR: openstack Heat Stack update failed.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 295, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 53, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 824, in take_action
    self._deploy_tripleo_heat_templates(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 498, in _deploy_tripleo_heat_templates
    parsed_args.timeout)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 443, in _heat_deploy
    raise Exception("Heat Stack update failed.")
Exception: Heat Stack update failed.
DEBUG: openstackclient.shell clean_up DeployOvercloud
DEBUG: openstackclient.shell got an error: Heat Stack update failed.
ERROR: openstackclient.shell Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 176, in run
    return super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 230, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 295, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 53, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 824, in take_action
    self._deploy_tripleo_heat_templates(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 498, in _deploy_tripleo_heat_templates
    parsed_args.timeout)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 443, in _heat_deploy
    raise Exception("Heat Stack update failed.")
Exception: Heat Stack update failed.

Deploying templates in the directory /usr/share/openstack-tripleo-heat-templates

Version-Release number of selected component (if applicable):

deploy command is 

 openstack overcloud deploy -e /home/stack/network-environment.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /home/stack/templates/bigswitch-config.yaml --neutron-network-type vlan --neutron-network-vlan-ranges datacentre:1001:2000 --neutron-disable-tunneling --compute-scale 5 --control-scale 3 --ceph-storage-scale 0 --templates --control-flavor control --compute-flavor compute --ntp-server 0.rhel.pool.ntp.org --debug 2>&1 | tee haha.log



How reproducible:
This is the first time I am seeing this, typically it was failing when attempting to scale to 10 compute nodes.

Steps to Reproduce:
1. Deploy 3 controller and 2 compute, it is successful
2. Deploy another 3 more compute, making total to 5
3. Deployment failed

Actual results:


Expected results:


Additional info:
Comment 3 bigswitch 2015-09-08 16:53:26 EDT
ironic log and sosreport is uploaded to 

https://bigswitch.box.com/s/3hk3qd5d2izxlpa2tuo1buknm8gj76rq
Comment 4 chris alfonso 2015-09-14 12:49:09 EDT
Is this the same issue you asked about via email? Was it the keystone timeout issue, or max resources?
Comment 5 bigswitch 2015-09-14 16:25:54 EDT
I seen this only once, so its not a maximum resource issue or the keystone timeout issue. I'm not sure if this is reproducible though.
Comment 6 bigswitch 2015-09-14 16:26:08 EDT
I seen this only once, so its not a maximum resource issue or the keystone timeout issue. I'm not sure if this is reproducible though.
Comment 7 chris alfonso 2015-09-22 12:51:02 EDT
Are you able to reproduce this issue?
Comment 8 chris alfonso 2015-09-30 12:52:06 EDT
Haven't heard anything on this. Just wanted to check to see if you're still running into this issue with a scale up and I saw you're able to scale to 50 so I'm going to close this bug. If it's still an issue, you can reopen.

Note You need to log in before you can comment on or make changes to this bug.