Description of problem:
Not able to deploy more than 10 nodes. I have deployed three controller and three computes, added another three computes and it went through fine. When attempting to add another three compute nodes it failed with the below error:


{"stack": {"parent": null, "disable_rollback": true, "description": "Nova API,Keystone,Heat Engine and API,Glance,Neutron,Dedicated MySQL server,Dedicated RabbitMQ Server,Group of Nova Computes\n", "links": [{"href": "http://192.0.2.1:8004/v1/2adf80a11b894d9b8da5dbdc36d56ae4/stacks/overcloud/5cf9eab1-5810-48b2-ace2-d3102620896d", "rel": "self"}], "stack_status_reason": "ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"ResourceUnknownStatus: Resource failed - Unknown status FAILED due to \"Resource CREATE failed: Forbidden: Maximum number of ports exceeded (HTTP 403) (Request-ID: req-36febe07-0fbb-4f0c-a001-b1b516be4761)\"\"", "stack_name": "overcloud", "stack_user_project_id": "78cc6826f58442d69cd8743f2f80f099", "stack_owner": "admin", "creation_time": "2015-09-08T21:01:12Z", "capabilities": [], "notification_topics": [], "updated_time": "2015-09-08T22:27:57Z", "timeout_mins": 240, "stack_status": "UPDATE_FAILED", "parameters": {"NeutronPublicInterfaceRawDevice": "", "KeystoneSigningKey": "******", "NtpServer": "0.rhel.pool.ntp.org", "NeutronDhcpAgentsPerNetwork": "3", "NeutronFlatNetworks": "datacentre", "NovaEnableRbdBackend": "False", "controllerImage": "overcloud-full", "GlancePassword": "******", "NeutronTunnelIdRanges": "1:1000", "CinderEnableRbdBackend": "False", "SwiftReplicas": "3", "NeutronPublicInterfaceTag": "", "ControllerHostnameFormat": "%stackname%-controller-%index%", "KeystoneSigningCertificate": "-----BEGIN CERTIFICATE-----\nMIIDJDCCAgygAwIBAgIBAjANBgkqhkiG9w0BAQUFADBTMQswCQYDVQQGEwJYWDEO\nMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEU\nMBIGA1UEAxMLS2V5c3RvbmUgQ0EwHhcNMTUwOTA4MjEwMTA4WhcNMjUwOTA1MjEw\nMTA4WjBYMQswCQYDVQQGEwJYWDEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVu\nc2V0MQ4wDAYDVQQKEwVVbnNldDEZMBcGA1UEAxMQS2V5c3RvbmUgU2lnbmluZzCC\nASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANlCHxE2vBQj3rpFXJIucPpZ\nsYYGZuVZx10XvtVYROV6yMBm6SE0cM/WsaVlKrCLHj2z662Z59R/8nlvZkDAroGx\nDGyiXSFBpHkAJ9MTTEbStSwTZDr03lIzbkcs0eBWBbz96M6/hqn78uZpTU7Mw6PM\n+mPSMijY9cYy+xYBj4wchxokqv90j4XkGk2TFu8+VfpEo7NOXOgJU9OKOwO2Nu3D\n15SeeQeF4Ub7lyMbPJP/FXWV+W7BNkYSAmORp/5WAYXYV1cSilzILGWvKNihxx0S\nCwLq65zQI/99iUiAX8vDEjNKNf8t8ZU/KTKVIpmOFsBIAuYsm0jj2NACkCkSeWcC\nAwEAATANBgkqhkiG9w0BAQUFAAOCAQEACvQvfMqkWAS9DbOLmJrA5XNW/wMmxHra\nkaCCl69LP3MrqVTZCUkgpfbnhMR/FjVc8SSYXo0PfnkZMe/Dj9bdAPprAdvnui8L\nnRuNy4QMwK0Dy/uxcAsDk0Y/OLS8SJoLd6EZj9rzIrv6QmJ6nAuRKDu+EGuO6hAf\nQPlJN0eFPNPVl7r0GBJuLOvYZpjiv/lrICstD/osZpY2167itB8GESIw0/p597pb\ndiQuAzO/JMeNYDsGwDUjMYfMKLiNU/JD4IIvykx7TqxAhbShr/58Hazhbqp9RQ7J\nSge8/sezR0wbQafPz5I3ymIUgGcvXL9RyTnIEV08cZD7078l1jj/cg==\n-----END CERTIFICATE-----\n", "CephStorageCount": "0", "ServiceNetMap": "{\"NovaVncProxyNetwork\": \"internal_api\", \"CinderApiNetwork\": \"internal_api\", \"NovaApiNetwork\": \"internal_api\", \"CeilometerApiNetwork\": \"internal_api\", \"CephStorageHostnameResolveNetwork\": \"storage\", \"SwiftMgmtNetwork\": \"storage_mgmt\", \"MemcachedNetwork\": \"internal_api\", \"RabbitMqNetwork\": \"internal_api\", \"KeystoneAdminApiNetwork\": \"internal_api\", \"SwiftProxyNetwork\": \"storage\", \"NeutronTenantNetwork\": \"tenant\", \"CephClusterNetwork\": \"storage_mgmt\", \"NovaMetadataNetwork\": \"internal_api\", \"ControllerHostnameResolveNetwork\": \"internal_api\", \"NeutronApiNetwork\": \"internal_api\", \"GlanceApiNetwork\": \"storage\", \"ObjectStorageHostnameResolveNetwork\": \"internal_api\", \"KeystonePublicApiNetwork\": \"internal_api\", \"HeatApiNetwork\": \"internal_api\", \"GlanceRegistryNetwork\": \"internal_api\", \"RedisNetwork\": \"internal_api\", \"MysqlNetwork\": \"internal_api\", \"BlockStorageHostnameResolveNetwork\": \"internal_api\", \"ComputeHostnameResolveNetwork\": \"internal_api\", \"CephPublicNetwork\": \"storage\", \"MongoDbNetwork\": \"internal_api\", \"HorizonNetwork\": \"internal_api\", \"CinderIscsiNetwork\": \"storage\"}", "SnmpdReadonlyUserName": "ro_snmp_user", "KeyName": "default", "AdminToken": "******", "CinderISCSIHelper": "lioadm", "BlockStorageImage": "overcloud-full", "OvercloudControlFlavor": "control", "CloudName": "overcloud", "RabbitPassword": "******", "OS::stack_id": "5cf9eab1-5810-48b2-ace2-d3102620896d", "PublicVirtualInterface": "br-ex", "Debug": "True", "NeutronPublicInterface": "nic1", "GlanceFilePcmkDevice": "", "OvercloudComputeFlavor": "compute", "CinderEnableIscsiBackend": "True", "NeutronMetadataProxySharedSecret": "unset", "CephStorageRemovalPolicies": "[]", "GlancePort": "9292", "SwiftPartPower": "10", "NeutronPublicInterfaceDefaultRoute": "", "HeatPassword": "******", "SwiftMountCheck": "False", "ControlVirtualInterface": "br-ex", "HeatStackDomainAdminPassword": "******", "OS::stack_name": "overcloud", "OvercloudSwiftStorageFlavor": "baremetal", "CephAdminKey": "", "SwiftPassword": "******", "ComputeHostnameFormat": "%stackname%-compute-%index%", "controllerExtraConfig": "{\"neutron_ovs_use_veth\": true, \"neutron::agents::dhcp::enable_metadata_network\": true, \"neutron::agents::dhcp::enable_isolated_metadata\": true}", "SSLCertificate": "******", "ControllerEnableCephStorage": "False", "ControllerRemovalPolicies": "[]", "GlanceBackend": "swift", "RabbitClientUseSSL": "False", "NeutronAllowL3AgentFailover": "False", "GlanceFilePcmkOptions": "", "RabbitUserName": "guest", "GlanceLogFile": "", "NeutronExternalNetworkBridge": "''", "BlockStorageHostnameFormat": "%stackname%-blockstorage-%index%", "SwiftStorageImage": "overcloud-full", "RabbitCookieSalt": "unset", "AdminPassword": "******", "NeutronDVR": "False", "CephClusterFSID": "", "CephStorageHostnameFormat": "%stackname%-cephstorage-%index%", "NeutronAgentMode": "dvr_snat", "NeutronEnableTunnelling": "False", "CephStorageImage": "overcloud-full", "NeutronDnsmasqOptions": "dhcp-option-force=26,1400", "MysqlMaxConnections": "1024", "ControllerCount": "3", "NeutronTunnelTypes": "gre", "SSLCACertificate": "", "GlanceFilePcmkManage": "False", "PublicVirtualFixedIPs": "[]", "NeutronPassword": "******", "NeutronBridgeMappings": "datacentre:br-ex", "NeutronVniRanges": "1:1000", "PublicVirtualNetwork": "ctlplane", "NovaImage": "overcloud-full", "SSLKey": "******", "GlanceFilePcmkFstype": "nfs", "OvercloudCephStorageFlavor": "baremetal", "CinderPassword": "******", "HypervisorNeutronPublicInterface": "nic1", "ObjectStorageCount": "0", "CinderLVMLoopDeviceSize": "5000", "CeilometerBackend": "mongodb", "CinderNfsServers": "", "ImageUpdatePolicy": "REBUILD_PRESERVE_EPHEMERAL", "MysqlInnodbBufferPoolSize": "0", "BlockStorageCount": "0", "ControlFixedIPs": "[]", "GlanceProtocol": "http", "SwiftHashSuffix": "******", "SwiftMinPartHours": "1", "KeystoneSSLCertificate": "", "NeutronL3HA": "True", "NovaComputeLibvirtType": "qemu", "EnableGalera": "True", "CephMonKey": "", "ComputeRemovalPolicies": "[]", "HypervisorNeutronPhysicalBridge": "br-ex", "CeilometerMeteringSecret": "******", "CinderEnableNfsBackend": "False", "OS::project_id": "2adf80a11b894d9b8da5dbdc36d56ae4", "NovaComputeDriver": "libvirt.LibvirtDriver", "ExtraConfig": "{}", "SnmpdReadonlyUserPassword": "******", "OvercloudBlockStorageFlavor": "baremetal", "BlockStorageRemovalPolicies": "[]", "RabbitClientPort": "5672", "NovaComputeExtraConfig": "{}", "ComputeCount": "6", "KeystoneSSLCertificateKey": "******", "NeutronControlPlaneID": "23e23f02-947c-455f-ad13-2309d36ad583", "EnablePacemaker": "False", "NeutronNetworkType": "vlan", "CeilometerPassword": "******", "ObjectStorageRemovalPolicies": "[]", "NeutronMechanismDrivers": "openvswitch,bsn_ml2", "ControllerEnableSwiftStorage": "True", "GlanceNotifierStrategy": "noop", "KeystoneCACertificate": "-----BEGIN CERTIFICATE-----\nMIIDNzCCAh+gAwIBAgIBATANBgkqhkiG9w0BAQUFADBTMQswCQYDVQQGEwJYWDEO\nMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVuc2V0MQ4wDAYDVQQKEwVVbnNldDEU\nMBIGA1UEAxMLS2V5c3RvbmUgQ0EwHhcNMTUwOTA4MjEwMTA4WhcNMjUwOTA1MjEw\nMTA4WjBTMQswCQYDVQQGEwJYWDEOMAwGA1UECBMFVW5zZXQxDjAMBgNVBAcTBVVu\nc2V0MQ4wDAYDVQQKEwVVbnNldDEUMBIGA1UEAxMLS2V5c3RvbmUgQ0EwggEiMA0G\nCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDAHBh7RtJxa4wTOlAHr7a3NkR3Tk/t\n05wsDC67YlbShq0dGt51w3lIPML56XqEBxlBln9gZOoQKyf6HF2qaf+Cgw+c/4pS\nbgWEOzJBmFrLU0Ui51mndujd48m/ITri3ABy2hs1kUiEpjcQlAxMvMyizZ/my2oV\nbGz/kGazgDPL7kR6w25sJh2Np5Cmuc7xPri7QBmqgn81I13yxvnfzyvjhzpWhAaN\nounxh1RqqvNje4p1nt7+tyhCoOot/0rohd7vFDemjoZX1JHs7KTBG8Hz7U5EpBKo\nzTszT2Tv8goBkUaH5o2d1JPQMAIWIUPbOjz6frzCFFEYhnNFH1ELMsX3AgMBAAGj\nFjAUMBIGA1UdEwEB/wQIMAYBAf8CAQAwDQYJKoZIhvcNAQEFBQADggEBACquR1l3\nmrzODQwcCv1lwP/3jFRzCqryUxWzabFoOOWQOgOQ8sEiZ/WoMm5Ht+ekWv9KKGTF\nQYhWCCtyEB31nI5bnmtyiL23ILQ3yr5tSHFkLVDNgOVHY51haCmUkmLL+h4RGFjq\n8zqQY0PyB49AdKONkvitkMjl+8QoVcal2j/QVcNCS2f3R/qDwAR4Yu+/baF0aKq6\nlQJrhO67iuBDGqQjlGs9vyqmIMJtJ8qhRqGENr2P5g/D4+dhGBIZ3bcGoa1BBMSa\nTEaUgcjR4SUY8ibXQrQqFuCabF2imTTDqU/IZVt/Kxa3aX+C4zUE7l7vMIIf4ftN\nT6+K037zcPqMJwc=\n-----END CERTIFICATE-----\n", "NovaPassword": "******", "CinderNfsMountOptions": "", "NeutronPublicInterfaceIP": "", "ObjectStorageHostnameFormat": "%stackname%-objectstorage-%index%", "CeilometerComputeAgent": "", "UpdateIdentifier": "", "NeutronComputeAgentMode": "dvr", "NeutronNetworkVLANRanges": "datacentre:1001:2000"}, "id": "5cf9eab1-5810-48b2-ace2-d3102620896d", "outputs": [{"output_value": "172.17.0.11", "description": "VIP for Ceilometer API internal endpoint", "output_key": "CeilometerInternalVip"}, {"output_value": "172.18.0.10", "description": "VIP for Glance API internal endpoint", "output_key": "GlanceInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Heat API internal endpoint", "output_key": "HeatInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Keystone API internal endpoint", "output_key": "KeystoneInternalVip"}, {"output_value": "10.17.66.11", "description": "Controller VIP for public API endpoints", "output_key": "PublicVip"}, {"output_value": "http://10.17.66.11:5000/v2.0/", "description": "URL for the Overcloud Keystone service", "output_key": "KeystoneURL"}, {"output_value": "172.17.0.11", "description": "VIP for Neutron API internal endpoint", "output_key": "NeutronInternalVip"}, {"output_value": "172.18.0.10", "description": "VIP for Swift Proxy internal endpoint", "output_key": "SwiftInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Cinder API internal endpoint", "output_key": "CinderInternalVip"}, {"output_value": "172.17.0.11", "description": "VIP for Nova API internal endpoint", "output_key": "NovaInternalVip"}], "template_description": "Nova API,Keystone,Heat Engine and API,Glance,Neutron,Dedicated MySQL server,Dedicated RabbitMQ Server,Group of Nova Computes\n"}}

ERROR: openstack Heat Stack update failed.
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 295, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 53, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 824, in take_action
    self._deploy_tripleo_heat_templates(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 498, in _deploy_tripleo_heat_templates
    parsed_args.timeout)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 443, in _heat_deploy
    raise Exception("Heat Stack update failed.")
Exception: Heat Stack update failed.
DEBUG: openstackclient.shell clean_up DeployOvercloud
DEBUG: openstackclient.shell got an error: Heat Stack update failed.
ERROR: openstackclient.shell Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/openstackclient/shell.py", line 176, in run
    return super(OpenStackShell, self).run(argv)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 230, in run
    result = self.run_subcommand(remainder)
  File "/usr/lib/python2.7/site-packages/cliff/app.py", line 295, in run_subcommand
    result = cmd.run(parsed_args)
  File "/usr/lib/python2.7/site-packages/cliff/command.py", line 53, in run
    self.take_action(parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 824, in take_action
    self._deploy_tripleo_heat_templates(stack, parsed_args)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 498, in _deploy_tripleo_heat_templates
    parsed_args.timeout)
  File "/usr/lib/python2.7/site-packages/rdomanager_oscplugin/v1/overcloud_deploy.py", line 443, in _heat_deploy
    raise Exception("Heat Stack update failed.")
Exception: Heat Stack update failed.

Deploying templates in the directory /usr/share/openstack-tripleo-heat-templates


Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Deploy three controller and three compute
2. Deploy another three compute
3. Attempt to deploy another three or more compute nodes will fail

Actual results:


Expected results:
Should succeed

Additional info: