Description of problem: I registered Ubutu 14.04 image (also Ubntu 15.10 glance image was tested) with SAHARA on RDO Liberty with corresponding tags "hdp" and "2.0.6" Then I created Master && Slave templates initializing daemons follow samples :- [root ServerCentOS7 version_2_0_6(keystone_admin)]# cat master.json { "plugin_name": "hdp", "hadoop_version": "2.0.6", "node_processes": [ "NAMENODE", "AMBARI_SERVER", "RESOURCEMANAGER", "HISTORYSERVER", "ZOOKEEPER_SERVER" ], "name": "hdp-206-default-master", "floating_ip_pool": "{floating_ip_pool}", "flavor_id": "{flavor_id}", "auto_security_group": "{auto_security_group}", "security_groups": "{security_groups}" } [root ServerCentOS7 version_2_0_6(keystone_admin)]# cat worker.json { "plugin_name": "hdp", "hadoop_version": "2.0.6", "node_processes": [ "DATANODE", "NODEMANAGER" ], "name": "hdp-206-default-worker", "floating_ip_pool": "{floating_ip_pool}", "flavor_id": "{flavor_id}", "auto_security_group": "{auto_security_group}", "security_groups": "{security_groups} Creating Cluster's template I multiplied by 3 Slave's template trying to fork 4 VMs at a time. Finally spawning the HDP 2.0.6 Cluster results an error /val/log/sahara/sahara-engine.log :- 3eb98f8c6e7] HeatStackException: Heat stack failed with status Resource CREATE failed: resources.hdp-206-default-master: OverQuotaClient: resources[0].resources.hdp-206-cluster1-hdp-206-default-master-b33b997e: Quota exceeded for resources: ['security_group_rule'] Version-Release number of selected component (if applicable): [root ServerCentOS7 ~]# rpm -qa \*sahara\* openstack-sahara-engine-3.0.0-5.cc218ddgit.el7.noarch python-saharaclient-0.11.1-1.el7.noarch openstack-sahara-api-3.0.0-5.cc218ddgit.el7.noarch openstack-sahara-common-3.0.0-5.cc218ddgit.el7.noarch How reproducible: Steps to Reproduce: 1. Create Master and Work templates 2. Create Cluster template 1xMaster + 3xWork 3. Attempt to launch cluster via registered glance image Ubuntu 14.04 ( or or Ubuntu 15.10 ) Actual results: Finally spawning the HDP 2.0.6 Cluster results an error /val/log/sahara/sahara-engine.log :- 3eb98f8c6e7] HeatStackException: Heat stack failed with status Resource CREATE failed: resources.hdp-206-default-master: OverQuotaClient: resources[0].resources.hdp-206-cluster1-hdp-206-default-master-b33b997e: Quota exceeded for resources: ['security_group_rule'] 26-02-07 21:21:01.359 1384 ERROR sahara.service.ops [instance: none, cluster: 902d7bc4-a0cb-45dc-9e81-33eb98f8c6e7] Error ID: 1826c6f4-bbc2-4c91-be0c-57bdcd02faf2 2016-02-07 21:21:01.359 1384 ERROR sahara.service.ops [instance: none, cluster: 902d7bc4-a0cb-45dc-9e81-33eb98f8c6e7] 2016-02-07 21:21:10.701 1384 WARNING sahara.service.heat.heat_engine [req-c0192e44-e84c-4f59-bf80-19c9584baa39 ] [instance: none, cluster: 902d7bc4-a0cb-45dc-9e81-33eb98f8c6e7] Cluster creation rollback (reason: Heat stack failed with status Resource CREATE failed: resources.hdp-206-default-master: OverQuotaClient: resources[0].resources.hdp-206-cluster1-hdp-206-default-master-b33b997e: Quota exceeded for resources: ['security_group_rule'] Error ID: 1826c6f4-bbc2-4c91-be0c-57bdcd02faf2) 2016-02-07 21:21:10.958 1384 INFO sahara.utils.cluster [req-c0192e44-e84c-4f59-bf80-19c9584baa39 ] [instance: none, cluster: 902d7bc4-a0cb-45dc-9e81-33eb98f8c6e7] Cluster status has been changed. New status=Error Expected results: Cluster goes goes to Active. Hadoop Cluster HDP 2.0.6 gets deployed across Master and 3 Work cloud VMs. Additional info:
Please raise the quota of your tenant. This is something that should be probably reported in the upstream documentation. For example, the upstream Sahara CI increases the values of security group rules as follow: https://github.com/openstack/sahara-ci-config/blob/master/config/devstack/local.sh#L63
I apologize for inconvenience. I am doing upstream image download for HDP 2.0.6 ( 100 KB/sec ). Would I have problems with upstream image I will reopen the bug. For now status is closed. Upstream image for Vanilla 2.7.1 does work.
I think you will still have to increase the quota for security group rules in your tenant, please consider this when performing your testing.
Luigi is correct; the default sec group rules can be insufficient for spawning Sahara clusters. Because the HDP cluster requires more rules (there are more services running), it is quite possible that the less rule-intense vanilla cluster could spawn using sec groups beneath a certain threshold, but that the HDP cluster would not. This is a stack operations issue, and the quota system is behaving appropriately (I increase the quotas for this setting regularly in my own envs, for this exact reason). If you find any other issues as you go, though, please do report them. Thanks!
Thanks to advise of Luigi Toscano [1] HDP 2.0.6 plug-in was successfully tested. I was able to launch 5 VMs Namenode, SecondaryNamenode , 3*(Worker Nodes) on top of RDO Liberty via templates generated in Sahara GUI environment. Diagnostic obtained via http://master:50070 looks OK for me. Quotas update via https://github.com/openstack/sahara-ci-config/blob/master/config/devstack/local.sh#L63 seems to be important for both HDP 2.0.6 and CDH 5.0.4 Hadoop clusters launching
Configuration fix r/t quotas successful per reporter.