Description of problem: Error creating cluster with HDP version 2.0.6 sahara cluster-create --json cluster_create-hdp.json ERROR: Cluster is missing a service: YARN Error ID: 33c7d728-bd47-453e-96cb-0ef03775b56a Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1.Create Node group template -- missing PROCESSES Ensure the existence of a JOBTRACKER should any TASKTRACKER be deployed to the cluster -- if you follow http://docs.openstack.org/developer/sahara/userdoc/hdp_plugin.html ??? its available with Vanilla plugins but not in HDP 2. Create cluster template 3. Create cluster 1. [root@sahara ~(keystone_admin)]# cat ng_worker_template_create-hdp.json { "name": "worker-hdp-206-tmpl", "flavor_id": "2", "plugin_name": "hdp", "hadoop_version": "2.0.6", "node_processes": ["DATANODE"], "auto_security_group": true, "floating_ip_pool" : "04946cae-672c-4b34-9f59-3b6066cdef0f" } [root@sahara ~(keystone_admin)]# cat ng_master_template_create-hdp.json { "name": "master-hdp-206-tmpl", "flavor_id": "2", "plugin_name": "hdp", "hadoop_version": "2.0.6", "node_processes": ["NAMENODE"], "auto_security_group": true, "floating_ip_pool" : "04946cae-672c-4b34-9f59-3b6066cdef0f" } 2. [root@sahara ~(keystone_admin)]# cat cluster_template_create-hdp.json { "name": "cluster-hdp-206-template", "plugin_name": "hdp", "hadoop_version": "2.0.6", "node_groups": [ { "name": "master", "node_group_template_id": "61490289-3a18-415b-8a49-bce29155b792", "count": 1 }, { "name": "workers", "node_group_template_id": "8d6d0be4-0fc5-4640-908e-1a95f78dff93", "count": 2 } ] } 3. [root@sahara ~(keystone_admin)]# cat cluster_create-hdp.json { "name": "cluster-hdp-1", "plugin_name": "hdp", "hadoop_version": "2.0.6", "cluster_template_id" : "971f9d96-7f41-4512-84f2-f50c1ffa1608", "user_keypair_id": "sahara-key", "default_image_id": "3b398a98-3869-4718-b1df-ac71e1cb7bea", "neutron_management_network": "c7982086-7fde-4572-9d8e-a43fc71c3b6f" } Actual results: Expected results: Additional info:
The cluster is incomplete, you need much more services than a NAMENODE and a DATANODE. The information about validation on the upstream doc are outdated/for HDP 1.x (please file a bug *upstream* for that). JobTracker and TaskTracker are Hadoop 1.x only. A more complete example of HDP 2.0.6 is available as template used for integration tests: http://git.openstack.org/cgit/openstack/sahara/tree/etc/scenario/sahara-ci/hdp-2.0.6.yaml?h=stable/kilo Instead of starting from scratch, you can create a set of "default templates" with sane values. See here for more details: http://git.openstack.org/cgit/openstack/sahara/tree/sahara/db/templates/README.rst?h=stable/kilo