Description of problem: Scenario tests (from master commit bce48970e81baf61a07d43cca27e1e4699ac62a0) fails with a CDH 5.3.0 RHEL 7.2 image. The cluster is created but the jobs stays in RUNNING state "forever" (raising the waiting time for jobs to 7200 seconds did not help). Please note that the tests do *pass* for CDH 5.4.0 and HDP 2.0.6. Version-Release number of selected component (if applicable): openstack-sahara-common-3.0.0-5.el7ost.noarch openstack-sahara-engine-3.0.0-5.el7ost.noarch openstack-sahara-api-3.0.0-5.el7ost.noarch The images have been generated with sahara-image-elements-3.0.0-3.el7ost.noarch Apparently no relevant information in sahara-api and sahara-engine logs. How reproducible: $ python sahara/tests/scenario/runner.py --verbose -V scenario.ini etc/scenario/sahara-ci/credentials.yaml.mako etc/scenario/sahara-ci/edp.yaml.mako etc/scenario/sahara-ci/cdh-5.3.0.yaml.mako [...] Captured stderr: ~~~~~~~~~~~~~~~~ /usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py:196: UserWarning: Providing attr without filter_value to get_urls() is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. Either both should be provided or neither should be provided. 'Providing attr without filter_value to get_urls() is ' Traceback (most recent call last): File "sahara/tests/scenario/base.py", line 63, in wrapper return fct(self, *args, **kwargs) File "sahara/tests/scenario/base.py", line 198, in check_run_jobs self._job_batching(pre_exec) File "sahara/tests/scenario/base.py", line 207, in _job_batching self._poll_jobs_status(job_exec_ids) File "sahara/tests/scenario/base.py", line 314, in _poll_jobs_status self.fail("\n".join(report)) File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail raise self.failureException(msg) AssertionError: Job with id=9665bb83-b8fb-40ac-9adb-c082b8a5023b, name=test-0ba01c5f, type=Pig has status RUNNING Job with id=c480e198-2c19-4aa6-ae43-8d4178da6ed4, name=test-b1105ee4, type=MapReduce has status RUNNING Job with id=6847e9ec-f13b-4c7c-b24e-56e5748a36be, name=test-7802e9f5, type=MapReduce.Streaming has status RUNNING Job with id=31ebcb98-9f4a-4014-9ac7-e6d0929ce761, name=test-da2b82d1, type=Java has status RUNNING Captured stdout: ~~~~~~~~~~~~~~~~ Results of testing plugin cdh 5.3.0 +-----------------------------+--------+-------------+ | Check | Status | Duration, s | +-----------------------------+--------+-------------+ | Create node group templates | OK | 0 | | Set flavor | OK | 0 | | Set flavor | OK | 0 | | Set flavor | OK | 0 | | Set flavor | OK | 0 | | Set flavor | OK | 0 | | Set flavor | OK | 0 | | Create cluster template | OK | 0 | | Create cluster | OK | 4 | | Check cluster state | OK | 1561 | | Check cinder volumes | OK | 2 | | Check event logs | OK | 0 | | Check EDP jobs | FAILED | 7203 | | Check sentry | OK | 0 | +-----------------------------+--------+-------------+
Created attachment 1114445 [details] Scenario test log of CDH 5.3.0 RHEL 7.2 image, RHEL-OSP8
We are moving to CDH5.4.