Bug 1298265 - CDH 5.3 scenario tests fails because jobs stay in running state
CDH 5.3 scenario tests fails because jobs stay in running state
Status: CLOSED WONTFIX
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-sahara (Show other bugs)
8.0 (Liberty)
Unspecified Unspecified
high Severity high
: ga
: 8.0 (Liberty)
Assigned To: Elise Gafford
Luigi Toscano
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-13 11:00 EST by Luigi Toscano
Modified: 2016-01-19 11:35 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-19 11:34:46 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Scenario test log of CDH 5.3.0 RHEL 7.2 image, RHEL-OSP8 (13.48 KB, text/plain)
2016-01-13 11:08 EST, Luigi Toscano
no flags Details

  None (edit)
Description Luigi Toscano 2016-01-13 11:00:35 EST
Description of problem:
Scenario tests (from master commit bce48970e81baf61a07d43cca27e1e4699ac62a0) fails with a CDH 5.3.0 RHEL 7.2 image. The cluster is created but the jobs stays in RUNNING state "forever" (raising the waiting time for jobs to 7200 seconds did not help).

Please note that the tests do *pass* for CDH 5.4.0 and HDP 2.0.6.

Version-Release number of selected component (if applicable):
openstack-sahara-common-3.0.0-5.el7ost.noarch
openstack-sahara-engine-3.0.0-5.el7ost.noarch
openstack-sahara-api-3.0.0-5.el7ost.noarch

The images have been generated with 
sahara-image-elements-3.0.0-3.el7ost.noarch

Apparently no relevant information in sahara-api and sahara-engine logs.

How reproducible:
$ python sahara/tests/scenario/runner.py --verbose -V scenario.ini etc/scenario/sahara-ci/credentials.yaml.mako etc/scenario/sahara-ci/edp.yaml.mako etc/scenario/sahara-ci/cdh-5.3.0.yaml.mako
[...]
Captured stderr:
~~~~~~~~~~~~~~~~
    /usr/lib/python2.7/site-packages/keystoneclient/service_catalog.py:196: UserWarning: Providing attr without filter_value to get_urls() is deprecated as of the 1.7.0 release and may be removed in the
 2.0.0 release. Either both should be provided or neither should be provided.
      'Providing attr without filter_value to get_urls() is '
    Traceback (most recent call last):

      File "sahara/tests/scenario/base.py", line 63, in wrapper
        return fct(self, *args, **kwargs)

      File "sahara/tests/scenario/base.py", line 198, in check_run_jobs
        self._job_batching(pre_exec)

      File "sahara/tests/scenario/base.py", line 207, in _job_batching
        self._poll_jobs_status(job_exec_ids)

      File "sahara/tests/scenario/base.py", line 314, in _poll_jobs_status
        self.fail("\n".join(report))

      File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail
        raise self.failureException(msg)

    AssertionError: Job with id=9665bb83-b8fb-40ac-9adb-c082b8a5023b, name=test-0ba01c5f, type=Pig has status RUNNING
    Job with id=c480e198-2c19-4aa6-ae43-8d4178da6ed4, name=test-b1105ee4, type=MapReduce has status RUNNING
    Job with id=6847e9ec-f13b-4c7c-b24e-56e5748a36be, name=test-7802e9f5, type=MapReduce.Streaming has status RUNNING
    Job with id=31ebcb98-9f4a-4014-9ac7-e6d0929ce761, name=test-da2b82d1, type=Java has status RUNNING



Captured stdout:
~~~~~~~~~~~~~~~~
    Results of testing plugin cdh 5.3.0
    +-----------------------------+--------+-------------+
    | Check                       | Status | Duration, s |
    +-----------------------------+--------+-------------+
    | Create node group templates |   OK   |      0      |
    | Set flavor                  |   OK   |      0      |
    | Set flavor                  |   OK   |      0      |
    | Set flavor                  |   OK   |      0      |
    | Set flavor                  |   OK   |      0      |
    | Set flavor                  |   OK   |      0      |
    | Set flavor                  |   OK   |      0      |
    | Create cluster template     |   OK   |      0      |
    | Create cluster              |   OK   |      4      |
    | Check cluster state         |   OK   |     1561    |
    | Check cinder volumes        |   OK   |      2      |
    | Check event logs            |   OK   |      0      |
    | Check EDP jobs              | FAILED |     7203    |
    | Check sentry                |   OK   |      0      |
    +-----------------------------+--------+-------------+
Comment 1 Luigi Toscano 2016-01-13 11:08 EST
Created attachment 1114445 [details]
Scenario test log of CDH 5.3.0 RHEL 7.2 image, RHEL-OSP8
Comment 2 Keith Basil 2016-01-19 11:35:11 EST
We are moving to CDH5.4.

Note You need to log in before you can comment on or make changes to this bug.