Bug 1571382 - CDH 5.7/5.9/5.11 images generated by sahara-image-pack do not properly execute jobs
Summary: CDH 5.7/5.9/5.11 images generated by sahara-image-pack do not properly execut...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-sahara
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: rc
: 13.0 (Queens)
Assignee: Telles Nobrega
QA Contact: Luigi Toscano
URL:
Whiteboard:
Depends On:
Blocks: 1502840 1502843 1502845
TreeView+ depends on / blocked
 
Reported: 2018-04-24 16:13 UTC by Luigi Toscano
Modified: 2018-06-27 13:55 UTC (History)
7 users (show)

Fixed In Version: openstack-sahara-8.0.1-0.20180328233739.36531cb.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-06-27 13:53:37 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack Storyboard 2001909 0 None None None 2018-04-24 16:13:31 UTC
OpenStack gerrit 566188 0 None master: MERGED sahara: Fix the installation of the Swift Hadoop connector (CDH) (Ibc0a5f7348365f2a6c5658846ee74aa92e78314b) 2018-05-07 15:41:43 UTC
Red Hat Product Errata RHEA-2018:2086 0 None None None 2018-06-27 13:55:00 UTC

Description Luigi Toscano 2018-04-24 16:13:32 UTC
Description of problem:
CDH images, for all those 3 versions, generated by sahara-image-pack and based on CentOS7.5/RHEL 7.5 returns errors when running the scenario tests. The same images generated by sahara-image-create from sahara-image-elements work. This was observed on Queens but it is probably the same on master, where the code did not diverge.

From the oozie logs it's not clear where the issue lies.

Excerpt of the logs:

The generated test file located at: /tmp/tmpM_lamv
/usr/lib/python2.7/site-packages/os_testr/ostestr.py:120: UserWarning: No .stestr.conf file found in the CWD. Please create one to to replace the .testr.conf. You can find a script to do this in the stestr git rep
ository.
  warnings.warn(msg)
Results of testing plugin cdh 5.11.0
20180424_111124
Traceback (most recent call last):

  File "/usr/lib/python2.7/site-packages/sahara_tests/scenario/base.py", line 62, in wrapper
    return fct(self, *args, **kwargs)

  File "/usr/lib/python2.7/site-packages/sahara_tests/scenario/base.py", line 222, in check_run_jobs
    self._job_batching(pre_exec)

  File "/usr/lib/python2.7/site-packages/sahara_tests/scenario/base.py", line 232, in _job_batching
    self._poll_jobs_status(job_exec_ids)

  File "/usr/lib/python2.7/site-packages/sahara_tests/scenario/base.py", line 343, in _poll_jobs_status
    self.fail("\n".join(report))

  File "/usr/lib/python2.7/site-packages/unittest2/case.py", line 666, in fail
    raise self.failureException(msg)

AssertionError: Job with id=1ba87077-0469-4a42-aef7-f2fd495fbe05, name=test-a62f378f, type=Pig has status KILLED
Job with id=f4a1ffa8-09b2-4c36-9366-b372fc8582ce, name=test-237e56d4, type=MapReduce has status KILLED
Job with id=c1fa2764-0cd1-4309-8bc4-5e58c3871be6, name=test-8971ee1b, type=MapReduce.Streaming has status KILLED
Job with id=c3b97cc8-4a39-444c-8efb-0a86037bedb6, name=test-7249054e, type=Spark has status DONEWITHERROR


+-----------------------------+--------+-------------+----------------------------+
| Check                       | Status | Duration, s |         Start time         |
+-----------------------------+--------+-------------+----------------------------+
| Create node group templates |   OK   |      5      | 2018-04-24 10:54:46.109545 |
| Set flavor                  |   OK   |      0      | 2018-04-24 10:54:46.546975 |
| Set flavor                  |   OK   |      0      | 2018-04-24 10:54:49.868471 |
| Set flavor                  |   OK   |      0      | 2018-04-24 10:54:50.160210 |
| Set flavor                  |   OK   |      0      | 2018-04-24 10:54:50.852614 |
| Set flavor                  |   OK   |      0      | 2018-04-24 10:54:51.161623 |
| Set flavor                  |   OK   |      0      | 2018-04-24 10:54:51.478533 |
| Create cluster template     |   OK   |      1      | 2018-04-24 10:54:51.787267 |
| Create cluster              |   OK   |      4      | 2018-04-24 10:54:53.406396 |
| Check cluster state         |   OK   |     813     | 2018-04-24 10:54:58.362200 |
| Check cinder volumes        |   OK   |      1      | 2018-04-24 11:08:32.355995 |
| Check event logs            |   OK   |      0      | 2018-04-24 11:08:33.584707 |
| Check EDP jobs              | FAILED |     171     | 2018-04-24 11:08:33.585016 |
| Check sentry                |   OK   |      0      | 2018-04-24 11:11:24.724782 |
+-----------------------------+--------+-------------+----------------------------+



Observed with :
openstack-sahara-8.0.1-0.20180328233736.36531cb.el7ost
openstack-sahara-tests-0.6.0-2.el7ost

Comment 6 Luigi Toscano 2018-05-10 09:56:10 UTC
The generated images now contain the Swift/Hadoop jar by Sahara and the tests pass.

Verified (for all 3 CDH versions) with:
openstack-sahara-8.0.1-0.20180328233739.36531cb.el7ost
openstack-sahara-tests-0.6.0-2.el7ost

Comment 9 errata-xmlrpc 2018-06-27 13:53:37 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:2086


Note You need to log in before you can comment on or make changes to this bug.