Description of problem: Hadoop already offers filesystem libraries with support for s3a:///path URIs, so we need to add support to S3-compatible object stores on Sahara. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Upstream spec can be found here: https://specs.openstack.org/openstack/sahara-specs/specs/pike/support-for-s3-compatible-object-stores.html
Tested using RHEL 7.6 for both the image and the deployment (the latter on packstack for resource issues). The official implementation of S3 (AWS) was used. The following images were tested: CDH 5.13, CDH 5.11, CDH 5.9, MapR 5.2, Ambari 2.6/HDP 2.6, Ambari 2.6/HDP 2.4. openstack-sahara-9.0.1-0.20181010221802.eb72c78.el7ost openstack-sahara-tests-0.7.0-0.20180925152245.bb23a5b.el7ost Note for self: in order to avoid an heap-related error ("Error: Java heap space") the following value was set for the S3 EDP job: mapreduce.map.java.opts = -Xmx1433m Probably it's not needed in a different environment.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045