Bug 1547710

Summary:	Support for S3-compatible object stores
Product:	Red Hat OpenStack	Reporter:	Telles Nobrega <tenobreg>
Component:	openstack-sahara	Assignee:	Telles Nobrega <tenobreg>
Status:	CLOSED ERRATA	QA Contact:	Luigi Toscano <ltoscano>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	14.0 (Rocky)	CC:	cschwede, jamsmith, matt, mimccune, pkshiras, scohen, srevivo
Target Milestone:	Upstream M2	Keywords:	FutureFeature, Triaged
Target Release:	14.0 (Rocky)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-sahara-9.0.0-0.20180709181524.6838a21.el7ost	Doc Type:	Enhancement
Doc Text:	This update adds support of s3-compatible object stores for OpenStack Sahara.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-01-11 11:48:58 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1636970

Description Telles Nobrega 2018-02-21 19:28:00 UTC

Description of problem:

Hadoop already offers filesystem libraries with support for s3a:///path URIs, so we need to add support to S3-compatible object stores on Sahara.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Telles Nobrega 2018-02-21 19:31:41 UTC

Upstream spec can be found here: 
https://specs.openstack.org/openstack/sahara-specs/specs/pike/support-for-s3-compatible-object-stores.html

Comment 7 Luigi Toscano 2018-11-21 16:34:15 UTC

Tested using RHEL 7.6 for both the image and the deployment (the latter on packstack for resource issues). The official implementation of S3 (AWS) was used.
The following images were tested: CDH 5.13, CDH 5.11, CDH 5.9, MapR 5.2, Ambari 2.6/HDP 2.6, Ambari 2.6/HDP 2.4.

openstack-sahara-9.0.1-0.20181010221802.eb72c78.el7ost
openstack-sahara-tests-0.7.0-0.20180925152245.bb23a5b.el7ost

Note for self: in order to avoid an heap-related error ("Error: Java heap space")
the following value was set for the S3 EDP job: 
mapreduce.map.java.opts = -Xmx1433m
Probably it's not needed in a different environment.

Comment 9 errata-xmlrpc 2019-01-11 11:48:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045