Bug 1189517 – [RFE][sahara]: Allow placeholders in datasource URLs

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1189517 - [RFE][sahara]: Allow placeholders in datasource URLs

Summary:	[RFE][sahara]: Allow placeholders in datasource URLs

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat OpenStack
Classification:	Red Hat
Component:	openstack-sahara (Show other bugs)
Sub Component:
Version:	unspecified
Hardware:	Unspecified Unspecified

Priority	high Severity high
Target Milestone:	beta
Target Release:	8.0 (Liberty)
Assigned To:	Elise Gafford
QA Contact:	Luigi Toscano
Docs Contact:

URL:	https://blueprints.launchpad.net/saha...
Whiteboard:	upstream_milestone_liberty-1 upstream...
Keywords:	FutureFeature

Depends On:
Blocks:
	Show dependency tree / graph

Reported:	2015-02-05 08:50 EST by RHOS Integration
Modified:	2016-04-07 17:00 EDT (History)
CC List:	7 users (show)

See Also:
Fixed In Version:	openstack-sahara-3.0.0-3.el7ost
Doc Type:	Enhancement
Doc Text:	When creating a job template intended for re-use, you can now register a variable for datasource URLs with OpenStack Data Processing (sahara). Doing so allows you to easily change input and output paths per run, rather than an actual URL (which would require revising the template, or manually revising the URL per run between jobs). This makes it easier to reuse job templates when data source jobs are mutable between runs, as is true for most real-world cases.
Story Points:	---
Clone Of:
Environment:
Last Closed:	2016-04-07 17:00:33 EDT
Type:	---
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Launchpad	1560890	None	None	None	2016-04-01 13:01 EDT
Red Hat Product Errata	RHEA-2016:0603	normal	SHIPPED_LIVE	Red Hat OpenStack Platform 8 Enhancement Advisory	2016-04-07 20:53:53 EDT

Groups: None (edit)

Description RHOS Integration 2015-02-05 08:50:20 EST

Cloned from launchpad blueprint https://blueprints.launchpad.net/sahara/+spec/edp-datasource-placeholders.

Description:

Common use case: user wants to run EDP job two times. Now the only way to do that with the same data sources is to erase result of the first run before running job the second time. Allowing to have random part in URL will allow to use output with random suffix.

Specification URL (additional information):

None

Comment 6 Elise Gafford 2015-11-19 10:45:04 EST

Feature is implemented in upstream L.

Comment 8 Luigi Toscano 2016-04-01 13:01:15 EDT

The placeholders in datasources (%RANDSTR(<len>)%, %JOB_EXEC_ID%) are correctly replaced according the spec for both HDFS and Swift data sources. 

There is only a minor issue: when <len> is really big, for example 2^32+1 but also smaller values, something goes wrong and the job stays in PREPARE state. But this is really a corner case and it comes from a deliberate choice of the user that defines the data source, who can easily avoid the problem.
Therefore I don't think this is a blocker for the general availability of the feature.
The issue has been captured in the following upstream bug:
https://bugs.launchpad.net/sahara/+bug/1560890

Verified on an up-to-date RHEL 7.2, with
openstack-sahara-api-3.0.1-4.el7ost.noarch
openstack-sahara-common-3.0.1-4.el7ost.noarch
openstack-sahara-engine-3.0.1-4.el7ost.noarch
python-sahara-3.0.1-4.el7ost.noarch
python-sahara-tests-3.0.1-4.el7ost.noarch

Comment 10 errata-xmlrpc 2016-04-07 17:00:33 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0603.html

Note You need to log in before you can comment on or make changes to this bug.