Bug 1217107 - [rfe] Ambari (HDP 2.3 stack) should configure hive warehouse to work with hcfs
Summary: [rfe] Ambari (HDP 2.3 stack) should configure hive warehouse to work with hcfs
Keywords:
Status: CLOSED EOL
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: rhs-hadoop
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: RHGS 3.1.0
Assignee: Erin Boyd
QA Contact: BigData QE
URL:
Whiteboard:
Depends On:
Blocks: 1202842
TreeView+ depends on / blocked
 
Reported: 2015-04-29 14:47 UTC by Martin Bukatovic
Modified: 2016-02-01 16:16 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-01 16:16:11 UTC
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1118806 1 None None None 2024-09-18 00:37:29 UTC

Internal Links: 1118806

Description Martin Bukatovic 2015-04-29 14:47:10 UTC
Description of problem
======================

Based on the BZ 1118806 (see Brad's comment 30 there), additional
configuration is required for Hive to work properly:

> Solution to HIVE issue:
> 
> HIVE will propagate permissions with this:
> 
> set hive.warehouse.subdir.inherit.perms = true as custom property in hive-site.xml during ambari configuration.
> 
> permission of glusterfs:///apps/hive/warehouse should be 0775
>
> HIVE will propagate parent 775 permissions to the warehouse sub directories

Since the installer aready creates warehouse directory, it makes sense to
implement this Hive configuration in the installer.

Version-Release number of selected component (if applicable)
============================================================

rhs-hadoop-install-2_47-1.el6rhs.noarch

Steps to Reproduce
==================

1. install rhs-hadoop cluster using rhs-hadoop-install scripts
2. make sure you install Hive (when Hadoop cluster is created in Ambari)

Actual results
==============

Hive property `hive.warehouse.subdir.inherit.perms` is undefined.

Access rights on warehouse directory are set to 755:

~~~
$ hadoop fs -ls -d /apps/hive/warehouse 2>/dev/null
drwxr-xr-x   - hive hadoop         30 2015-04-29 13:48 /apps/hive/warehouse
~~~

Expected results
================

Hive property `hive.warehouse.subdir.inherit.perms` is set to true.

Access rights on warehouse directory are set to 775:

~~~
$ hadoop fs -ls -d /apps/hive/warehouse 2>/dev/null
drwxrwxr-x   - hive hadoop         30 2015-04-29 13:48 /apps/hive/warehouse
~~~

Additional info
===============

To properly verify this issue, run Hive reproducer from comment 2 of BZ 1118806.

Comment 1 Jeff Vance 2015-04-29 15:18:15 UTC
The installer does set perms/owner on required hadoop directories, and it also sets 2 core-site properties related to enabling/disabling volumes. However, most of the core-site, yarn-site properties that are special for rhs are set by Ambari. Why shouldn't the property mentioned in this BZ also be set by Ambari?

Comment 2 Jeff Vance 2015-04-29 16:24:11 UTC
per conversation with Erin I transferred this to Ambari.

Comment 3 Martin Bukatovic 2015-04-30 06:30:10 UTC
(In reply to Jeff Vance from comment #1)
> The installer does set perms/owner on required hadoop directories, and it
> also sets 2 core-site properties related to enabling/disabling volumes.
> However, most of the core-site, yarn-site properties that are special for
> rhs are set by Ambari. Why shouldn't the property mentioned in this BZ also
> be set by Ambari?

That's a good point, you are right. Installer should do this only if we fail
to add hive.warehouse.subdir.inherit.perms property into GlusterFS stack in
Ambari.

Comment 4 Erin Boyd 2015-06-02 06:00:01 UTC
Fixed in 2.3

Comment 7 Steve Watt 2016-02-01 16:16:11 UTC
This solution is no longer available from Red Hat.


Note You need to log in before you can comment on or make changes to this bug.