Bug 1389386

Summary: [RFE] Allow non root GIDs ( GID >0 ) to access gluster dynamically provisioned volumes.
Product: Red Hat Gluster Storage Reporter: Humble Chirammal <hchiramm>
Component: CNS-deploymentAssignee: Raghavendra Talur <rtalur>
Status: CLOSED ERRATA QA Contact: Anoop <annair>
Severity: medium Docs Contact:
Priority: medium    
Version: cns-3.4CC: akhakhar, annair, divya, hchiramm, jrivera, kramdoss, madam, mliyazud, mzywusko, nerawat, pprakash, rcyriac, rhs-bugs, rmekala, rreddy, rtalur, storage-qa-internal
Target Milestone: ---Keywords: FutureFeature
Target Release: CNS 3.4   
Hardware: x86_64   
OS: Linux   
URL: https://github.com/openshift/origin/pull/11893
Whiteboard:
Fixed In Version: heketi-3.1.0-11.el7 Doc Type: If docs needed, set a value
Doc Text:
The gluster dynamic provisioner creates volumes which can be accessed by non-root users. This allows the non-privileged application pod to mount and use the persistent volumes created by the gluster dynamic provisioner. To use gluster dynamic provisioner, you must install heketi version 3 or higher.
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-19 15:29:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1385245    

Description Humble Chirammal 2016-10-27 13:53:36 UTC
Description of problem:

Heketi server has added support for creating volumes with a GID > 0. 
As an api client, dynamic provisioner has to make use of this new api change
and create volumes with > 0. However this need some research on which GID has to be used and other security concerns which may arise in environments like kubernetes/Openshift.

Version-Release number of selected component (if applicable):

CNS 3.4

How reproducible:

N/A

Steps to Reproduce:

N/A


Additional info:

This is a common issue for all the network FS provisioners  and need a common solution for all the plugins which provision network FS volumes.
The upstream discussions on how to use GID when dynamically provision volumes is on going and its tracked under https://github.com/kubernetes/kubernetes/pull/35460

Comment 5 Humble Chirammal 2016-11-11 07:00:24 UTC
[Status update] The subjected patch ( https://github.com/kubernetes/kubernetes/pull/35460 ) is ready to be moved to upstream k8s merge queue since 08-11-2016, however due to release milestone readiness it has not yet merged.

Comment 6 Humble Chirammal 2016-11-16 16:09:49 UTC
[Status update] The patch have "LGTM" and it has passed all the tests required for submit queue. Still waiting for the post code freeze merge.

Comment 7 Humble Chirammal 2016-11-16 18:19:11 UTC
The GID patch  (https://github.com/kubernetes/kubernetes/pull/35460 ) is merged in upstream K8s 1.5 tree !

Comment 9 Humble Chirammal 2016-11-17 19:18:12 UTC
(In reply to Humble Chirammal from comment #8)
> WIP Origin PRs.
> 
> 1) https://github.com/openshift/origin/pull/11893
> 2) https://github.com/openshift/origin/pull/11901

A consolidated Origin PR for this feature is available @ https://github.com/openshift/origin/pull/11893 . It has passed all the tests as well.

Comment 11 Humble Chirammal 2016-12-05 10:12:46 UTC
(In reply to Humble Chirammal from comment #9)
> (In reply to Humble Chirammal from comment #8)
> > WIP Origin PRs.
> > 
> > 1) https://github.com/openshift/origin/pull/11893
> > 2) https://github.com/openshift/origin/pull/11901
> 
> A consolidated Origin PR for this feature is available @
> https://github.com/openshift/origin/pull/11893 . It has passed all the tests
> as well.

The new PR for this feature is available @

Origin 1.4 : https://github.com/openshift/origin/pull/12131 ( Merged )
Origin Master : https://github.com/openshift/origin/pull/12135 ( Waiting for the merge )

Comment 14 Michael Adam 2016-12-08 17:46:19 UTC
Build has been provided.

Comment 15 krishnaram Karthick 2016-12-29 06:47:27 UTC
Mountpoint now has 775 permission, however directories created under the mountpoint has 755 permission.

sh-4.2$ ls -ld /var/lib/mongodb/data
drwxrwxr-x. 7 root 2001 4096 Dec 29 05:55 /var/lib/mongodb/data
sh-4.2$ ls -ld /var/lib/mongodb/data/krk
drwxr-xr-x. 2 1000060000 root 4096 Dec 28 14:09 /var/lib/mongodb/data/krk

I don't see any stickybit set for the sub-directories. 

@Humble, can you please confirm if the patch for stickybit is available dowstream?

Comment 16 Humble Chirammal 2016-12-29 06:52:47 UTC
(In reply to krishnaram Karthick from comment #15)
> Mountpoint now has 775 permission, however directories created under the
> mountpoint has 755 permission.
> 
> sh-4.2$ ls -ld /var/lib/mongodb/data
> drwxrwxr-x. 7 root 2001 4096 Dec 29 05:55 /var/lib/mongodb/data
> sh-4.2$ ls -ld /var/lib/mongodb/data/krk
> drwxr-xr-x. 2 1000060000 root 4096 Dec 28 14:09 /var/lib/mongodb/data/krk
> 
> I don't see any stickybit set for the sub-directories. 
> 
> @Humble, can you please confirm if the patch for stickybit is available
> dowstream?

As discussed over irc, we are not setting sticky bit what we do is 'setGid',

for ex:
-			fmt.Sprintf("chmod 775 %v/brick", mountpoint),
+			fmt.Sprintf("chmod 2775 %v/brick", mountpoint),

I feel, the patch is missing in downstream. Cross checking the same.

Comment 17 Humble Chirammal 2016-12-29 06:55:06 UTC
Cross check the PR https://github.com/heketi/heketi/pull/612/ availability in downstream build.

Comment 18 Mohamed Ashiq 2016-12-29 07:01:53 UTC
(In reply to Humble Chirammal from comment #17)
> Cross check the PR https://github.com/heketi/heketi/pull/612/ availability
> in downstream build.

Above PR was closed with this new PR:
https://github.com/heketi/heketi/pull/617

Comment 19 Humble Chirammal 2016-12-29 07:03:47 UTC
Can you please create a file in /var/lib/mongodb/data/krk and provide the ls -l on it ?

Comment 20 krishnaram Karthick 2016-12-29 07:07:58 UTC
(In reply to Humble Chirammal from comment #19)
> Can you please create a file in /var/lib/mongodb/data/krk and provide the ls
> -l on it ?

sh-4.2$ pwd
/var/lib/mongodb/data/krk
sh-4.2$
sh-4.2$ ls -l
total 0
-rw-r--r--. 1 1000060000 root 0 Dec 28 14:09 file

Moving the bug to 'assigned' for now based on the above comments.

Comment 21 Humble Chirammal 2016-12-29 19:58:53 UTC
Fixed in heketi-3.1.0-11.el7 .
https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=530094

Comment 22 krishnaram Karthick 2017-01-02 14:47:18 UTC
Verified the bug with heketi build - 3.1.0-11.el7

 - Mountpoint has 775 permission
 - directories created under mountpoint has 2755 permission
 - two pods with same userid can write into directories created by each other
 - two pods with different userid cannot write into directories created by each other pod
 - non-privileged container able to do IO
 - group id assigned to a pvc gets reclaimed when deleted
 - gid range can be provided in storage class definition
 - proper error message is thrown when gid limit defined in storageclass exceeds the limit

Based on the above test results, moving the bug to verified.

Comment 24 Raghavendra Talur 2017-01-09 07:59:13 UTC
Doc text looks good.

Comment 25 Humble Chirammal 2017-01-09 08:23:26 UTC
LGTM.

Comment 27 errata-xmlrpc 2017-01-19 15:29:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2017-0169.html