Bug 1005189

Summary:	nfs-root-squash: rename of a new file returns "Permission denied" for a non-root user
Product:	Red Hat Gluster Storage	Reporter:	Saurabh <saujain>
Component:	distribute	Assignee:	Bug Updates Notification Mailing List <rhs-bugs>
Status:	CLOSED DEFERRED	QA Contact:	storage-qa-internal <storage-qa-internal>
Severity:	urgent	Docs Contact:
Priority:	unspecified
Version:	2.1	CC:	mzywusko, rhs-bugs, spalai, vagarwal, vbellur
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1286158 (view as bug list)		Environment:
Last Closed:	2015-11-27 12:05:02 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	1286158

Description Saurabh 2013-09-06 11:47:32 UTC

Description of problem:
set up is like I am logged into a client with a non-root user.
on a nfs mount I have a directory for this user.
If I try to rename a file it fails.

This is behaviour is seen when root squash is on.

Volume Name: dr1
Type: Distributed-Replicate
Volume ID: 40922fbc-a3a4-4760-a17e-73ce4cc4423f
Status: Started
Number of Bricks: 6 x 2 = 12
Transport-type: tcp
Bricks:
Brick1: rhsauto032.lab.eng.blr.redhat.com:/rhs/bricks/d1r1
Brick2: rhsauto033.lab.eng.blr.redhat.com:/rhs/bricks/d1r2
Brick3: rhsauto034.lab.eng.blr.redhat.com:/rhs/bricks/d2r1
Brick4: rhsauto035.lab.eng.blr.redhat.com:/rhs/bricks/d2r2
Brick5: rhsauto032.lab.eng.blr.redhat.com:/rhs/bricks/d3r1
Brick6: rhsauto033.lab.eng.blr.redhat.com:/rhs/bricks/d3r2
Brick7: rhsauto034.lab.eng.blr.redhat.com:/rhs/bricks/d4r1
Brick8: rhsauto035.lab.eng.blr.redhat.com:/rhs/bricks/d4r2
Brick9: rhsauto032.lab.eng.blr.redhat.com:/rhs/bricks/d5r1
Brick10: rhsauto033.lab.eng.blr.redhat.com:/rhs/bricks/d5r2
Brick11: rhsauto034.lab.eng.blr.redhat.com:/rhs/bricks/d6r1
Brick12: rhsauto035.lab.eng.blr.redhat.com:/rhs/bricks/d6r2
Options Reconfigured:
server.root-squash: on
features.alert-time: 1s
features.quota-deem-statfs: on
features.default-soft-limit: 50%
features.quota: on


Version-Release number of selected component (if applicable):
glusterfs-3.4.0.31rhs-1.el6rhs.x86_64

How reproducible:
always

Steps to Reproduce:
1. create a volume, start it
2. mount it over nfs
3. create a dir.
4. chown the dir permission for non-root user
4a. enable root squash
5. login to system client as the non-root user
6. create a file
7. rename the file.

Actual results:
[saurabh@rhsauto036 saurabh-dir]$ touch f.n1
[saurabh@rhsauto036 saurabh-dir]$ mv f.n1 f.n1-rename
mv: cannot move `f.n1' to `f.n1-rename': Permission denied

Expected results:
the rename should happen, as with root-squash off it does work

Additional info:

nfs.log reports this,

[2013-09-06 08:52:40.024472] I [glusterfsd-mgmt.c:1559:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2013-09-06 08:52:40.028243] I [glusterfsd-mgmt.c:1559:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2013-09-06 08:52:40.029992] I [glusterfsd-mgmt.c:1559:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing
[2013-09-06 08:52:52.056620] W [client-rpc-fops.c:256:client3_3_mknod_cbk] 0-dr1-client-2: remote operation failed: Permission denied. Path: /saurabh-dir/f.n1
[2013-09-06 08:52:52.056902] W [client-rpc-fops.c:256:client3_3_mknod_cbk] 0-dr1-client-3: remote operation failed: Permission denied. Path: /saurabh-dir/f.n1
[2013-09-06 08:52:52.057343] W [client-rpc-fops.c:2453:client3_3_link_cbk] 0-dr1-client-10: remote operation failed: Permission denied (/saurabh-dir/f.n1 -> /saurabh-dir/f.n1-rename)
[2013-09-06 08:52:52.057477] W [client-rpc-fops.c:2453:client3_3_link_cbk] 0-dr1-client-11: remote operation failed: Permission denied (/saurabh-dir/f.n1 -> /saurabh-dir/f.n1-rename)
[2013-09-06 08:52:52.062617] W [dht-rename.c:365:dht_rename_unlink_cbk] 0-dr1-dht: /saurabh-dir/f.n1: unlink on dr1-replicate-1 failed (No such file or directory)
[2013-09-06 08:52:52.063807] W [dht-rename.c:365:dht_rename_unlink_cbk] 0-dr1-dht: /saurabh-dir/f.n1: unlink on dr1-replicate-5 failed (No such file or directory)
[2013-09-06 08:52:52.064049] W [nfs3.c:3663:nfs3svc_rename_cbk] 0-nfs: 9236d348: rename /saurabh-dir/f.n1 -> /saurabh-dir/f.n1-rename => -1 (Permission denied)

Comment 6 rjoseph 2013-09-10 13:29:09 UTC

I just checked the behavior, the rename is not failing all the time. 

It is failing only when the rename operation will cause the file to move from one brick to another due to DHT. Actually the file is not moved physically but a link file is created in the target brick.

As per my discussion with Shishir the link file is created with root permission. If this link file creation is subjected to root-squashing then the rename will fail with "permission denied".

Comment 8 Saurabh 2014-11-26 11:15:01 UTC

Similar issue
[qa1@rhsauto015 ~]$ mv /mnt/nfs-test/qa1-user-dir/f.1 /mnt/nfs-test/qa1-user-dir/dir1/f.1-new
mv: cannot move `/mnt/nfs-test/qa1-user-dir/f.1' to `/mnt/nfs-test/qa1-user-dir/dir1/f.1-new': Permission denied
[qa1@rhsauto015 ~]$


Happens on glusterfs-3.6.0.33-1.el6rhs.x86_64