Bug 764285 (GLUSTER-2553)

Summary: NFS file create with Mac client and UNCHECKED mode returns "Operation not permitted"
Product: [Community] GlusterFS Reporter: Vikas Gorur <vikas>
Component: nfsAssignee: Shehjar Tikoo <shehjart>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: mainlineCC: gluster-bugs, saurabh
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: RTP Mount Type: nfs
Documentation: DNR CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:

Description Vikas Gorur 2011-03-17 17:19:29 EDT
Interestingly, this seems to have nothing to do with the UNCHECKED mode itself. Here is a Wireshark trace from an NFS mount on the Linux server itself:

55	14.465616	10.1.10.143	10.1.10.143	NFS	V3 CREATE Call, DH:0xf78306ce/bar Mode:UNCHECKED
61	14.466880	10.1.10.143	10.1.10.143	NFS	V3 CREATE Reply (Call In 55)

So something else is different in the Mac case.
Comment 1 Vikas Gorur 2011-03-17 17:28:35 EDT
Another important piece of info:

The file creation command ("touch") is being run as a non-root user (vikas). The UID of this user is the same (numerically) on the Mac client as well as the Linux server.

The root user has none of these problems.

Mac version: 10.6.3
Gluster version: Latest git.
Comment 2 Vikas Gorur 2011-03-17 20:13:53 EDT
Gluster volume mounted on Mac using NFS:

10.1.10.143:/one on /Volumes/one (nfs)

Trying to create a file:

$ touch foo
touch: foo: Operation not permitted
$ ls -l
total 0
-rw-r--r--  1 vikas  staff  0 Mar 17 17:05 foo

This is because the CREATE itself succeeds, but the setattr that the nfs xlator generates fails:

From the brick log:

[2011-03-17 17:05:49.926925] E [access-control.c:1842:ac_setattr_stat_cbk] one-access-control: setattr failed with error: Operation not permitted
[2011-03-17 17:05:49.926935] I [server3_1-fops.c:1527:server_setattr_cbk] one-server: 182: SETATTR /vikas/foo (557061) ==> -1 (Operation not permitted)

Debugging a little with Wireshark shows that:

90327	5094.317616	10.1.30.211	10.1.10.143	NFS	V3 CREATE Call, DH:0xf78306ce/foo Mode:UNCHECKED
90328	5094.320294	10.1.10.143	10.1.30.211	NFS	V3 CREATE Reply (Call In 90327) Error:NFS3ERR_PERM

Note that this is when file creation is attempted from the terminal. If I try to copy a file to the NFS directory using Finder,

94192	5503.276489	10.1.30.211	10.1.10.143	NFS	V3 CREATE Call, DH:0xf78306ce/ami-build.sh Mode:EXCLUSIVE
94197	5503.279119	10.1.10.143	10.1.30.211	NFS	V3 CREATE Reply (Call In 94192)

So it appears that creates succeed if the mode is EXCLUSIVE, and not when mode is UNCHECKED.
Comment 3 Shehjar Tikoo 2011-03-18 02:01:52 EDT
Can I get access to the Mac?
Comment 4 Vikas Gorur 2011-03-18 11:31:42 EDT
This test was done from my laptop, so access is a little difficult. Can you try reproducing this with an available Mac?
Comment 5 Shehjar Tikoo 2011-03-18 12:37:21 EDT
OK, i'll see if the one here is free. BTW, if this was just your own test and not a customer bug, can i treat this as lower prio?
Comment 6 Shehjar Tikoo 2011-03-21 02:27:27 EDT
My bad. I see your patch is correct. Did it actually fix this problem?
Comment 7 Vikas Gorur 2011-03-21 02:30:30 EDT
It doesn't fix anything. While debugging this issue I saw this piece of redundant code and therefore sent a patch. As you can see, removing it or keeping it there does nothing.

So this patch just removes some dead code.
Comment 8 Vijay Bellur 2011-03-25 04:11:37 EDT
PATCH: http://patches.gluster.com/patch/6507 in master (nfs: Remove redundant and erroneous check for create mode EXCLUSIVE.)
Comment 9 Vijay Bellur 2011-03-29 05:33:05 EDT
PATCH: http://patches.gluster.com/patch/6620 in release-3.1 (nfs: capture attrs of create request in cs->stbuf for later use)
Comment 10 Vijay Bellur 2011-03-30 03:24:36 EDT
PATCH: http://patches.gluster.com/patch/6640 in release-3.1 (nfs: more fixes to capture sattr for further usage)
Comment 11 Vijay Bellur 2011-03-30 03:24:41 EDT
PATCH: http://patches.gluster.com/patch/6623 in release-3.1 (nfs: Remove redundant and erroneous check for create mode EXCLUSIVE)
Comment 12 Vijay Bellur 2011-03-30 04:28:32 EDT
PATCH: http://patches.gluster.com/patch/6639 in master (nfs: more fixes to capture sattr for further usage)
Comment 13 Vijay Bellur 2011-04-01 06:08:22 EDT
PATCH: http://patches.gluster.com/patch/6621 in master (nfs: capture attrs of create request in cs->stbuf for later use)
Comment 14 Vijay Bellur 2011-04-08 02:35:54 EDT
PATCH: http://patches.gluster.com/patch/6748 in release-3.1 (NFS : Exclusive Create storing verifier.)
Comment 15 Vijay Bellur 2011-04-08 03:26:58 EDT
PATCH: http://patches.gluster.com/patch/6749 in master (NFS : Exclusive create storing verifier.)
Comment 16 Anand Avati 2011-06-08 11:17:05 EDT
PATCH: http://patches.gluster.com/patch/7412 in master (NFS : Don't set setattr_valid for exclusive create call.)
Comment 17 Anand Avati 2011-06-08 11:17:46 EDT
PATCH: http://patches.gluster.com/patch/7414 in release-3.1 (NFS : Don't set setattr_valid for exclusive create call.)
Comment 18 Anand Avati 2011-06-08 11:18:10 EDT
PATCH: http://patches.gluster.com/patch/7413 in release-3.2 (NFS : Don't set setattr_valid for exclusive create call.)