Hide Forgot
I forgot the entry for the LDAP user lookup showing the primary group: user1@wall-e:/mnt/cvol00/share$ getent passwd user1 user1:x:10001:11001:Test User:/home/user1:/bin/bash user1@wall-e:/mnt/cvol00/share$ getent group staff staff:*:11001:user1 user1@wall-e:/mnt/cvol00/share$ getent group admins admins:*:11000:admin,user1
When using LDAP with a user that has multiple group memberships, only the primary group is honored when trying to write to an NFS or Gluster mounted volume. Here is an excerpt from my system: user1@wall-e:/mnt/cvol00$ groups staff admins vcs user1@wall-e:/mnt/cvol00$ ls -lah total 28K drwxr-xr-x 4 root root 8.0K 2010-10-29 11:35 . drwxr-xr-x 3 root root 4.0K 2010-10-27 14:28 .. drwxrwsr-x 2 root staff 8.0K 2010-10-29 11:38 share user1@wall-e:/mnt/cvol00$ touch share/test1 user1@wall-e:/mnt/cvol00$ ls -lah share/ total 24K drwxrwsr-x 2 root staff 8.0K 2010-10-29 11:48 . drwxr-xr-x 4 root root 8.0K 2010-10-29 11:35 .. -rw-r--r-- 1 user1 staff 0 2010-10-29 11:48 test1 user1@wall-e:/mnt/cvol00$ sudo chgrp -R admins share/ user1@wall-e:/mnt/cvol00$ sudo chmod -R 2775 share/ user1@wall-e:/mnt/cvol00$ ls -lah total 28K drwxr-xr-x 4 root root 8.0K 2010-10-29 11:35 . drwxr-xr-x 3 root root 4.0K 2010-10-27 14:28 .. drwxrwsr-x 2 root admins 8.0K 2010-10-29 11:48 share user1@wall-e:/mnt/cvol00$ touch share/test2 touch: cannot touch `share/test2': Permission denied In the gluster brick log I see it only looking up the user's primary group ID (of the group 'staff'): [2010-10-29 11:54:50.472210] T [auth-glusterfs.c:176:auth_glusterfs_authenticate] rpc-service: Auth Info: pid: 0, uid: 10001, gid: 11001, owner: 30 It never changes from "gid: 11001" which is the user's primary group ID.
I added some quick debug messages to find out if gluster was reporting the correct group ids -- forgive my "C" ignorance -- I'm horribly in-experienced: [2010-11-01 17:03:18.389750] T [access-control.c:216:ac_test_access] access-control: Testing owner access [2010-11-01 17:03:18.389763] T [access-control.c:226:ac_test_access] access-control: Testing group access [2010-11-01 17:03:18.389775] T [access-control.c:126:ac_test_group_access] access-control: Group found [2010-11-01 17:03:18.389787] T [access-control.c:129:ac_test_group_access] access-control: Group (owner) ID: 11000 Group ID: 0 [2010-11-01 17:03:18.389808] T [access-control.c:126:ac_test_group_access] access-control: Group found [2010-11-01 17:03:18.389820] T [access-control.c:129:ac_test_group_access] access-control: Group (owner) ID: 11000 Group ID: 0 [2010-11-01 17:03:18.389841] T [access-control.c:126:ac_test_group_access] access-control: Group found [2010-11-01 17:03:18.389852] T [access-control.c:129:ac_test_group_access] access-control: Group (owner) ID: 11000 Group ID: 0 [2010-11-01 17:03:18.389874] T [access-control.c:126:ac_test_group_access] access-control: Group found [2010-11-01 17:03:18.389885] T [access-control.c:129:ac_test_group_access] access-control: Group (owner) ID: 11000 Group ID: 0 [2010-11-01 17:03:18.389906] T [access-control.c:238:ac_test_access] access-control: Testing other access [2010-11-01 17:03:18.389924] T [access-control.c:246:ac_test_access] access-control: No access allowed In the above it loops through 4 groups which is the amount of group memberships the user has. It correctly finds the owner group. However, it keeps reporting a group id of 0 which doesn't correspond. This is about the extent that I can take it -- I'm not well versed in C enough to try to dig in the stat call. The above is with the latest Git version.
Thanks Evans, that looks like a bug. I'll be looking into it.
Thanks!
Hi Evan I tried to reproduce it and am not running into a permission denied. Here is the sequence of commands I used. Can you please point out whether I am correctly replicating the commands: shehjart@indus:glusterfs$ groups shehjart adm dialout cdrom plugdev lpadmin admin sambashare shehjart@indus:glusterfs$ mkdir /mnt/share shehjart@indus:glusterfs$ ls /mnt -lah total 16K drwxrwxrwx 3 root root 4.0K 2010-11-09 10:53 . drwxr-xr-x 24 root root 4.0K 2010-11-08 14:48 .. drwxr-xr-x 2 shehjart shehjart 4.0K 2010-11-09 10:53 share shehjart@indus:glusterfs$ chgrp admin /mnt/share shehjart@indus:glusterfs$ ls /mnt -lah total 16K drwxrwxrwx 3 root root 4.0K 2010-11-09 10:53 . drwxr-xr-x 24 root root 4.0K 2010-11-08 14:48 .. drwxr-xr-x 2 shehjart admin 4.0K 2010-11-09 10:53 share shehjart@indus:glusterfs$ chmod 2775 /mnt/share shehjart@indus:glusterfs$ ls /mnt -lah total 16K drwxrwxrwx 3 root root 4.0K 2010-11-09 10:53 . drwxr-xr-x 24 root root 4.0K 2010-11-08 14:48 .. drwxrwsr-x 2 shehjart admin 4.0K 2010-11-09 10:53 share shehjart@indus:glusterfs$ touch /mnt/share/testfile shehjart@indus:glusterfs$ ls /mnt -lah total 16K drwxrwxrwx 3 root root 4.0K 2010-11-09 10:53 . drwxr-xr-x 24 root root 4.0K 2010-11-08 14:48 .. drwxrwsr-x 2 shehjart admin 4.0K 2010-11-09 10:54 share
Change the directory (/mnt/share) owner to root (or some other user) and leave the group as "admin". I didn't get any error when my user owned the directory -- only when I tried to use my group membership. (I was also using LDAP but since I assume it's an issue with the gluster access lookup it shouldn't matter).
I'm experiencing the same behavior on my 3.1 install. I have a GlusterFS 3.1 volume being mounted on a client using NFS. From the client I created a directory under the mount point and set the permissions to root:groupa 750 My user account is a member of groupa on the client, yet I am unable to list the contents of the directory: $ ls -l /gfs/dir1 ls: /gfs/dir1/: Permission denied $ ls -ld /gfs/dir1 rwxr-x--- 9 root groupa 73728 Nov 9 09:44 /gfs/dir1/ $ groups myuser groupa I am able to list the directory as the user root. If I change the group ownership to my primary group, myuser, then I can successfully list the contents of the directory. $ sudo chgrp myuser /gfs/dir1 $ ls -ld /gfs/dir1 rwxr-x--- 9 root myuser 73728 Nov 9 09:44 /gfs/dir1/ $ ls -l /gfs/dir1 drwxr-xr-x 5 root root 73728 Mar 26 2010 testdir1 drwxr-x--- 4 root root 73728 Apr 8 2010 testdir2 drwxr-x--- 2 root root 73728 Aug 4 21:23 testdir3 The volume is being exported using the builtin GlusterFS NFS server. The servers and client are all CentOS 5.5 x86_64 boxes.
The bug is in auth-glusterfs.c. The authentication code is not copying the auxiliary group ids that were sent by protocol/client. Patch coming soon.
PATCH: http://patches.gluster.com/patch/5712 in master (rpc-lib: Copy auxiliary GIDs into request from de-XDRed buffer)
PATCH: http://patches.gluster.com/patch/5715 in master (rpc-lib: Fail auth-unix if more than 16 gids found)
PATCH: http://patches.gluster.com/patch/5716 in master (nfsrpc: Fail auth-unix if more than 16 gids found)
(In reply to comment #8) > The bug is in auth-glusterfs.c. The authentication code is not copying the > auxiliary group ids that were sent by protocol/client. Patch coming soon. Thank you very much!
(In reply to comment #9) > PATCH: http://patches.gluster.com/patch/5712 in master (rpc-lib: Copy auxiliary > GIDs into request from de-XDRed buffer) Confirmed on my end. Tested with multiple group ownerships and everything is working as expected. Thanks for the quick fix! Evan
After upgrading my servers to 3.1.1 I'm still experiencing the issue with secondary groups not working with the client using NFS: In this scenario root owns the directory and the admins non primary group (of which my id is a member) is the group: $ groups mikeh admins $ ls -ld /research/project/ drwxrwx--- 10 root admins 73728 Nov 22 12:13 /gluster/project/ $ ls -l /gluster/project/ ls: /gluster/project/: Permission denied $ mount|grep research nas-01.mydom:/research-storage on /gluster type nfs (rw,addr=192.168.1.11) Both Gluster servers and the client are CentOS 5 x86_64 systems. The servers had glusterfs-core / glusterfs-fuse 3.0.1-1 updated to 3.1.1-1 via the RPMs. The storage is a simple distributed file system, no striping or replication. Following the update I rebooted both servers and remounted the NFS file system on the client.
(In reply to comment #14) > After upgrading my servers to 3.1.1 I'm still experiencing the issue with > secondary groups not working with the client using NFS: > > In this scenario root owns the directory and the admins non primary group (of > which my id is a member) is the group: > Thanks. The problem being faced now is slightly different. A patch is on the way.
This is what is going on: nfs3_stat_to_accessbits (buf=0x7fff246b2840, request=31, uid=1000, gid=1000) at nfs3-helpers.c:682 682 uint32_t accresult = 0; (gdb) n 683 ia_prot_t prot = {0, }; (gdb) n 684 ia_type_t type = 0; (gdb) n 686 prot = buf->ia_prot; (gdb) n 687 type = buf->ia_type; (gdb) n 689 if (uid == 0) (gdb) n 691 else if (buf->ia_uid == uid) (gdb) n 693 else if (buf->ia_gid == gid) (gdb) n 696 accresult = nfs3_other_accessbits (prot, type, request); That is, NFS server access callback is not accounting for supplementary group ids to check for non-primary membership.
Created attachment 394
(In reply to comment #15) > (In reply to comment #14) > > After upgrading my servers to 3.1.1 I'm still experiencing the issue with > > secondary groups not working with the client using NFS: > > > > In this scenario root owns the directory and the admins non primary group (of > > which my id is a member) is the group: > > > > Thanks. The problem being faced now is slightly different. A patch is on the > way. Mike, please test with the patch I've attached to this bug. It fixes the problem on my setup. To apply it, you'll have to download the source on the NFS server machine: $ git clone git://git.gluster.com/glusterfs.git ./glusterfs $ cd glusterfs $ git am 0001-nfs3-Access-cbk-must-account-for-auxgids-on-group-ac.patch Then build and install the source. Once thats done, restart Glusterfs volumes and run the same commands again. Thanks
(In reply to comment #18) > (In reply to comment #15) > > (In reply to comment #14) > Mike, please test with the patch I've attached to this bug. It fixes the > problem on my setup. > > To apply it, you'll have to download the source on the NFS server machine: > > $ git clone git://git.gluster.com/glusterfs.git ./glusterfs > $ cd glusterfs > $ git am 0001-nfs3-Access-cbk-must-account-for-auxgids-on-group-ac.patch > > Then build and install the source. Once thats done, restart Glusterfs volumes > and run the same commands again. Thanks I installed my servers via the CentOS rpms, so here's how I tested the above patch: Downloaded the 3.1.1 src.rpm, extracted it, untar'd the source code, applied the patch, tar'd it back up, bumped the release to 3.1.1-2, rpmbuild and rpm -U Following that the simple test listed in my previous test now works via an nfs client. I'll have to do some further testing to ensure that users who are not members of the secondary group aren't allowed access, etc... Since the above method is using patched 3.1.1 source, the Gluster client access is still broken. How do I generate a gluster-3.1.1.tar.gz from the git tree to get all of the latest changes so that I can test both NFS and Gluster clients? Do I simply do the git steps above, rename the directory to gluster-3.1.1 and compress, rpmbuild?
PATCH: http://patches.gluster.com/patch/5916 in master (nfs3: Access cbk must account for auxgids on group access checks)
This seems to have regressed, I'm experiencing the problem with with glusterfs-3.2.6-2.el6.x86_64 (from EPEL). Is it possibly related to https://bugzilla.redhat.com/show_bug.cgi?id=764789 ? The gid handling code was removed from nfs3-helpers.c.