Bug 503142

Summary: NFSv4+krb5. Group write access denied client-side until NFS-share is remounted.
Product: Red Hat Enterprise Linux 5 Reporter: Espen Braastad <espen>
Component: nfs-utilsAssignee: Steve Dickson <steved>
Status: CLOSED NOTABUG QA Contact: yanfu,wang <yanwang>
Severity: high Docs Contact:
Priority: low    
Version: 5.3CC: bfields
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-01-22 15:10:34 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Espen Braastad 2009-05-29 08:43:52 UTC
Description of problem:
-----------------------
The (simplified) scenario is one server running RHEL5.3, krb5 and NFSv4 and one client. The client is also RHEL5.3. Users log onto the client using SSH (authenticating using kerberos) and access the filesystems shared by the NFSv4 server (exported with krb5, krb5i or krb5p).

The problem is that whenever a new gidnumber is created, existing users are added to this group and a directory (on one of the NFSv4 shares) is chgrp-ed to this gidnumber - the group does not have write access even though the users re-authenticates (by logging out and in or using su - <username>)

The workaround is to umount and mount (mount -o remount does not work) the share on the client.



Version-Release number of selected component (if applicable):
-------------------------------------------------------------
It's a bit uncertain where the problem is located but the server and client are both running on the latest RHEL5.3 x86_64 packages (today is 2009-05-29). The version numbers from some of the central packages in the setup are:

 krb5-workstation-1.6.1-31.el5_3.3
 krb5-server-1.6.1-31.el5_3.3
 krb5-libs-1.6.1-31.el5_3.3
 pam_krb5-2.2.14-10
 pam_krb5-2.2.14-10
 krb5-libs-1.6.1-31.el5_3.3
 nfs-utils-lib-1.0.8-7.2.z2
 nfs-utils-1.0.9-40.el5
 nss_ldap-253-17.el5
 nss_db-2.2-35.3
 nss-3.12.2.0-4.el5
 nss-tools-3.12.2.0-4.el5



How reproducible:
-----------------
 Always



Steps to Reproduce:
-------------------
1. Have a working setup with one server and one client.

The server is running krb5kdc, krb524d, NFS mountd, NFS daemon, NFS services, RPC svcgssd and rpc.idmapd.

Server:
/etc/exports:
 /exports      gss/krb5p(sync,rw,fsid=0)

/etc/sysconfig/nfs:
 SECURE_NFS="yes"
 RPCGSSDARGS="-vvv"
 RPCSVCGSSDARGS="-vvv"

/etc/nsswitch.conf:
 passwd:     files ldap
 shadow:     files ldap
 group:      files ldap
 hosts:      files dns
 [...]

nscd is not running.


The client is:

/etc/fstab:
 [...]
 server:/ /nfs nfs4 rw,sec=krb5p,addr=192.168.0.1 0 0
 [...]

nscd is not running.
 
2. Add a user foo on both the client and the server with the same uidnumber and gidnumber (or add the user in ldap, the end result is the same) and add a principal for the user in the KDC.

3. Verify that the user is able to log onto both the server and the client using SSH.

4. Create a directory on the NFS-share that is owned by the user foo uidnumer and gidnumer. Verify that the user has write and read access.

5. Add a new group bar, and add the user as a member to this group.

6. Create a directory on the NFS-share that is owned by root:bar, and set the mode to 2775. This mode should give the group bar write access.

7. Verify that the user foo is a member of group bar running id -a as foo.

8. See that touching a file as foo in the directory owned by root:bar fails. This is normal behavior until foo re-authenticates by logging out and in, or run su - foo.

9. Log out and in, or su - foo, try to touch the file again. The same permission denied message is shown:

[foo@client directory]$ touch file
touch: cannot touch `file': Permission denied

10. On the NFS-client, umount and mount the /nfs-directory. Try to touch the file again, which will succeed.
  


Actual results:
---------------
The user is unable to write to directories owned by the one of the groups the user is a member in, even though the permissions are set correct (rwx for the group). Only when the NFS-share is umounted and mounted, the user gets write access from the client.



Expected results:
-----------------
The user should have gain access without first having to umount+mount the NFS-share.



Additional info:
----------------
We have done quite a bit of debugging. We have tried to restart and/or lowering the timeouts in rpc.idmapd, it didn't help. We know that setup using NFSv3 (without kerberos) works as expected. We know also that the NFS-client is communicating with the LDAP-server whenever a user is navigating in the directory tree (to map GIDs and UIDs), so the gidmember information is not outdated to the client.

Comment 1 J. Bruce Fields 2010-12-06 23:07:02 UTC
The server looks up the group membership, and passes that information down to the kernel, just once--at the time that the client and server initially negotiate a new rpcsec_gss context for that user.

I suspect that's the cause of the failure to update.  If so,

date +%s >/proc/net/rpc/auth.rpcsec.context/flush

instead of unmounting and remounting on the client, should clear the problem.

Comment 2 Steve Dickson 2011-01-22 14:51:30 UTC
Does 
   date +%s >/proc/net/rpc/auth.rpcsec.context/flush

Clear up the problem?

Comment 3 Espen Braastad 2011-01-22 15:00:08 UTC
Yes, it does! Thanks.