Description of problem:
If a RHEL 5.5 server is set up as a kerberized NFS (sec=krb5) server in conjunction with Active Directory, users with large GSSAPI tokens cannot access the file shares, and whenever they attempt to, they hang the NFS daemons for 30 seconds.

Note that there are two problems with this:
- Users in lots of security groups cannot access the file server;
- A malicious user can set up their user ID to satisfy the conditions, and launch a denial of service attack on the file server.

When a client accesses an NFS share that uses krb5 for the first time, an NFS NULL call is made to the server to establish a GSSAPI security context.  If this NULL call has a GSSAPI token (past the 64-byte RPC header portion) of 2052 or larger bytes, the NFS daemons on the server will go into a busy-wait loop until they time out (I observed 3 daemons consuming CPU).  On my test machine, this consumes 50% CPU for 30 seconds for a single call (on a dual-cpu system - the NFS processes were using 100% of one of the CPUs).  With a GSSAPI token of 2044 bytes (or less), the NFS NULL call is successful, establishing a valid GSSAPI security context (I did not set up a test for 2048 bytes).  This happens in both NFSv3 and v4 (the issue appears to be in net/sunrpc in the kernel tree).

Using MIT or Heimdal servers, you won't get these large GSSAPI tokens in the initial NFS NULL RPC call.  But with Active Directory, the ticket contains an AuthorizationData field with additional information.  One of the things in this field is an array of the groups the user is in.  On my test user, it took 113 group memberships to push the GSSAPI token to 2052 bytes.

It is not infrequent in large organizations for users to be in enough groups to trip this condition (the number includes all nested groups, not just top-level groups).


Version-Release number of selected component (if applicable):

2.6.18-194.8.1.el5 #1 SMP Wed Jun 23 10:58:38 EDT 2010 i686 i686 i386 GNU/Linux

Did a yum update to update all components just before submitting bug.

How reproducible:

Using a user in lots of Active Directory groups, attempt to access a previously-mounted NFS share that uses krb5 for security (either v3 or v4).


Steps to Reproduce:
1.  In Active Directory, configure a user with ~ 115 group memberships.  The key is to set it up so that the GSSAPI token is 2052 or larger (unknown if this will also happen at 2048).

2.  Set up a RHEL5.5 system that shares NFS with krb5 security, using an Active Directory master Kerberos server.  

3.  Grant access to the file share to the user created in step 1

4.  Export the filesystems (note that I tried a variety of combinations of secure/root_squash, and they didn't make a difference for this bug)

contents of /etc/exports:
/export/NFS4 gss/krb5(insecure,rw,fsid=0,no_root_squash)

/home gss/krb5(rw,fsid=0)
/home client.uiuc.edu(rw,root_squash)

(/home is bind mounted to /export/NFS4)

5.  On a client machine, mount the NFS share with sec=krb5
NFS3:
mount -t nfs -o sec=krb5 server.uiuc.edu:/home /mnt

NFS4:
mount -t nfs4 -o sec=krb5 server.uiuc.edu:/ /mnt

6.  kinit to that user on a client machine (I used a Fedora 11 client).

7.  Do a list of the shared directory (e.g., "ls /mnt")
  
Actual results:

CPU will spike to ~50% on the server for 30 seconds, and the client will appear to hang.  After 30 seconds, the client will return with "permission denied", and CPU will return to normal on the server


Expected results:

List of the directory /mnt.  This is what happens if the GSS Token is 2044 bytes or lower (unknown what the behavior is between 2045 and 2051 bytes).


Additional info:

Please contact me for more information.  I have done a lot of debugging and tracing on this, since it is a blocking issue for implementing a new fileserver in our environment.  

I have traced it to the server handling of the call, rather than the formation of the call on the client side.

I don't know the *exact* location of the problem, but I've narrowed it down to a couple of calls in net/sunrpc and net/sunrpc/auth_gss (using rpcdebug, tcpdump/wireshark, a debug version of librpcsecgss on the client, and a lot of code inspection).  I just don't have the expertise to trace the RPC calls through the kernel any deeper.