Bug 643528

Summary: Invalid kerberos token error on EMC Celerra when trying 'mount -t nfs4 -o sec=krb5'
Product: Red Hat Enterprise Linux 6 Reporter: John Dickerson <jedicker>
Component: nfs-utilsAssignee: Steve Dickson <steved>
Status: CLOSED DUPLICATE QA Contact: desktop-bugs <desktop-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: bfields, elfed_lewis
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-23 21:01:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
wireshark capture of sec=krb5 nfs4 mount between rhel6 client and Solaris 10 server.
none
wireshark caputure of sec=krb5 nfs4 mount from centos5.4 client and Solaris 10 NFS4 server none

Description John Dickerson 2010-10-15 21:07:10 UTC 
Description of problem:
It appears there is a problem in RHEL6 with the way kerberos tokens are passed to an EMC Celerra running secure NFS.  

In RHEL5.5, I can mount NFS4 shares with sec=krb5 just fine.  With RHEL6 beta2, I get a pemission denied error.  Checking the logs on the EMC Celerra, there is an error:

RPC: 3: RpcGssClient::acquireServerCred: gss_inquire_init_token failed
RPC: 3: GSS-API major error: A token was invalid
RPC: 3: GSS-API minor error: Token header is malformed or corrupt

Otherwise, the kerberos host credentials on the RHEL6 client appear to work fine.  I can use them to do kerberized LDAP requests against our Windows R2008 R2 domain controllers.  And I can use my personal kerberos credentials to mount my home share with CIFS on the Linux client.

Version-Release number of selected component (if applicable):
nfs-utils-1.2.2-4.el6.x86_64
krb5-workstation-1.8.2-2.el6.x86_64

How reproducible:
Initialize kerberos host credentials against the Windows domain.
Try to mount an NFS4 share with the option 'sec=krb5' against a Window R0008 R2 server.
Get permission denied.

Steps to Reproduce:
1.  Store host credentials in /etc/krb5.keytab.
Then do:  kinit -k "nfs/`hostname`@REALM.EDU"
2. mount -t nfs4 -o sec=krb5 file.server:/home/jedicker  /mnt/jedicker
3. Mount fails with "permission denied"
  
Actual results:
# mount --verbose -t nfs4 -o sec=krb5 files.engineering.iastate.edu:/homedirs/jedicker /mnt/files/jedicker
mount.nfs4: timeout set for Fri Oct 15 16:08:11 2010
mount.nfs4: trying text-based options 'sec=krb5,addr=10.10.142.42,clientaddr=129.186.29.131'
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting files.engineering.iastate.edu:/homedirs/jedicker


Expected results:
Mount should succeed.

Additional info:
EMC Celerra DartOS version is 5.6.50-2
Comment 1 John Dickerson 2010-10-15 21:09:11 UTC 
oops.  Accidentally selected gssdp as the component.  Should be nfs-utils.
Comment 3 Bastien Nocera 2010-10-20 12:15:18 UTC 
(In reply to comment #1)
> oops.  Accidentally selected gssdp as the component.  Should be nfs-utils.

You should also reset the owner...
Comment 4 John Dickerson 2010-10-25 22:10:25 UTC 
Created attachment 455635 [details]
wireshark capture of sec=krb5 nfs4 mount between rhel6 client and Solaris 10 server.
Comment 5 John Dickerson 2010-10-25 22:14:09 UTC 
Created attachment 455636 [details]
wireshark caputure of sec=krb5 nfs4 mount from centos5.4 client and Solaris 10 NFS4 server
Comment 6 elf 2010-11-23 19:33:52 UTC 
Not sure if this helps, But I recently had a problem hooking up a rhel6 client to my celerra.  We were using active directory, but it was the kerberos component that was failing.  We had to add the following lines to our krb5.conf file to get it to work:

default_tgs_enctypes = des-cbc-crc des-cbc-md5
default_tkt_enctypes = des-cbc-crc des-cbc-md5

You may want to add other enctypes too, but those worked for us.  Seems older versions of the Celerra dont support strong enctypes.
Comment 7 elf 2010-11-23 20:09:53 UTC 
you may also want to investigate this option:

allow_weak_crypto
Comment 8 J. Bruce Fields 2010-11-23 20:22:14 UTC 
In the bad case, the client sends a truncated init-sec-ctx null packet, the
body of which is truncated immediately after the length field (0x0523 in this
case).

What version of libtirpc is installed, and does it include the patch
referenced in https://bugzilla.redhat.com/show_bug.cgi?id=613682#c20 ?
Comment 9 John Dickerson 2010-11-23 20:39:45 UTC 
It appears that this bug was fixed in RHEL 6.0.  Whatever the cause, it seems to be have been corrected between beta2 and the final release.

# rpm -q libtirpc nfs-utils
libtirpc-0.2.1-1.el6.x86_64
nfs-utils-1.2.2-7.el6.x86_64
 

I have tested this on a cleanly built RHEL 6.0 machine.  I can mount nfs4 with sec=krb5 from our Celerra (Dart OS 5.6.50) as expected.

Also, it appears that the "allow weak crypto = true" setting in /etc/krb5.conf is not needed if ArcFour with HMAC/md5 is used as the default kerberos encryption type.

in /etc/krb5.conf:

[libdefaults]

# allow_weak_crypto = true
  default_tkt_enctypes = rc4-hmac
  default_tgs_enctypes = rc4-hmac
  permitted_enctypes = rc4-hmac

According to an EMC engineer who investigated this problem, ArcFour with HMAC/md5 is the stronest encryption type supported by the Celerra.  Apparently, this is strong enough encryption to obviate the need for the "weak crypto" setting.
Comment 10 J. Bruce Fields 2010-11-23 21:01:42 UTC 

*** This bug has been marked as a duplicate of bug 613682 ***