Bug 621238 - Cannot mount nfsv4 krb5.
Cannot mount nfsv4 krb5.
Status: CLOSED DUPLICATE of bug 613682
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: krb5 (Show other bugs)
6.0
All Linux
low Severity medium
: rc
: ---
Assigned To: Nalin Dahyabhai
BaseOS QE Security Team
: RHELNAK
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-08-04 11:07 EDT by Patrik Martinsson
Modified: 2010-08-04 13:02 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-08-04 13:02:56 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Patrik Martinsson 2010-08-04 11:07:05 EDT
Description of problem:
Not really sure this is a bug of krb5, but i suspect so. I cant mount nfsv4 with krb5, without the sec=krb5 option it works like a charm. 

Version-Release number of selected component (if applicable):

krb5-libs-1.8.2-2.el6.x86_64 
krb5-workstation-1.8.2-2.el6.x86_64
gssdp-0.7.1-1.el6.x86_64

How reproducible:
Always. 

Steps to Reproduce:

# /etc/krb5.conf 
[libdefaults]
  default_realm = XX.XXXX.XX
  clockskew = 300
  dns_lookup_realm = true
  dns_lookup_kdc = true
  default_tkt_enctypes =  des-cbc-md5
  default_tgs_enctypes =  des-cbc-md5
  forwardable = true
  allow_weak_crypto = true

[realms]
  XX.XXXX.XX = {
    default_domain = XX.XXXX.XX
  }

[logging]
  kdc = FILE:/var/log/krb5/krb5kdc.log
  admin_server = FILE:/var/log/krb5/kadmind.log
  default = SYSLOG:NOTICE:DAEMON

[domain_realm]
  .XX.XXXX.XX = XX.XXXX.XX
  .xxxx.xx = XX.XXXX.XX
 
 # /etc/idmapd.conf 
 Domain = xxxx.xx
 
 # /etc/sysconfig/nfs 
 RPCIDMAPDARGS="-vvvv"
 SECURE_NFS="yes"
 RPCGSSDARGS="-vvvv"
 RPCSVCGSSDARGS="-vvvv"
 
 # Stop all relevant services  
 /etc/init.d/rpcbind stop; /etc/init.d/rpcgssd stop; /etc/init.d/rpcidmapd stop; /etc/init.d/nfs stop
 
 # Make sure we are not in domain, leave and flush keytab. 
 net ads leave -U xx%xx
 net ads keytab flush -U xx%xx
 
 # Make sure we have no tickets/keytabs or whatsoever.
 kdestroy
 rm -rf /etc/krb5.keytab
 rm -rf /tmp/kr* 
 
 # Join machine to AD and create computer account with both service principle and user principle. 
 net ads join createupn="nfs/$HOSTNAME@XX.XXXX.XX" createcomputer="/FOO/BAR" osName="Linux Red Hat Workstation" osVer="6" -U xx%xx
 
 # Add nfs principle to keytab, there is already one entry (with diff encryptions) so i'm not totally sure if this is needed. 
 net ads keytab add nfs -U xx%xx
 This is nfs section before i issue the following command, 
 2 08/04/10 16:32:03 nfs/client.xxxx.xx@XX.XXXX.XX (DES cbc mode with CRC-32) 
 2 08/04/10 16:32:03 nfs/client.xxxx.xx@XX.XXXX.XX (DES cbc mode with RSA-MD5) 
 2 08/04/10 16:32:03 nfs/client.xxxx.xx@XX.XXXX.XX (ArcFour with HMAC/md5) 
 -- After -- 
 2 08/04/10 16:33:18 nfs/client.xxxx.xx@XX.XXXX.XX (DES cbc mode with CRC-32) 
 2 08/04/10 16:33:18 nfs/client.xxxx.xx@XX.XXXX.XX (DES cbc mode with RSA-MD5) 
 2 08/04/10 16:33:18 nfs/client.xxxx.xx@XX.XXXX.XX (ArcFour with HMAC/md5) 
 2 08/04/10 16:33:19 nfs/CLIENT@XX.XXXX.XX (DES cbc mode with CRC-32) 
 2 08/04/10 16:33:19 nfs/CLIENT@XX.XXXX.XX (DES cbc mode with RSA-MD5) 
 2 08/04/10 16:33:19 nfs/CLIENT@XX.XXXX.XX (ArcFour with HMAC/md5) 
 
 # Now we have joined the machine (created a machineaccount), we got a keytab that looks ok. Start the relevant services. 
 /etc/init.d/rpcbind start; /etc/init.d/rpcgssd start; /etc/init.d/rpcidmapd start; /etc/init.d/nfs start
 
 # Issue mount command, 
 mount -t nfs4 -o sec=krb5 xx:/xx/xx/xx/ /nfstest/ -vvvvv 
 
 # Output, 
 -- mount -- 
 mount: fstab path: "/etc/fstab" 
 mount: mtab path:  "/etc/mtab"
 mount: lock path:  "/etc/mtab~"
 mount: temp path:  "/etc/mtab.tmp"
 mount: spec:  "xx:/xx/xx/xx/"
 mount: node:  "/nfstest/"
 mount: types: "nfs4"
 mount: opts:  "sec=krb5"
 mount: external mount: argv[0] = "/sbin/mount.nfs4"
 mount: external mount: argv[1] = "xx:/xx/xx/xx/"
 mount: external mount: argv[2] = "/nfstest/"
 mount: external mount: argv[3] = "-v"
 mount: external mount: argv[4] = "-o"
 mount: external mount: argv[5] = "rw,sec=krb5"
 mount.nfs4: timeout set for Wed Aug  4 16:24:41 2010
 mount.nfs4: text-based options: 'sec=krb5,clientaddr=xx.xx.x.xx,addr=xxx.xx.xx.xx'
 mount.nfs4: mount(2): Permission denied
 mount.nfs4: access denied by server while mounting xx:/xx/xx/xx/
 
 -- /var/log/messages -- 
 Aug  4 14:45:09 client rpc.idmapd[4656]: New client: 13
 Aug  4 14:45:09 client rpc.idmapd[4656]: Opened /var/lib/nfs/rpc_pipefs//nfs/clnt13/idmap
 Aug  4 14:45:09 client rpc.idmapd[4656]: New client: 14
 Aug  4 14:45:09 client rpc.gssd[4635]: handling gssd upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt13)
 Aug  4 14:45:09 client rpc.gssd[4635]: handle_gssd_upcall: 'mech=krb5 uid=0 enctypes=18,17,16,23,3,1,2 '
 Aug  4 14:45:09 client rpc.gssd[4635]: handling krb5 upcall (/var/lib/nfs/rpc_pipefs/nfs/clnt13)
 Aug  4 14:45:09 client rpc.gssd[4635]: process_krb5_upcall: service is '<null>'
 Aug  4 14:45:09 client rpc.gssd[4635]: Full hostname for 'xx.xxxx.xx' is 'xx.xx.xx'
 Aug  4 14:45:09 client rpc.gssd[4635]: Full hostname for 'client.xxxx.xx' is 'client.xxxx.xx'
 Aug  4 14:45:09 client rpc.gssd[4635]: Key table entry not found while getting keytab entry for ' root/client.xxxx.xx@XX.XXXX.XX' 
 Aug  4 14:45:09 client rpc.gssd[4635]: Success getting keytab entry for 'nfs/client.xxxx.xx@XX.XXXX.XX'
 Aug  4 14:45:09 client rpc.gssd[4635]: Successfully obtained machine credentials for principal ' nfs/client.xxxx.xx@XX.XXXX.XX' stored in ccache 'FILE:/tmp/krb5cc_machine_XX.XXXX.XX' 
 Aug  4 14:45:09 client rpc.gssd[4635]: INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_XX.XXXX.XX' are good until 1280961909
 Aug  4 14:45:09 client rpc.gssd[4635]: using FILE:/tmp/krb5cc_machine_XX.XXXX.XX as credentials cache for machine creds
 Aug  4 14:45:09 client rpc.gssd[4635]: using environment variable to select krb5 ccache FILE:/tmp/krb5cc_machine_XX.XXXX.XX
 Aug  4 14:45:09 client rpc.gssd[4635]: creating context using fsuid 0 (save_uid 0)
 Aug  4 14:45:09 client rpc.gssd[4635]: creating tcp client for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: DEBUG: port already set to 2049
 Aug  4 14:45:09 client rpc.gssd[4635]: creating context with server nfs@xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Failed to create krb5 context for user with uid 0 for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Failed to create machine krb5 context with credentials cache FILE:/tmp/krb5cc_machine_XX.XXXX.XX for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Machine cache is prematurely expired or corrupted trying to recreate cache for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: Full hostname for 'xx.xxxx.xx' is 'xx.xxxx.xx'
 Aug  4 14:45:09 client rpc.gssd[4635]: Full hostname for 'client.xx.xxxx.xx' is 'client.xx.xxxx.xx'
 Aug  4 14:45:09 client rpc.gssd[4635]: Key table entry not found while getting keytab entry for 'root/client.xxxx.xx@XX.XXXX.XX'
 Aug  4 14:45:09 client rpc.gssd[4635]: Success getting keytab entry for 'nfs/client.xxxx.xx@XX.XXXX.XX'
 Aug  4 14:45:09 client rpc.gssd[4635]: INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_XX.XXXX.XX' are good until 1280961909
 Aug  4 14:45:09 client rpc.gssd[4635]: INFO: Credentials in CC 'FILE:/tmp/krb5cc_machine_XX.XXXX.XX' are good until 1280961909
 Aug  4 14:45:09 client rpc.gssd[4635]: using FILE:/tmp/krb5cc_machine_XX.XXXX.XX as credentials cache for machine creds
 Aug  4 14:45:09 client rpc.gssd[4635]: using environment variable to select krb5 ccache FILE:/tmp/krb5cc_machine_XX.XXXX.XX
 Aug  4 14:45:09 client rpc.gssd[4635]: creating context using fsuid 0 (save_uid 0)
 Aug  4 14:45:09 client rpc.gssd[4635]: creating tcp client for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: DEBUG: port already set to 2049
 Aug  4 14:45:09 client rpc.gssd[4635]: creating context with server nfs@xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Failed to create krb5 context for user with uid 0 for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Failed to create machine krb5 context with credentials cache FILE:/tmp/krb5cc_machine_XX.XXXX.XX for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Failed to create machine krb5 context with any credentials cache for server xx.xxxx.xx
 Aug  4 14:45:09 client rpc.gssd[4635]: doing error downcall
 Aug  4 14:45:09 client rpc.idmapd[4656]: Stale client: 13
 Aug  4 14:45:09 client rpc.idmapd[4656]: #011-> closed /var/lib/nfs/rpc_pipefs//nfs/clnt13/idmap
 Aug  4 14:45:09 client rpc.idmapd[4656]: Stale client: 14
 Aug  4 14:45:09 client rpc.idmapd[4656]: #011-> closed /var/lib/nfs/rpc_pipefs//nfs/clnt14/idmap 
 Aug  4 14:45:09 client rpc.gssd[4635]: destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt14
 Aug  4 14:45:09 client rpc.gssd[4635]: destroying client /var/lib/nfs/rpc_pipefs/nfs/clnt13

Actual results:
mount.nfs4: access denied by server while mounting xx:/xx/xx/xx/

Expected results:
A successful mount. 

Additional info:
If i do the exact same steps on a Fedora 10 client this method works. 
Is there something im missing ? 

Best Regards, 
Patrik Martinsson, Sweden.
Comment 2 Patrik Martinsson 2010-08-04 11:22:06 EDT
This thread has kinda the same issue, it seems to work if he downgrades his
krb. 
I tried that though, but without luck. 
I downgraded my krb-libs/krb5-workstation and gssdbp, but without success, but
maybe there is some more packages involved..?

http://www.spinics.net/lists/linux-nfs/msg12307.html
Comment 3 RHEL Product and Program Management 2010-08-04 11:27:38 EDT
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **
Comment 4 Nalin Dahyabhai 2010-08-04 11:41:00 EDT
We're changing a few things between F10 and RHEL6 -- namely, the limitation of only being able to use DES should be gone, which is good because unless configured otherwise, the krb5 libraries won't advertise support for DES (though you already got that).

(In reply to comment #0)
> Description of problem:
> Not really sure this is a bug of krb5, but i suspect so. I cant mount nfsv4
> with krb5, without the sec=krb5 option it works like a charm. 
> 
> Version-Release number of selected component (if applicable):
> 
> krb5-libs-1.8.2-2.el6.x86_64 
> krb5-workstation-1.8.2-2.el6.x86_64
> gssdp-0.7.1-1.el6.x86_64

The gssdp package isn't really involved here; the other key components are nfs-utils, nfs-utils-lib, libtirpc and the kernel.

> How reproducible:
> Always. 
> 
> Steps to Reproduce:
> 
> # /etc/krb5.conf 
> [libdefaults]
>   default_realm = XX.XXXX.XX
>   clockskew = 300
>   dns_lookup_realm = true
>   dns_lookup_kdc = true
>   default_tkt_enctypes =  des-cbc-md5
>   default_tgs_enctypes =  des-cbc-md5
>   forwardable = true
>   allow_weak_crypto = true

Okay, setting allow_weak_crypto when configuring ticket types to just DES is necessary, though at this point setting default_tgs_enctypes and default_tkt_enctypes shouldn't be needed any more.

>  # Add nfs principle to keytab, there is already one entry (with diff
> encryptions) so i'm not totally sure if this is needed. 
>  net ads keytab add nfs -U xx%xx
>  This is nfs section before i issue the following command, 
>  2 08/04/10 16:32:03 nfs/client.xxxx.xx@XX.XXXX.XX (DES cbc mode with CRC-32) 
>  2 08/04/10 16:32:03 nfs/client.xxxx.xx@XX.XXXX.XX (DES cbc mode with RSA-MD5)

Based on your krb5.conf file, which is configured to only use des-cbc-md5, this second one's the only one that should really be necessary.

>  # Now we have joined the machine (created a machineaccount), we got a keytab
> that looks ok. Start the relevant services. 
>  /etc/init.d/rpcbind start; /etc/init.d/rpcgssd start; /etc/init.d/rpcidmapd
> start; /etc/init.d/nfs start
> 
>  # Issue mount command, 
>  mount -t nfs4 -o sec=krb5 xx:/xx/xx/xx/ /nfstest/ -vvvvv 
> 
>  # Output, 
>  -- mount -- 
>  mount: fstab path: "/etc/fstab" 
>  mount: mtab path:  "/etc/mtab"
>  mount: lock path:  "/etc/mtab~"
>  mount: temp path:  "/etc/mtab.tmp"
>  mount: spec:  "xx:/xx/xx/xx/"
>  mount: node:  "/nfstest/"
>  mount: types: "nfs4"
>  mount: opts:  "sec=krb5"
>  mount: external mount: argv[0] = "/sbin/mount.nfs4"
>  mount: external mount: argv[1] = "xx:/xx/xx/xx/"
>  mount: external mount: argv[2] = "/nfstest/"
>  mount: external mount: argv[3] = "-v"
>  mount: external mount: argv[4] = "-o"
>  mount: external mount: argv[5] = "rw,sec=krb5"
>  mount.nfs4: timeout set for Wed Aug  4 16:24:41 2010
>  mount.nfs4: text-based options:
> 'sec=krb5,clientaddr=xx.xx.x.xx,addr=xxx.xx.xx.xx'
>  mount.nfs4: mount(2): Permission denied
>  mount.nfs4: access denied by server while mounting xx:/xx/xx/xx/
> 
>  -- /var/log/messages -- 
>  Aug  4 14:45:09 client rpc.idmapd[4656]: New client: 13
>  Aug  4 14:45:09 client rpc.idmapd[4656]: Opened
> /var/lib/nfs/rpc_pipefs//nfs/clnt13/idmap
>  Aug  4 14:45:09 client rpc.idmapd[4656]: New client: 14
>  Aug  4 14:45:09 client rpc.gssd[4635]: handling gssd upcall
> (/var/lib/nfs/rpc_pipefs/nfs/clnt13)
>  Aug  4 14:45:09 client rpc.gssd[4635]: handle_gssd_upcall: 'mech=krb5 uid=0
> enctypes=18,17,16,23,3,1,2 '
>  Aug  4 14:45:09 client rpc.gssd[4635]: handling krb5 upcall
> (/var/lib/nfs/rpc_pipefs/nfs/clnt13)
>  Aug  4 14:45:09 client rpc.gssd[4635]: process_krb5_upcall: service is
> '<null>'
>  Aug  4 14:45:09 client rpc.gssd[4635]: Full hostname for 'xx.xxxx.xx' is
> 'xx.xx.xx'
>  Aug  4 14:45:09 client rpc.gssd[4635]: Full hostname for 'client.xxxx.xx' is
> 'client.xxxx.xx'
>  Aug  4 14:45:09 client rpc.gssd[4635]: Key table entry not found while getting
> keytab entry for ' root/client.xxxx.xx@XX.XXXX.XX' 
>  Aug  4 14:45:09 client rpc.gssd[4635]: Success getting keytab entry for
> 'nfs/client.xxxx.xx@XX.XXXX.XX'
>  Aug  4 14:45:09 client rpc.gssd[4635]: Successfully obtained machine
> credentials for principal ' nfs/client.xxxx.xx@XX.XXXX.XX' stored in ccache
> 'FILE:/tmp/krb5cc_machine_XX.XXXX.XX' 
>  Aug  4 14:45:09 client rpc.gssd[4635]: INFO: Credentials in CC
> 'FILE:/tmp/krb5cc_machine_XX.XXXX.XX' are good until 1280961909
>  Aug  4 14:45:09 client rpc.gssd[4635]: using
> FILE:/tmp/krb5cc_machine_XX.XXXX.XX as credentials cache for machine creds
>  Aug  4 14:45:09 client rpc.gssd[4635]: using environment variable to select
> krb5 ccache FILE:/tmp/krb5cc_machine_XX.XXXX.XX
>  Aug  4 14:45:09 client rpc.gssd[4635]: creating context using fsuid 0
> (save_uid 0)
>  Aug  4 14:45:09 client rpc.gssd[4635]: creating tcp client for server
> xx.xxxx.xx
>  Aug  4 14:45:09 client rpc.gssd[4635]: DEBUG: port already set to 2049
>  Aug  4 14:45:09 client rpc.gssd[4635]: creating context with server
> nfs@xx.xxxx.xx

So far, so good.

>  Aug  4 14:45:09 client rpc.gssd[4635]: WARNING: Failed to create krb5 context
> for user with uid 0 for server xx.xxxx.xx

This sounds exactly like bug #613682, which looks to be fixed with the testing update from http://people.redhat.com/steved/.tmp/libtirpc-0.2.1-1.bz613682.el6.x86_64.rpm.  Can you check if this has any effect in your case?

> Additional info:
> If i do the exact same steps on a Fedora 10 client this method works. 
> Is there something im missing ?

That's actually a bit surprising -- I wouldn't tend to expect DES-only setups to work with one release but not with the other.  First let's find out if it's the libtirpc problem.
Comment 5 Patrik Martinsson 2010-08-04 11:50:16 EDT
Updating that rpm solved the issue, mount works as a charm now... 
I've been at this for almost 2 days now, weird that i missed that bugreport. 

Thanks for the quick answer and a great job !

Best regards, 
Patrik Martinsson, Sweden.
Comment 6 Nalin Dahyabhai 2010-08-04 13:02:56 EDT
Glad it worked.  I'll mark this as a duplicate, then.  Thanks!

*** This bug has been marked as a duplicate of bug 613682 ***

Note You need to log in before you can comment on or make changes to this bug.