Description of problem: setting up krb5-NFS - gssproxy refuses mount request. it permanently reports: Feb 21 11:08:51 flash gssproxy[723]: GSSX_RES_ACCEPT_SEC_CONTEXT( status: { 851968 <None> 2529638956 "Unspecified GSS failure. Minor code may provide more information" "Request ticket server nfs/flash.chaosnet.lan kvno 1 not found in keytab; ticket is likely out of date" [ ] } context_handle: <Null> output_token: <Null> delegated_cred_handle: <Null> ) interesting enough regardless what kvno the keytab contains - gssproxy always flags 'kvno 1' as the issue. Version-Release number of selected component (if applicable): fedora 33 server 5.10.16-200.fc33.x86_64 gssproxy-0.8.3-3.fc33.x86_64 nfs-utils-2.5.2-1.rc4.fc33.x86_64 How reproducible: setup a krb5 nfs4-export - and try to mount it from another machine. Steps to Reproduce: 1. setup a correct krb5-membership (like IPA-Client!) 2. get the nfs/hostname principal into the server's keytab via 'ipa-getkeytab -p nfs/hostname -k /etc/krb5.keytab' 3. export a tree via /etc/exports like: /dir gss/krb5(fsid=0,rw,no_root_squash,no_subtree_check,async,nohide,insecure) Actual results: A valid mount-attempt from any client fails. what is a 'valid' mount-attempt ? - well - the same mount-cmd to a different server - works! works on centos-8.3 to centos 7.9 server or vice versa (centos-7.9 server, centos 8.3 client) EXPORTING FAILS on Fedora_33. -- Mounting another krb5-nfs4 server WORKS! Expected results: Exporting (and mounting it from a client) an nfs4-krb5-FS should work on fedora 33 too. Additional info: (SERVER-SIDE - FEDORA-33) [root@flash log]# klist -k Keytab name: FILE:/etc/krb5.keytab KVNO Principal ---- -------------------------------------------------------------------------- 5 host/flash.chaosnet.lan 5 host/flash.chaosnet.lan 4 nfs/flash.chaosnet.lan 4 nfs/flash.chaosnet.lan [root@flash log]# kvno nfs/flash.chaosnet.lan nfs/flash.chaosnet.lan: kvno = 4 [root@flash log]# if you change gssproxy to run with '--debug-level=2' - each mount attempt from a client leaves this in /var/log/messages: Feb 21 11:08:51 flash gssproxy[723]: GSSX_RES_ACCEPT_SEC_CONTEXT( status: { 851968 <None> 2529638956 "Unspecified GSS failure. Minor code may provide more information" "Request ticket server nfs/flash.chaosnet.lan kvno 1 not found in keytab; ticket is likely out of date" [ ] } context_handle: <Null> output_token: <Null> delegated_cred_handle: <Null> ) I just don't have an idea, where this idea of 'kvno 1' comes from ... it's a fresh install. ... rebooting the server doesn't change anything on this behavior. A client mount-attempt (centos-8.3, but also centos-7.9) always just fails: [root@hpe-lt ~]# /usr/bin/mount -v -v -v -t nfs4 flash:/ -o vers=4,sec=krb5 /mnt/flash mount.nfs4: timeout set for Sun Feb 21 11:10:50 2021 mount.nfs4: trying text-based options 'sec=krb5,vers=4.2,addr=10.20.30.50,clientaddr=10.20.30.202' mount.nfs4: mount(2): Permission denied mount.nfs4: trying text-based options 'sec=krb5,vers=4,minorversion=1,addr=10.20.30.50,clientaddr=10.20.30.202' mount.nfs4: mount(2): Permission denied mount.nfs4: trying text-based options 'sec=krb5,vers=4,addr=10.20.30.50,clientaddr=10.20.30.202' mount.nfs4: mount(2): Permission denied mount.nfs4: access denied by server while mounting flash:/ INTERESTING ENOUGH - this 'server' - has no problems to mount as a client - a krb5-NFS4-mount !!! THAT WORKS. like this: (hpe-lt is a centos-8.3 machine) [root@flash ~]# mkdir /mnt/hpe-lt [root@flash ~]# mount -v -v -t nfs4 -o sec=krb5 hpe-lt:/ /mnt/hpe-lt mount.nfs4: timeout set for Sun Feb 21 17:48:29 2021 mount.nfs4: trying text-based options 'sec=krb5,vers=4.2,addr=10.20.30.202,clientaddr=10.20.30.50' [root@flash ~]# df -h /mnt/hpe-lt Filesystem Size Used Avail Use% Mounted on hpe-lt:/ 72G 43G 30G 60% /mnt/hpe-lt CURRENT suspicion is, that 'gssproxy' on fedora-33 is broken.
This sound like the client has cached a ticket for key #1, if that key was removed from the server side (where gssproxy is), then no amount of changing keys will help. The solution is to clear the clients cache, and set up policies to *not* replace older keys once a new key is issued. If the server machine was reinstalled then your only options are either to reduce the lifetime of client tickets to reduce the timeframe of this occurrences, or to save and restore keys. Closing for now, reopen if you ahve evidence that this is not the explanation.
YES - you were right :-) After your comment - I started to focus on the questionable clients! I've first reduced the ticket-lifetimes on the ipa-server to a low time. (via: 'ipa krbtpolicy-mod --maxlife=3600 --maxrenew=18000') then cleaned out again (did this before too, but no luck) the client's krb-stuff (kdestroy -A) restarted gssproxy and rpc.gssd - and now it works ! - I can indeed now mount the fedora server from clients.