NFS mounts on of my computers with kerberos set up have stopped working after a kernel update. It can mount filesystems successfully when running 4.15.3-300.fc27.x86_64 but stops working with 4.15.4-300.fc27.x86_64 and later. It does work if I rename /etc/krb5.keytab to disable kerberos. This is true even when mounting with sec=sys where it shouldn't be trying to use kerberos at all (though my testing suggests it is trying to mount with kerberos even when sec=sys is requested). The error I get when trying a mount command is mount.nfs: mount system call failed
Hi, There are a couple NFS-related changes in 4.15.4 - I've reverted the one that looks the most likely to me. Can you try the kernel from https://koji.fedoraproject.org/koji/taskinfo?taskID=25858412 when it is finished building? Is there anything in the logs in either the client or the server when you try to mount the filesystem? I assume the client is Fedora, what's the server? Finally, have you tested the latest 4.15 kernel from updates-testing? It would also be helpful to test the latest 4.16 kernel (https://bodhi.fedoraproject.org/updates/FEDORA-2018-5ffc6f479f) to see if it's already been fixed. Thanks!
There are errors in the client such as the following in dmesg [ 1232.522816] gss_marshal: gss_get_mic FAILED (851968) [ 1232.522819] RPC: couldn't encode RPC header, exit EIO [ 1232.522856] gss_marshal: gss_get_mic FAILED (851968) [ 1232.522857] RPC: couldn't encode RPC header, exit EIO [ 1232.522863] NFS: nfs4_discover_server_trunking unhandled error -5. Exiting with error EIO [ 1232.525039] gss_marshal: gss_get_mic FAILED (851968) [ 1232.525042] RPC: couldn't encode RPC header, exit EIO There aren't any errors on the server (but it doesn't log much as it is a busy server). The issue is also present in 4.16 - I have tested kernel-4.16.0-0.rc1.git0.1.fc28.x86_64 and kernel-4.16.0-0.rc6.git0.2.fc28.x86_64 while investigating if it was fixed in Fedora 28.
4.15.4-301.fc27.x86_64 still shows the problem.
Thanks for testing that. Nothing else looks very suspicious to me. Since you have a straight-forward reproducer and a known good version, I recommend bisecting the upstream kernel to find the commit that introduced the problem.
I traced the problem patch which is [46e8d06e423c4f35eac7a8b677b713b3ec9b0684] crypto: hash - prevent using keyed hashes without setting key NFS works for me with kernel-4.15.12-301.fc27.x86_64 built with this patch reverted.
Great! The next thing to do is report this to upstream. I'm happy to do this if you don't want to, but it's simpler if you do it since I don't have a system to reproduce this on at the moment. I recommend sending an email to the original committer and reviewer to the patch that introduced the problem and CC the maintainers of NFS and the relevant lists: Author and reviewer: * Eric Biggers <ebiggers> * Herbert Xu <herbert.org.au> NFS maintainers: * "J. Bruce Fields" <bfields> * Jeff Layton <jlayton> * Trond Myklebust <trond.myklebust> * Anna Schumaker <anna.schumaker> Lists: * linux-nfs.org * netdev.org * linux-kernel.org
The patch posted upstream for testing in https://marc.info/?l=linux-kernel&m=152218980114708&w=2 fixes the problem.
Great! When it's accepted upstream I'll add it to the stable builds. The updates system will post on this bug when it's in the updates-testing repository.
kernel-4.15.14-200.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2018-18754260e4
kernel-4.15.14-300.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-7802740586
kernel-4.15.14-200.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-18754260e4
kernel-4.15.14-300.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-7802740586
kernel-4.16.0-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-cf40575a4e
kernel-4.15.14-200.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.
kernel-4.15.14-300.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.
kernel-4.16.0-300.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-cf40575a4e
kernel-4.16.0-300.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.