From what little I understand of the cifs mount workflow I believe the issue is with cifs-utils, but I could be wrong. I'm not sure when this changed exactly but previously when automounts were configured for cifs+dfs with krb5 using either autofs or systemd automounts the mount would be executed as the uid of the process and use the credential cache belonging to that uid. This was extremely nice because it allowed us to mount a dfs root that was accessible to all users (and machines) but restrict access to the actual shares based on group membership. Now as of Fedora 39 what seems to happen is the mount is first attempted as uid=0, then if successful attempted as the uid of the triggering process. Working mount (just mounting the dfs root and a folder under it that the machine creds have access to to demonstrate what seems to be happening now): kernel: CIFS: Attempting to mount //<dfs host>/<dfs root> cifs.upcall[7308]: key description: cifs.spnego;0;0;39010000;ver=0x2;host=<dfs host>;ip4=>dfs host>;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x1c89 cifs.upcall[7309]: ver=2 cifs.upcall[7309]: host=<dfs host> cifs.upcall[7309]: ip=<dfs host> cifs.upcall[7309]: sec=1 cifs.upcall[7309]: uid=0 cifs.upcall[7309]: creduid=0 cifs.upcall[7309]: user=root cifs.upcall[7309]: pid=7305 cifs.upcall[7308]: get_cachename_from_process_env: pid == 0 cifs.upcall[7308]: get_existing_cc: default ccache is KCM:0:75577 cifs.upcall[7308]: handle_krb5_mech: getting service ticket for <dfs host> cifs.upcall[7308]: handle_krb5_mech: using native krb5 cifs.upcall[7308]: handle_krb5_mech: obtained service ticket cifs.upcall[7308]: Exit status 0 cifs.upcall[7311]: key description: cifs.spnego;0;0;39010000;ver=0x2;host=<dfs host>;ip4=<dfs host>;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x1c89 cifs.upcall[7312]: ver=2 cifs.upcall[7312]: host=<dfs host> cifs.upcall[7312]: ip=<dfs host> cifs.upcall[7312]: sec=1 cifs.upcall[7312]: uid=0 cifs.upcall[7312]: creduid=0 cifs.upcall[7312]: user=root cifs.upcall[7312]: pid=7305 cifs.upcall[7311]: get_cachename_from_process_env: pid == 0 cifs.upcall[7311]: get_existing_cc: default ccache is KCM:0:75577 cifs.upcall[7311]: handle_krb5_mech: using native krb5 cifs.upcall[7311]: handle_krb5_mech: obtained service ticket cifs.upcall[7311]: Exit status 0 cifs.upcall[7313]: key description: cifs.spnego;0;0;39010000;ver=0x2;host=<dfs host>;ip4=<dfs host>;sec=krb5;uid=<hex user uid>;creduid=<hex user uid>;pid=0x1c86 cifs.upcall[7315]: ver=2 cifs.upcall[7315]: host=<dfs host> cifs.upcall[7315]: ip=<dfs host> cifs.upcall[7315]: sec=1 cifs.upcall[7315]: uid=<user uid> cifs.upcall[7315]: creduid=<user uid> cifs.upcall[7315]: pid=7302 cifs.upcall[7313]: get_cachename_from_process_env: pathname=/proc/7302/environ cifs.upcall[7313]: get_existing_cc: default ccache is KCM:<user uid> cifs.upcall[7313]: handle_krb5_mech: getting service ticket for <dfs host> cifs.upcall[7313]: handle_krb5_mech: using native krb5 cifs.upcall[7313]: handle_krb5_mech: obtained service ticket cifs.upcall[7313]: Exit status 0 failing mount (actual share pointed to in dfs): kernel: CIFS: Attempting to mount //<dfs host>/<dfs root>/<folder in dfs root>/<dfs ptr to actual share> cifs.upcall[7319]: key description: cifs.spnego;0;0;39010000;ver=0x2;host=<actual samba server>;ip4=<actual samba server>;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x1c95 cifs.upcall[7320]: ver=2 cifs.upcall[7320]: host=<actual samba server> cifs.upcall[7320]: ip=<actual samba server> cifs.upcall[7320]: sec=1 cifs.upcall[7320]: uid=0 cifs.upcall[7320]: creduid=0 cifs.upcall[7320]: user=root cifs.upcall[7320]: pid=7317 cifs.upcall[7319]: get_cachename_from_process_env: pid == 0 cifs.upcall[7319]: get_existing_cc: default ccache is KCM:0:75577 cifs.upcall[7319]: handle_krb5_mech: getting service ticket for <actual samba server> cifs.upcall[7319]: handle_krb5_mech: using native krb5 cifs.upcall[7319]: handle_krb5_mech: obtained service ticket cifs.upcall[7319]: Exit status 0 kernel: CIFS: Status code returned 0xc000006d STATUS_LOGON_FAILURE Send error in SessSetup = -13 cifs.upcall[7322]: key description: cifs.spnego;0;0;39010000;ver=0x2;host=<actual samba server>;ip4=<actual samba server>;sec=krb5;uid=0x0;creduid=0x0;user=root;pid=0x1c95 cifs.upcall[7323]: ver=2 cifs.upcall[7323]: host=<actual samba server> cifs.upcall[7323]: ip=<actual samba server> cifs.upcall[7323]: sec=1 cifs.upcall[7323]: uid=0 cifs.upcall[7323]: creduid=0 cifs.upcall[7323]: user=root cifs.upcall[7323]: pid=7317 cifs.upcall[7322]: get_cachename_from_process_env: pid == 0 cifs.upcall[7322]: get_existing_cc: default ccache is KCM:0:75577 cifs.upcall[7322]: handle_krb5_mech: getting service ticket for <actual samba server> cifs.upcall[7322]: handle_krb5_mech: using native krb5 cifs.upcall[7322]: handle_krb5_mech: obtained service ticket cifs.upcall[7322]: Exit status 0 kernel: CIFS: Status code returned 0xc000006d STATUS_LOGON_FAILURE Send error in SessSetup = -13 kernel: CIFS: VFS: cifs_mount failed w/return code = -13 With previous versions it wouldn't try to mount the dfs targets or dfs root as uid=0 at all, but as the uid of the triggering process. Reproducible: Always Steps to Reproduce: 1. Configure and start mount and automount units for dfs root 2. As user with permissions (and a valid krb5 ccache) attempt to trigger the mount by cd-ing to or ls-ing the relevant directory 3. Get permission denied Actual Results: Permissions are denied as the ccache uid=0 has does not have permission to the share. Expected Results: Mount is attempted as the uid of the triggering user causing cifs.upcall to use the uid's ccache and be granted access. I don't test this as often as I should because the server in question doesn't get rebuilt that frequently and tends to stay on the same fedora release for quite some time. So I am not really sure when the behavior changed.
Could you please add details on exact mount options and autofs/systemd configuration? cifs.upcall is called by the kernel's cifs.ko/smb3.ko drivers, so their mount options would influence the calls. It may well be that some daemon tries to access the share to be automounted as root prior to user's access.
Certainly, The systemd .mount contains Type=smb3 Options=_netdev,sec=krb5,multiuser,nounix,noserverino,file_mode=0770,dir_mode=0770,nosfu,vers=3.0 in the mount section the automount unit is configured for the mountpoint the mount unit defines. My test with autofs uses the same mount arguments as the systemd mount unit.
The last time this is verified to work was somewhere around fc34 or fc35. On the off chance it was a kernel change I installed the latest 5.15 kernel from kernel.org. Same behavior. I'll pull the actual source for the latest fc34 kernel just in case it is the result of some backported fix.
Interesting. It is the kernel client. So with 5.15.148 the issue persists, but with 5.13.12-200.fc34.x86_64 it works as we expect. From the logs a notable difference when attempting to access a directory in the dfs structure that refers to a path on an actual host it shows: kernel: CIFS: Attempting to mount \\<dfs host>\<dfs root> instead of the non functional kernel: kernel: CIFS: Attempting to mount //<dfs host>/<dfs root>/<folder in dfs root>/<dfs ptr to actual share> moving forward in time and attempting with 5.15.18-100.fc34.x86_64 it still works as we expect. So whatever broke it was something that was also backported to 5.15.
Works in 6.1.18, broken in the next kernel release (6.2.7) for fc36.
Thank you for this research, Shane! I'll talk to Steve (upstream cifs.ko maintainer) and ask him to look at this issue.
I'm working on bisecting 6.1 right now, hopefully I'll have results sometime today.
Shane, thanks for looking into this. Does it work if you add 'cruid=$USER' to mount options? Besides, can you provide the following from both working and non-working kernels: # umount any existing CIFS shares in the system dmesg --clear modprobe dns_resolver modprobe cifs echo 'module cifs +p' > /sys/kernel/debug/dynamic_debug/control echo 'file fs/cifs/* +p' > /sys/kernel/debug/dynamic_debug/control echo 1 > /proc/fs/cifs/cifsFYI echo 1 > /sys/module/dns_resolver/parameters/debug echo 0 > /proc/sys/kernel/printk_ratelimit echo 0 > /proc/fs/cifs/dfscache tcpdump -s 0 -w trace.pcap port 445 & pid=$! sleep 3 klist > trace.log # mount share sleep 3 kill $pid echo "========" >> trace.log cat /proc/fs/cifs/DebugData >> trace.log echo "========" >> trace.log cat /proc/fs/cifs/dfscache >> trace.log echo "========" >> trace.log dmesg >> trace.log Thanks.
Result of the bisect suggests 9fd29a5bae6e8f94b410374099a6fddb253d2d5f is the culprit.
Thanks for bisecting it. Yeah, that was the commit I had in mind. The problem seems related to the automounted DFS link inheriting *uid and *gid values from the DFS root superblock without checking if they were actually set. That is, if the user didn't specify them, we would need to set their new values for the filesystem context used for automounting the DFS link. Could you please check if below changes fix you issue diff --git a/fs/smb/client/cifs_dfs_ref.c b/fs/smb/client/cifs_dfs_ref.c index 020e71fe1454..1edb7bcc8b81 100644 --- a/fs/smb/client/cifs_dfs_ref.c +++ b/fs/smb/client/cifs_dfs_ref.c @@ -308,6 +308,10 @@ static struct vfsmount *cifs_dfs_do_automount(struct path *path) tmp = *cur_ctx; tmp.source = full_path; tmp.UNC = tmp.prepath = NULL; + if (!tmp.cruid_specified) + tmp.cred_uid = current_fsuid(); + tmp.linux_uid = current_fsuid(); + tmp.linux_gid = current_fsgid(); rc = smb3_fs_context_dup(ctx, &tmp); if (rc) {
I believe I applied it correctly to 6.6.14: diff --git a/fs/smb/client/namespace.c b/fs/smb/client/namespace.c index a6968573b775..5843d8da26e4 100644 --- a/fs/smb/client/namespace.c +++ b/fs/smb/client/namespace.c @@ -206,6 +206,11 @@ static struct vfsmount *cifs_do_automount(struct path *path) tmp.UNC = tmp.prepath = NULL; tmp.dfs_root_ses = NULL; + if (!tmp.cruid_specified) + tmp.cred_uid = current_fsuid(); + tmp.linux_uid = current_fsuid(); + tmp.linux_gid = current_fsgid(); + rc = smb3_fs_context_dup(ctx, &tmp); if (rc) { mnt = ERR_PTR(rc); and it indeed does work as I would like, thank you!
(In reply to Shane Nehring from comment #11) > I believe I applied it correctly to 6.6.14: > > diff --git a/fs/smb/client/namespace.c b/fs/smb/client/namespace.c > index a6968573b775..5843d8da26e4 100644 > --- a/fs/smb/client/namespace.c > +++ b/fs/smb/client/namespace.c > @@ -206,6 +206,11 @@ static struct vfsmount *cifs_do_automount(struct path > *path) > tmp.UNC = tmp.prepath = NULL; > tmp.dfs_root_ses = NULL; > > + if (!tmp.cruid_specified) > + tmp.cred_uid = current_fsuid(); > + tmp.linux_uid = current_fsuid(); > + tmp.linux_gid = current_fsgid(); > + > rc = smb3_fs_context_dup(ctx, &tmp); > if (rc) { > mnt = ERR_PTR(rc); LGTM. > and it indeed does work as I would like, thank you! Thanks for testing it! I'll send it usptream and make sure that it goes through -stable as well.
FYI, submitted fix upstream [1]. Maintainer merged it into his for-next branch as of 4508ec173570 [2] and marked it for -stable. [1] https://lore.kernel.org/all/20240211231931.185193-1-pc@manguebit.com/ [2] https://git.samba.org/?p=sfrench/cifs-2.6.git;a=commit;h=4508ec17357094e2075f334948393ddedbb75157
FEDORA-2024-88847bc77a (kernel-6.7.5-200.fc39) has been submitted as an update to Fedora 39. https://bodhi.fedoraproject.org/updates/FEDORA-2024-88847bc77a
FEDORA-2024-987089eca2 (kernel-6.7.5-100.fc38) has been submitted as an update to Fedora 38. https://bodhi.fedoraproject.org/updates/FEDORA-2024-987089eca2
FEDORA-2024-88847bc77a has been pushed to the Fedora 39 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-88847bc77a` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-88847bc77a See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-987089eca2 has been pushed to the Fedora 38 testing repository. Soon you'll be able to install the update with the following command: `sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2024-987089eca2` You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2024-987089eca2 See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
FEDORA-2024-88847bc77a (kernel-6.7.5-200.fc39) has been pushed to the Fedora 39 stable repository. If problem still persists, please make note of it in this bug report.
FEDORA-2024-987089eca2 (kernel-6.7.5-100.fc38) has been pushed to the Fedora 38 stable repository. If problem still persists, please make note of it in this bug report.