Bug 2158496 - kernel-6.0.16 kernel panic cifs_get_inode_info
Summary: kernel-6.0.16 kernel panic cifs_get_inode_info
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 37
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-01-05 15:22 UTC by Jan Cipa
Modified: 2023-01-12 17:13 UTC (History)
28 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-01-11 01:23:52 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
vmcore-dmesg (90.39 KB, text/plain)
2023-01-05 15:22 UTC, Jan Cipa
no flags Details
Dmesg with crash (81.34 KB, text/plain)
2023-01-09 14:44 UTC, Slava Bacherikov
no flags Details

Description Jan Cipa 2023-01-05 15:22:39 UTC
Created attachment 1935957 [details]
vmcore-dmesg

1. Please describe the problem:
after upgrade to kernel-6.0.16-300.fc37.x86_64 kernel panic when trying to mount samba/cifs share

2. What is the Version-Release number of the kernel:
kernel-6.0.16-300.fc37.x86_64

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
yes, 6.0.15 works

4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
mount cifs volume and try to list any non empty directory 

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
n/a

6. Are you running any modules that not shipped with directly Fedora's kernel?:
n/a

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

attaching crashkernel vmcore

Comment 1 Jan Cipa 2023-01-05 15:30:21 UTC
generated vmcore can be downloaded from https://czipis.eu/vmcore.gz

Comment 2 Jan Cipa 2023-01-05 15:35:35 UTC
# cat /proc/mounts  | grep cifs 
//IP.ADDR.REDAC.TED/sata3 /mnt/cifs cifs rw,nosuid,nodev,noexec,relatime,vers=2.0,cache=strict,username=smb,uid=1000,noforceuid,gid=1000,noforcegid,addr=IP.ADDR.REDAC.TED,file_mode=0644,dir_mode=0755,soft,nounix,mapposix,rsize=65536,wsize=65536,bsize=1048576,echo_interval=60,actimeo=1,closetimeo=5 0 0

smb share is exported from miktrotik device (ROS v7.6)

# rpm -qa | egrep 'samba|cifs'
cifs-utils-info-6.15-2.fc37.x86_64
cifs-utils-6.15-2.fc37.x86_64
samba-common-4.17.4-2.fc37.noarch
samba-client-libs-4.17.4-2.fc37.x86_64
samba-common-libs-4.17.4-2.fc37.x86_64
samba-libs-4.17.4-2.fc37.x86_64
samba-dc-libs-4.17.4-2.fc37.x86_64
python3-samba-4.17.4-2.fc37.x86_64
python3-samba-dc-4.17.4-2.fc37.x86_64
samba-dcerpc-4.17.4-2.fc37.x86_64
samba-winbind-modules-4.17.4-2.fc37.x86_64
samba-ldb-ldap-modules-4.17.4-2.fc37.x86_64
samba-common-tools-4.17.4-2.fc37.x86_64
samba-4.17.4-2.fc37.x86_64
samba-winbind-4.17.4-2.fc37.x86_64
samba-dc-provision-4.17.4-2.fc37.noarch
samba-dc-4.17.4-2.fc37.x86_64
samba-client-4.17.4-2.fc37.x86_64

Comment 3 Andrew G. Dunn 2023-01-05 20:12:44 UTC
Want to confirm that I just hit this on Silverblue 37.

Comment 4 Andrew G. Dunn 2023-01-05 20:17:53 UTC
Apologies for the double update, submitted to early.

Client is Fedora Silverblue 37, fresh install from media today works fine, but updating to the composition available today causes system lockup. 

Server is Centos 9 Stream: rpm -qa | egrep 'samba|cifs'

samba-common-4.17.2-103.el9.noarch
samba-common-libs-4.17.2-103.el9.x86_64
samba-client-libs-4.17.2-103.el9.x86_64
samba-libs-4.17.2-103.el9.x86_64
python3-samba-4.17.2-103.el9.x86_64
samba-ldb-ldap-modules-4.17.2-103.el9.x86_64
samba-common-tools-4.17.2-103.el9.x86_64
samba-4.17.2-103.el9.x86_64

Comment 5 Andrew G. Dunn 2023-01-05 22:05:02 UTC
## Fresh Silverblue 37

Silverblue install (Fedora-Silverblue-ostree-x86_64-37-1.7.iso).

[agd@fedora ~]$ uname -r
6.0.7-301.fc37.x86_64

[agd@fedora ~]$ rpm -qa | egrep 'samba|cifs'
samba-common-4.17.1-1.fc37.noarch
samba-client-libs-4.17.1-1.fc37.x86_64
samba-common-libs-4.17.1-1.fc37.x86_64
cifs-utils-6.15-2.fc37.x86_64
cifs-utils-info-6.15-2.fc37.x86_64
samba-client-4.17.1-1.fc37.x86_64

cifs mounts work as expected.

## Update to todays composition

Update (Version: 37.20230105.0 (2023-01-05T00:41:12Z)):

kernel 6.0.7-301.fc37 -> 6.0.16-300.fc37
kernel-core 6.0.7-301.fc37 -> 6.0.16-300.fc37
kernel-modules 6.0.7-301.fc37 -> 6.0.16-300.fc37
kernel-modules-extra 6.0.7-301.fc37 -> 6.0.16-300.fc37

Reboot, cifs mounts cause full system lockup.

## Override with a more modern kernel

Attempt override by snagging a newer build (https://koji.fedoraproject.org/koji/buildinfo?buildID=2106802)

[agd@fedora ~]$ rpm-ostree override replace kernel-6.1.3-200.fc37.x86_64.rpm kernel-core-6.1.3-200.fc37.x86_64.rpm kernel-modules-6.1.3-200.fc37.x86_64.rpm kernel-modules-extra-6.1.3-200.fc37.x86_64.rpm

Reboot, cifs mounts work as expected.

Comment 6 Jan Cipa 2023-01-06 07:09:32 UTC
I can confirm, that Andrew's solution (upgrade to 6.1.3) works.

Comment 7 Marien Zwart 2023-01-07 14:43:17 UTC
https://bugzilla.kernel.org/show_bug.cgi?id=216895 looks like the same issue. It says that backporting commit 9ee2afe5207b ("cifs: prevent copying past input buffer boundaries") will fix it.

Comment 8 Slava Bacherikov 2023-01-09 14:38:42 UTC
I just hit this issue on Fedora 37 (6.0.17). Interestingly for me issue happens not on mount, but on calling find over cifs mount. While `ls -R` and other tools works just fine with the mounted filesystem (at least it appears so).

```
crash> bt
PID: 6118   TASK: ffff8ec812e7a8c0  CPU: 11  COMMAND: "find"
 #0 [ffffb725888c37e0] machine_kexec at ffffffffb4070a6c
 #1 [ffffb725888c3830] __crash_kexec at ffffffffb41be427
 #2 [ffffb725888c38f0] panic at ffffffffb4d51e7c
 #3 [ffffb725888c3970] cifs_get_inode_info at ffffffffc0f1a48c [cifs]
 #4 [ffffb725888c3b08] cifs_lookup at ffffffffc0f076de [cifs]
 #5 [ffffb725888c3b60] __lookup_slow at ffffffffb43c7636
 #6 [ffffb725888c3bb0] walk_component at ffffffffb43cbbcb
 #7 [ffffb725888c3be0] path_lookupat at ffffffffb43cc417
 #8 [ffffb725888c3c18] filename_lookup at ffffffffb43ce2d3
 #9 [ffffb725888c3d40] vfs_statx at ffffffffb43c0df2
#10 [ffffb725888c3d90] vfs_fstatat at ffffffffb43c10d1
#11 [ffffb725888c3db8] __do_sys_newfstatat at ffffffffb43c132e
#12 [ffffb725888c3e60] do_syscall_64 at ffffffffb4db915b
#13 [ffffb725888c3ea0] handle_mm_fault at ffffffffb430f49e
#14 [ffffb725888c3ed8] do_user_addr_fault at ffffffffb408084f
#15 [ffffb725888c3f28] exc_page_fault at ffffffffb4dbd9c0
#16 [ffffb725888c3f50] entry_SYSCALL_64_after_hwframe at ffffffffb4e0009b
    RIP: 00007f5106a065be  RSP: 00007ffca4e9d738  RFLAGS: 00000246
    RAX: ffffffffffffffda  RBX: 000055b7586e6a70  RCX: 00007f5106a065be
    RDX: 000055b7586e6ae0  RSI: 000055b7586e6b70  RDI: 00000000ffffff9c
    RBP: 000055b7586e6ae0   R8: 0000000000000001   R9: 000055b7586e4b60
    R10: 0000000000000100  R11: 0000000000000246  R12: 000055b7586e57c0
    R13: 0000000000000000  R14: 0000000000000000  R15: 00007ffca4e9e53f
    ORIG_RAX: 0000000000000106  CS: 0033  SS: 002b
crash> sym ffffffffc0f1a48c
ffffffffc0f1a48c (T) cifs_get_inode_info+3020 [cifs] /usr/src/debug/kernel-6.0.17/linux-6.0.17-300.fc37.x86_64/fs/cifs/inode.c: 608
```

For now I've switched to `5.15.86` from copr: https://copr.fedorainfracloud.org/coprs/kwizart/kernel-longterm-5.15/ (it works without issues).

Comment 9 Slava Bacherikov 2023-01-09 14:44:55 UTC
Created attachment 1936802 [details]
Dmesg with crash

Comment 10 Adam Williamson 2023-01-11 01:23:52 UTC
This is reported fixed by https://bodhi.fedoraproject.org/updates/FEDORA-2023-9dc8ecf7fa (by both a tester and the kernel team).

Comment 11 Henrique Martins 2023-01-12 17:13:59 UTC
Just updated to 6.0.18 from 6.0.15.  No kernel panics, but mounts with SMB 1.0 stopped working, taking a long time to timeout.  SMB 2.1 works, except for the two ancient XP machines where I need 1.0.


Note You need to log in before you can comment on or make changes to this bug.