Bug 2362414

Created attachment 2087321 [details]
mlx4_core failed to initialize the device, reporting command 0x23 timed out

Created attachment 2087321 [details]
mlx4_core failed to initialize the device, reporting command 0x23 timed out

1. Please describe the problem:

mlx4_core cannot initialize the network device.

2. What is the Version-Release number of the kernel:

Linux version 6.14.3-300.fc42.x86_64 (mockbuild@f38d0f39c8424fc98f6be30363aa5e29) (gcc (GCC) 15.0.1 20250329 (Red Hat 15.0.1-0), GNU ld version 2.44-3.fc42) #1 SMP PREEMPT_DYNAMIC Sun Apr 20 16:08:39 UTC >

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :

Yes. It works on Fedora 41 kernel + Fedora 42 remaining. Kernel version

Linux version 6.13.12-200.fc41.x86_64 (mockbuild@9778ff19adb64779af61fc903691b7f0) (gcc (GCC) 14.2.1 20250110 (Red Hat 14.2.1-7), GNU ld version 2.43.1-5.fc41) #1 SMP PREEMPT_DYNAMIC Sun Apr 20 15:52:43 U>

4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:

Reboot the system, and the initialization failed.
modprobe -r mlx4 related modules and modprobe mlx4_en can also reproduce the dmesg

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:

Yes. Rawhide still has this problem, kernel version:

kernel: Linux version 6.15.0-0.rc3.20250424gita79be02bba5c.31.fc43.x86_64 (mockbuild@f7d7d97eb2d74025a7388a5e09a583ad) (gcc (GCC) 15.0.1 20250418 (Red Hat 15.0.1-0), GNU ld version 2.44-3.fc43) #1 SMP PREEMPT_DYN>

6. Are you running any modules that not shipped with directly Fedora's kernel?:

No. This is newly upgraded Fedora 42

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

Related sections:

mlx4_core: Mellanox ConnectX core driver v4.0-0
mlx4_core: Initializing 0000:05:00.0
mlx4_core 0000:05:00.0: DMFS high rate steer mode is: disabled performance optimized steering
mlx4_core 0000:05:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x4 link at 0000:03:02.0 (capable of 63.008 Gb/s with 8.0 GT/s PCIe x8 link)
mlx4_core 0000:05:00.0: command 0x23 timed out (go bit not cleared)
mlx4_core 0000:05:00.0: device is going to be reset
mlx4_core 0000:05:00.0: crdump: FW doesn't support health buffer access, skipping
mlx4_core 0000:05:00.0: device was reset successfully
mlx4_core 0000:05:00.0: Failed to initialize queue pair table, aborting
mlx4_core 0000:05:00.0: probe with driver mlx4_core failed with error -5

Working dmesg looks like:

kernel: mlx4_core 0000:05:00.0: DMFS high rate steer mode is: disabled performance optimized steering
kernel: mlx4_core 0000:05:00.0: 31.504 Gb/s available PCIe bandwidth, limited by 8.0 GT/s PCIe x4 link at 0000:03:02.0 (capable of 63.008 Gb/s with 8.0 GT/s PCIe x8 link)
kernel: mlx4_en: Mellanox ConnectX HCA Ethernet driver v4.0-0
kernel: mlx4_en 0000:05:00.0: Activating port:1
kernel: mlx4_en: 0000:05:00.0: Port 1: Using 24 TX rings
kernel: mlx4_en: 0000:05:00.0: Port 1: Using 16 RX rings
kernel: mlx4_en: 0000:05:00.0: Port 1: Initializing port
kernel: mlx4_en 0000:05:00.0: registered PHC clock
kernel: mlx4_en 0000:05:00.0: Activating port:2
kernel: mlx4_en: 0000:05:00.0: Port 2: Using 24 TX rings
kernel: mlx4_en: 0000:05:00.0: Port 2: Using 16 RX rings
kernel: mlx4_en: 0000:05:00.0: Port 2: Initializing port