Bug 1945565 - Unable to add device to vfio driver
Summary: Unable to add device to vfio driver
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 34
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-01 09:52 UTC by Idar Lund
Modified: 2021-09-03 06:54 UTC (History)
18 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2021-09-03 06:54:50 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
dmesg log (1.05 MB, text/plain)
2021-04-01 09:58 UTC, Idar Lund
no flags Details

Description Idar Lund 2021-04-01 09:52:34 UTC
1. Please describe the problem:
When writing (echo) to /sys/bus/pci/drivers/vfio-pci/new_id in kernel 5.11.x (and 5.12.x and 5.13.x) I get the following error: "-bash: echo: write error: File exists"


2. What is the Version-Release number of the kernel:
Linux silje 5.11.9-200.fc33.x86_64 #1 SMP Wed Mar 24 12:09:04 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Linux silje 5.13.9-200.fc34.x86_64 #1 SMP Sun Aug 8 14:34:00 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux


3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
Yes, this worked in kernel 5.10. I've tested with:
Linux silje 5.10.17-200.fc33.x86_64 #1 SMP Wed Feb 17 21:21:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux


4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
[root@silje ~]# lspci -v -s 06:00.0
06:00.0 USB controller: Fresco Logic FL1100 USB 3.0 Host Controller (rev 10) (prog-if 30 [XHCI])
        Subsystem: Fresco Logic FL1100 USB 3.0 Host Controller
        Flags: bus master, fast devsel, latency 0, IRQ 41, IOMMU group 26
        Memory at f6600000 (64-bit, non-prefetchable) [size=64K]
        Memory at f6611000 (64-bit, non-prefetchable) [size=4K]
        Memory at f6610000 (64-bit, non-prefetchable) [size=4K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 00
        Capabilities: [b0] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [100] Advanced Error Reporting
        Kernel driver in use: xhci_hcd

[root@silje ~]# cat /sys/bus/pci/devices/0000\:06\:00.0/vendor
0x1b73

[root@silje ~]# cat /sys/bus/pci/devices/0000\:06\:00.0/device
0x1100

[root@silje ~]# ls -laFh /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
--w-------. 1 root root 4.0K Apr  1 11:33 /sys/bus/pci/devices/0000:06:00.0/driver/unbind

[root@silje ~]# ls -laFh /sys/bus/pci/drivers/vfio-pci/new_id
--w-------. 1 root root 4.0K Apr  1 11:33 /sys/bus/pci/drivers/vfio-pci/new_id

[root@silje ~]# echo '0000:06:00.0' > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind

[root@silje ~]# ls -laFh /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
ls: cannot access '/sys/bus/pci/devices/0000:06:00.0/driver/unbind': No such file or directory (as expected!)

[root@silje ~]# echo 0x1b73 0x1100 > /sys/bus/pci/drivers/vfio-pci/new_id
-bash: echo: write error: File exists

In 5.10 the error message "-bash: echo: write error: File exists" does not show and when checking the device, it's added to vfio:
[root@silje ~]# ls -laFh /sys/bus/pci/drivers/vfio-pci/|grep "06:"
lrwxrwxrwx.  1 root root    0 Apr  1 11:38 0000:06:00.0 -> ../../../../devices/pci0000:00/0000:00:01.2/0000:02:00.0/0000:03:06.0/0000:06:00.0/

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
Not tested

6. Are you running any modules that not shipped with directly Fedora's kernel?:
Yes (but unrelated);
5.10 I have VmWare modules
5.11 Does only have shipped modules.

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.
On 5.10 now, but will reboot and add kernel logs for 5.11 later.


EDIT: Updated to F34. Problem still exist in 5.13.

Comment 1 Idar Lund 2021-04-01 09:58:06 UTC
Created attachment 1768228 [details]
dmesg log

Added dmesg log for 5.11 kernel.
Log capture after trying to do "echo 0x1b73 0x1100 > /sys/bus/pci/drivers/vfio-pci/new_id"

Comment 2 Idar Lund 2021-08-13 07:36:05 UTC
Could be related to this:
https://www.spinics.net/lists/linux-pci/msg102243.html

Comment 3 Idar Lund 2021-09-03 06:54:50 UTC
Just for reference, this use-case was solved by Alex Williamson's proposal in the vfio mailinglist:

Instead of:
echo '0000:06:00.0' > /sys/bus/pci/devices/0000\:06\:00.0/driver/unbind
echo 0x1b73 0x1100 > /sys/bus/pci/drivers/vfio-pci/new_id

I did:
driverctl set-override 0000:06:00.0 vfio-pci

And that worked in kernel 5.12


Note You need to log in before you can comment on or make changes to this bug.