Bug 1830091 - VM does not start with PCI passthrough
Summary: VM does not start with PCI passthrough
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 32
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Libvirt Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-30 20:02 UTC by zabra
Modified: 2020-05-19 02:38 UTC (History)
11 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2020-05-05 13:05:34 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description zabra 2020-04-30 20:02:54 UTC
Description of problem:

I have a VM that uses PCI passthrough to pass a GPU to a windows guest.
Everything worked fine until I decided to update Fedora from version 31 to 32.
After that I cannot start the VM anymore.
I traced it down to qemu quitting with an error:

2020-04-30T19:30:03.900479Z qemu-system-x86_64: VFIO_MAP_DMA: -14
2020-04-30T19:30:03.900499Z qemu-system-x86_64: vfio_dma_map(0x5581f874a870, 0x812000000, 0x2000000, 0x7fb567ffd000) = -14 (Bad address)
qemu: hardware error: vfio: DMA mapping failed, unable to continue


Version-Release number of selected component (if applicable): 6.1.0-2.fc32

Version of qemu: 4.2.0-7.fc32


How reproducible: Always


Steps to Reproduce:
1. Setup vfio-pci kernel module to bind to the device
2. Add the PCI device to the VM
3. Start VM

Actual results: VM does not boot


Expected results: VM should boot


Additional info:

AMD Ryzen 3700X
Gigabyte x570 Aorus Pro

PCI-device to pass through: NVIDIA GeForce 1060

Comment 1 zabra 2020-05-01 08:22:46 UTC
Works with qemu 5.0.0 compiled from source, so I guess something changed in qemu from Fedora 32 repository that causes it to break.

Comment 2 Cole Robinson 2020-05-01 13:23:17 UTC
Thanks for the report. My GPU passthrough VM boots fine on Fedora 32 with stock virt packages. I also tested with qemu 5.0.0-rc3 from virt-preview repo and my VM booted fine.

Can you provide: rpm -q qemu-system-x86 libvirt-daemon

Comment 3 zabra 2020-05-01 13:38:20 UTC
Sure:

rpm -q qemu-system-x86 libvirt-daemon

qemu-system-x86-4.2.0-7.fc32.x86_64
libvirt-daemon-6.1.0-2.fc32.x86_64

Comment 4 Cole Robinson 2020-05-01 13:42:56 UTC
aw have you seen this error before:

2020-04-30T19:30:03.900479Z qemu-system-x86_64: VFIO_MAP_DMA: -14
2020-04-30T19:30:03.900499Z qemu-system-x86_64: vfio_dma_map(0x5581f874a870, 0x812000000, 0x2000000, 0x7fb567ffd000) = -14 (Bad address)
qemu: hardware error: vfio: DMA mapping failed, unable to continue

Comment 5 zabra 2020-05-01 14:02:43 UTC
When I disable SELinux I can also also start the VM with the package provided qemu.

To clarify here are the exact steps that I did:
* first notice that it does not work with package provided qemu
* compiled 5.0.0
* got a different error when starting: "Error starting domain: internal error: process exited while connecting to monitor: libvirt:  error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission denied"
* disabled SELinux
* everything worked fine
* assumed a problem in the package provided version
* tried to go back to package provided qemu and see if I can reproduce it with SELinux turned off
* could not reproduce, everything worked fine now
* enabled SELinux, rebooted, relabelling took place
* tried to start again: did not work again

Comment 6 zabra 2020-05-02 07:25:16 UTC
OK I think I know now what the problem is. I have a shared memory device that is used for looking glass that seems to be the problem:

type=AVC msg=audit(1588404111.774:675): avc:  denied  { getattr } for  pid=21842 comm="qemu-system-x86" path="/dev/shm/looking-glass" dev="tmpfs" ino=4467 scontext=system_u:system_r:svirt_t:s0:c74,c515 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1

Somehow SELinux is now blocking the access, but that seems to be a problem on my side then.

You can close the issue. Thanks!

Comment 7 Cole Robinson 2020-05-05 13:05:34 UTC
Thanks for following up, closing

If you are using libvirt though, and not using the qemu-commandline passthrough feature, maybe libvirt should be labeling that path for you, so selinux 'just works'. So it could be a libvirt bug. I don't know anything about looking-glass though

Comment 8 zabra 2020-05-09 19:20:07 UTC
Hm fair point, I'll lay out the things that I did so you or the person in charge to make the decision can judge.

I have a shmem entry in the libvirt config of my VM:

$ virsh edit win10

...
   <shmem name='looking-glass'>
      <model type='ivshmem-plain'/>
      <size unit='M'>32</size>
      <address type='pci' domain='0x0000' bus='0x0b' slot='0x01' function='0x0'/>
    </shmem>
...


Also I have a file in /etc/tmpfiles.d for changing the user

$ cat /etc/tmpfiles.d/10-looking-glass.conf 
f	/dev/shm/looking-glass	0660	fb	kvm	-


After the VM is up I start a program from the terminal as user fb that does r/w on the /dev.


Note You need to log in before you can comment on or make changes to this bug.