Bug 1575759

Summary: VIFO on intel graphics not working.
Product: [Fedora] Fedora Reporter: Chris K. <kaspro>
Component: qemuAssignee: Fedora Virtualization Maintainers <virt-maint>
Status: CLOSED DEFERRED QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 28CC: amit, berrange, cfergeau, crobinso, dwmw2, itamar, pbonzini, rjones, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-08 14:45:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chris K. 2018-05-07 20:07:25 UTC
Description of problem:

I'm unable to forward guest graphics adapter configured in libvirtd to the host graphics adapter via vfio.
Hardware: Skylake Celeron CPU G3900 on Fujitsu D3471-B11 Mainboard, integrated  IntelĀ® HD Graphics 510.


Version-Release number of selected component (if applicable):

Fedora 28, latest updates, libvirt 4.1.0, qemu 2.11.1-2.fc28


How reproducible / Steps to Reproduce:
1. Block i915 and other drivers from loading, force vfio drivers to load:

# cat /etc/modules-load.d/vfio.conf
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
# cat /etc/modprobe.d/vfio.conf
options vfio-pci ids=8086:1902
options vfio-pci disable_vga=1
# cat /etc/modprobe.d/blacklist.conf
blacklist i915
# dracut -f /boot/initramfs-4.16.5-300.fc28.x86_64.img 4.16.5-300.fc28.x86_64
# reboot
# dmesg | grep -e DMAR -e IOMMU
Kernel command line: BOOT_IMAGE=/vmlinuz-4.16.5-300.fc28.x86_64 root=/dev/mapper/fedora-root00 ro rd.lvm.lv=fedora/root00 rd.lvm.lv=fedora/swap rhgb quiet LANG=en_US.UTF-8 intel_iommu=on rd.driver.pre=vfio-pci bochs_drm.fbdev=off video=efifb:off,vesa:off,vesafb:off vga=normal iommu=no-igfx
    0.000000] ACPI: DMAR 0x00000000A6679ED8 0000C8 (v01 INTEL  SKL      00000001 INTL 00000001)
[    0.000000] DMAR: IOMMU enabled
[    0.001000] DMAR: Host address width 39
[    0.001000] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.001000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 7e3ff0505e
[    0.001000] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.001000] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.001000] DMAR: RMRR base: 0x000000a6404000 end: 0x000000a6423fff
[    0.001000] DMAR: RMRR base: 0x000000adb9e000 end: 0x000000adde7fff
[    0.001000] DMAR: RMRR base: 0x000000af000000 end: 0x000000cf7fffff
[    0.001000] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.001000] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.001000] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[    0.001000] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[    0.002000] DMAR-IR: Enabled IRQ remapping in xapic mode
[    0.357713] DMAR: No ATSR found
[    0.357755] DMAR: dmar0: Using Queued invalidation
[    0.357757] DMAR: dmar1: Using Queued invalidation
[    0.357872] DMAR: Setting RMRR:
[    0.357901] DMAR: Setting identity map for device 0000:00:02.0 [0xaf000000 - 0xcf7fffff]
[    0.357927] DMAR: Setting identity map for device 0000:00:14.0 [0xadb9e000 - 0xadde7fff]
[    0.357936] DMAR: Setting identity map for device 0000:00:14.0 [0xa6404000 - 0xa6423fff]
[    0.357938] DMAR: Prepare 0-16MiB unity mapping for LPC
[    0.357955] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[    0.357963] DMAR: Intel(R) Virtualization Technology for Directed I/O
...
...
[    1.113517] VFIO - User Level meta-driver version: 0.3
[    1.140028] vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    1.152032] vfio_pci: add [8086:1902[ffff:ffff]] class 0x000000/00000000
...

2.: Host boots with graphics adapter stalled after grub, as it should. Configuring a guest with a legacy BIOS:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
...
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </source>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </hostdev>
...
  </devices>
  <qemu:commandline>
    <qemu:arg value='-nographic'/>
    <qemu:arg value='-nodefaults'/>
  </qemu:commandline>
...

starts up with practically no errors, but no signal output. Logs:
...
...vfio-pci 0000:00:02.0: enabling device (0400 -> 0403)
...vfio_ecap_init: 0000:00:02.0 hiding ecap 0x1b@0x100
...

3.: Configuring a guest with UEFI BIOS, 440FX Chipset gives (and thousands of these... not just some):
[ 3419.435439] DMAR: DRHD: handling fault status reg 2
[ 3419.435446] DMAR: [DMA Write] Request device [00:02.0] fault addr 0 [fault reason 02] Present bit in context entry is clear
[ 3419.436683] DMAR: DRHD: handling fault status reg 3
[ 3419.436691] DMAR: [DMA Read] Request device [00:02.0] fault addr afa40000 [fault reason 06] PTE Read access is not set
[ 3419.438069] DMAR: DRHD: handling fault status reg 3
[ 3419.438076] DMAR: [DMA Read] Request device [00:02.0] fault addr afa40000 [fault reason 06] PTE Read access is not set
[ 3419.439473] DMAR: DRHD: handling fault status reg 3
[ 3419.439480] DMAR: [DMA Read] Request device [00:02.0] fault addr afa6e000 [fault reason 06] PTE Read access is not set
[ 3419.440872] DMAR: DRHD: handling fault status reg 3
[ 3419.440879] DMAR: [DMA Read] Request device [00:02.0] fault addr afaa6000 [fault reason 06] PTE Read access is not set
...

and a washed out/scrambled output on the digital output (like the actual startup of the guest, but unreadable and with lines all over the screen.


Actual results:
No correct signal on digital output, scrambled or blank.

Expected results:
Digital output to show guest graphics.

Additional info:
Also tried with iommu= "pt" and "on".

Comment 1 Cole Robinson 2018-05-08 14:45:21 UTC
I suggest you report your issues to the vfio-users list, the vfio developer actively watches things there along with other users who might be able to give feedback

If your thread there finds a bug we can reopen this issue to track getting the fix included in Fedora, but otherwise this bug isn't going to generate much usefulness

https://www.redhat.com/archives/vfio-users/