Bug 1463163

There is a familiar error info with the scsi controller enabled virtio iommu,so
I add the comment here . If they are caused by different reasons and need to file a new bug, I will.


1. In host, add "iommu=pt intel_iommu=on" to kernel line
2. Add "intel_iommu=on" to kernel line of  q35  guest
3. Add the xml as below in the guest 
   <features>
    <ioapic driver='qemu'/>
  </features>
  ...
   <iommu model='intel'>
      <driver intremap='on' iotlb='on'/>
   </iommu>
 
4. Add the iommu='on' ats='on' to scsi controller and change the disk bus to scsi.
....
     <controller type='scsi' index='0' model='virtio-scsi'>
       <driver iommu='on' ats='on'/>
     </controller>
....
     <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none  iommu='on' ats='on'/>
      <source file='/nfs/rhel7.3-q35.qcow2'/>
      <target dev='vda' bus='scsi'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </disk>


5. Start the guest and find the guest can not be accessed.

Created attachment 1293726 [details]
bios that supports IOMMU_PLATFORM

Please try the attached bios.bin to see if it solves the issue for 7.4 guest.

Note: you need to use qemu-kvm -bios bios.bin to test.

Thanks

(In reply to jason wang from comment #4)
> Please try the attached bios.bin to see if it solves the issue for 7.4 guest.
> 
> Note: you need to use qemu-kvm -bios bios.bin to test.
> 
> Thanks


Yes, I use the attached bios.bin to start the guest.

For the disk, it works and can start successfully enabled iommu
For the scsi controller, it does not work and the guest hanged

(In reply to Jingjing Shao from comment #6)
> (In reply to jason wang from comment #4)
> > Please try the attached bios.bin to see if it solves the issue for 7.4 guest.
> > 
> > Note: you need to use qemu-kvm -bios bios.bin to test.
> > 
> > Thanks
> 
> 
> Yes, I use the attached bios.bin to start the guest.
> 
> For the disk, it works and can start successfully enabled iommu
> For the scsi controller, it does not work and the guest hanged

Interesting, scsi seems to work for upstream qemu.

Please show me

1) qemu cli & version
2) guest kernel version
3) what phase did guest hang? did you see grub?

Thanks

Works for me with following cli:

$qemu_path -bios bios.bin \
           -drive file=$img_path,if=none,id=hd \
           -device virtio-scsi-pci,id=scsi,disable-legacy=on,disable-modern=off,iommu_platform=on \
           -device scsi-hd,drive=hd \
           -vnc :10 \
           -netdev tap,id=hn0,vhost=on,queues=4 \
           -device virtio-net-pci,netdev=hn0,vectors=10,mq=on,mrg_rxbuf=off,guest_tso4=off,guest_tso6=off,guest_ecn=off,guest_ufo=off,mac=$mac \
           -m 2G -enable-kvm \
           -cpu host \
           -smp 4 \
           -serial stdio \
           -snapshot

But I'm using upstream kernel as guest.

> Interesting, scsi seems to work for upstream qemu.
> 
> Please show me
> 
> 1) qemu cli & version
> 2) guest kernel version
> 3) what phase did guest hang? did you see grub?
> 
> Thanks

qemu-kvm-rhev-2.9.0-14.el7.x86_64
guest kernel : 3.10.0-691.el7.x86_64

qemu cli:
 -device virtio-scsi-pci,iommu_platform=on,ats=on,id=scsi0,bus=pci.3,addr=0x0
 -drive file=/var/lib/libvirt/images/q35-bios.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-0 -device scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1

But maybe I found the point.

If I add the "iommu=on" to the guest kernel command line, I can reproduce this scsi issue and add the image as attachment.

If I delete the "iommu=on" from the guest kernel command line, the guest can boot successfully.

But the disk part is OK for both scenarios. 


Additional info:
I also try the NIC device enable virtio iommu with the guest with "iommu=on" in kernel command line. The guest can not start successfully.

qemu cli:
-device virtio-net-pci,netdev=hostnet1,id=net1,mac=52:54:00:7e:af:e6,bus=pci.7,addr=0x0,iommu_platform=on,ats=on

Created attachment 1294097 [details]
scsi controller enable virtio iommu

Thanks, intel_iommu=on works for upstream kernel. Let me try RHEL.

Btw, could you please try gpxe to see if it works? If net, please open a new bz for gpxe and assign it to me.

Thanks a lot.

(In reply to jason wang from comment #11)
> Thanks, intel_iommu=on works for upstream kernel. Let me try RHEL.

Speak too fast, looks like upstream is broken too. Please file a new bug with component set to kernel, and assign it to me.

Thanks

(In reply to jason wang from comment #15)
> (In reply to jason wang from comment #11)
> > Thanks, intel_iommu=on works for upstream kernel. Let me try RHEL.
> 
> Speak too fast, looks like upstream is broken too. Please file a new bug
> with component set to kernel, and assign it to me.
> 
> Thanks

Double check with jason, file new qemu-kvm-rhev bug for the scsi issue. 

https://bugzilla.redhat.com/show_bug.cgi?id=1467811

Seabios path has been posted upstream.

Patch accepted upstream.

Fixed in seabios-1.10.2-4.el7

Reproduced bug with seabios-1.10.2-3.el7.x86_64 & qemu-kvm-rhev-2.10.0-7.el7.x86_64 & kernel-3.10.0-798.el7.x86_64

result:
The guest OS will report  "No bootable device" 

Verified bug with seabios-1.11.0-1.el7.x86_64 & qemu-kvm-rhev-2.10.0-7.el7.x86_64 & kernel-3.10.0-798.el7.x86_64

Guest works well.


steps:

For q35
1. In host, add "iommu=pt intel_iommu=on" to kernel line
2. Add "intel_iommu=on" to kernel line of q35  guest
3. iommu_platform=on,ats=on to "-device virtio-blk-pci...."
4. key qemu command

-machine pc-q35-rhel7.5.0
-drive file=/home/seabio-new-system-disk.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,bus=pci.4,addr=0x0,drive=drive-virtio-disk0,id=virtio-disk0,iommu_platform=on,ats=on,bootindex=1

For pc

5.key qemu command

/usr/libexec/qemu-kvm -name guest=q35-seabios,debug-threads=on -machine pc -cpu SandyBridge,vmx=on -m 8192 -realtime mlock=off -smp 4,sockets=2,cores=2,threads=1 -drive file=/home/seabio-new-system-disk.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,iommu_platform=on,ats=on,disable-legacy=on,disable-modern=off,bootindex=1 -netdev tap,fd=20,id=hostnet0,vhost=on -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:ee:67:31,iommu_platform=on,ats=on,disable-legacy=on,disable-modern=off -vnc :2 -monitor stdi

According to comment46, set this bug as verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0814

Clear the unnecessary needinfo flag as the bug has been fixed.

Thanks