Bug 1458192

Hi, to me this failure doesn't seem related to the guest OS. The OVMF log that you attached -- thanks for that BTW -- ends with:

> XhcSetRootHubPortFeature: status Success
> XhcClearRootHubPortFeature: status Success
> XhcClearRootHubPortFeature: status Success
> Disable device slot 1!
> Enable Slot Successfully, The Slot ID = 0x1
>     Address 1 assigned successfully
> UsbIoPortReset: device is now ADDRESSED at 1
> ASSERT /builddir/build/BUILD/ovmf-c325e41585e3/MdeModulePkg/Bus/Pci/XhciDxe/XhciSched.c(1800): TrsRing != ((void *) 0)

So there's an assertion failure in the USB3 / XHCI driver before we even attempt to boot the OS.

How reliably does the issue reproduce? If very reliably, we could try to reproduce it with upstream edk2 and/or upstream QEMU, and hunt for USB3 fixes.

Usb-bot devices are not supported in RHEL (libvirt doesn't support them). It might be worth fixing in the future (upstream), but it's not important right now.

Related patch:

[edk2] [PATCH 0/1] OvmfPkg/QemuBootOrderLib: recognize "usb-storage" devices
                   in XHCI ports
https://lists.01.org/pipermail/edk2-devel/2017-September/014664.html

Part of the log on the patch is worth quoting, so as to set expectations and
the scope:

> The feature is primarily useful for modern AARCH64 guests that have no
> built-in virtio block / SCSI drivers; as on "qemu-system-aarch64 -M virt",
> there are no IDE or AHCI controllers that could be used as fallback. XHCI
> is available in "-M virt" however, and because XHCI predates AARCH64 by
> several years, said guests are expected to have built-in drivers for it.
>
> Other device models ("usb-uas", "usb-bot") are out of scope for now,
> similarly to USB1.x (UHCI) and USB2 (EHCI) host controllers, and similarly
> to USB hubs (which are USB1.1 only). In particular, port mapping between
> EHCI and companion UHCI controllers is very complex; it even leads to PCI
> slot/function differences between the OpenFirmware device paths exported
> by QEMU and the the UEFI device paths generated by edk2.
>
> The number of ports on the XHCI controller defaults to 4, but it can be
> raised via the "p3" property to 15. In addition, several XHCI controllers
> can be grouped into a single-slot, multi-function PCI device. These allow
> for a good number of usb-storage devices, while their desired boot order
> remains recognizable to this patch.

"usb-storage" is what libvirt supports, and plugging such directly into qemu-xhci or nec-xhci controllers will work once the above patch is committed. Using 8 XHCI controllers (one per function) in a given PCI slot, and using 15 XHCI ports per controller, a single PCI slot can accommodate 120 usb-storage devices.

Furthermore, physical USB sticks and drives should be exposed to the guest
via "SCSI LUN passthrough", with virtio-scsi. (These devices appear as SCSI
disks on the host side).

(In reply to Laszlo Ersek from comment #5)

> "usb-storage" is what libvirt supports, and plugging such directly into
> qemu-xhci or nec-xhci controllers will work once the above patch is
> committed.

The devices already work; I meant that boot order manipulation will start working.

Example command line and domain XML fragments, also from the upstream patch log:

qemu:

  -device qemu-xhci,id=xhci1,p3=15,addr=02.0,multifunction=on \
  -device qemu-xhci,id=xhci2,p3=15,addr=02.1 \
  \
  -drive id=disk1,if=none,format=qcow2,$DISK1_OPTIONS \
  -drive id=disk2,if=none,format=qcow2,$DISK2_OPTIONS \
  \
  -device usb-storage,drive=disk1,bus=xhci1.0,port=3,bootindex=1 \
  -device usb-storage,drive=disk2,bus=xhci2.0,port=9,bootindex=2 \

libvirt:

  <controller type='usb' index='1' model='qemu-xhci' ports='15'>
    <address type='pci'
     domain='0x0000' bus='0x00' slot='0x02' function='0x0'
     multifunction='on'/>
  </controller>
  <controller type='usb' index='2' model='qemu-xhci' ports='15'>
    <address type='pci'
     domain='0x0000' bus='0x00' slot='0x02' function='0x1'/>
  </controller>
  <disk type='file' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source file='...'/>
    <target dev='sda' bus='usb'/>
    <boot order='1'/>
    <address type='usb' bus='1' port='3'/>
  </disk>
  <disk type='file' device='disk'>
    <driver name='qemu' type='qcow2'/>
    <source file='...'/>
    <target dev='sdb' bus='usb'/>
    <boot order='2'/>
    <address type='usb' bus='2' port='9'/>
  </disk>

Hit the same error with OVMF-20171011-1.git92d07e48907f.el7.noarch &qemu-kvm-rhev-2.10.0-3.el7.x86_64


UsbMassReadBlocks: UsbBootReadBlocks (Device Error) -> Reset
ASSERT /builddir/build/BUILD/ovmf-92d07e48907f/MdeModulePkg/Bus/Pci/XhciDxe/XhciSched.c(1847): TrsRing != ((void *) 0)


Could you help to confirm it?

Thanks
Jing

Hi Jing,

(sorry about the late reply, I've been out of office,)

did you notice that I had changed the scope of this RHBZ significantly, in comment 5 through comment 7?

I also changed the title of the bug between comment 5 and comment 6.

In summary:

- "usb-bot" devices should not be used at all,

- only "usb-storage" devices may be used, and only with virtual disk images
  (i.e. they should not be used for "passing through" physical USB disks or
  flash drives),

- physical USB sticks and drives should be exposed to the guest only via
  "SCSI LUN passthrough", with virtio-scsi. (These devices appear as SCSI
  disks on the host side.)

For "usb-storage", QEMU command line options, and a libvirt domain XML fragment, can be found in comment 7.

"usb-bot" is not supported by libvirt, boot order mapping for "usb-bot" is basically unsolvable in UEFI, and "usb-storage" can be used in a scalable way (meaning both bandwidth and number of disks) for exposing virtual disks to the guest through an USB interface. Thus, there are no plans for supporting usb-<anything> in OVMF, beyond "usb-storage". This BZ is about boot order mapping for "usb-storage".

Thanks.

Steps to test the patch in comment 5:

(1) Create a new domain with

  - a virtio-scsi CD-ROM containing RHEL or Fedora installer media,

  - a system disk on a "usb-storage" device, placed into a qemu-xhci or
    nec-usb-xhci controller,

  - a data disk on another "usb-storage" device, placed into the same
    qemu-xhci or nec-usb-xhci controller (using a different port).

(2) Optional: if you have more than 4 USB devices in the XHCI controller --
    because two "usb-storage" devices come from point (1), and you will
    likely have a USB tablet and a USB keyboard too --, then raise the port
    count on the XHCI controller. See the "p3" property in comment 7. The
    maximum port count is 15 for a single XHCI controller.

(3) Assign bootindex 1 to the system disk, and bootindex 2 to the CD-ROM.

(4) Launch the VM for the first time. The system disk will be skipped (it is
    empty) and the installer media will be booted.

(5) In the installer, make sure that the logical volume that contains the
    /home filesystem is in a volume group where the data disk is a physical
    volume. The point is that at least a part of /home be placed on the data
    disk.

(6) Reboot as instructed by the installer. The reboot should work correctly
    from the now-installed system disk. Check whether /home can be accessed.

(7) Power off the VM gracefully. Assign bootindex 1 to the CD-ROM and
    bootindex 2 to the system disk. Launch the VM. The installer should be
    booted. At the first Anaconda dialog, force off the VM.

(8) Assign bootindex 1 to the system disk and bootindex 2 to the CD-ROM.
    Launch the VM -- it should boot into the installed system again. Check
    if /home can be accessed.

Verified this bug with OVMF-20171011-1.git92d07e48907f.el7.noarch & qemu-kvm-rhev-2.10.0-4.el7.x86_64 and 3.10.0-774.el7.x86_64

(In reply to Laszlo Ersek from comment #13)
> Steps to test the patch in comment 5:
> 
> (1) Create a new domain with
> 
>   - a virtio-scsi CD-ROM containing RHEL or Fedora installer media,
> 
>   - a system disk on a "usb-storage" device, placed into a qemu-xhci or
>     nec-usb-xhci controller,
> 
>   - a data disk on another "usb-storage" device, placed into the same
>     qemu-xhci or nec-usb-xhci controller (using a different port).
> 
> (2) Optional: if you have more than 4 USB devices in the XHCI controller --
>     because two "usb-storage" devices come from point (1), and you will
>     likely have a USB tablet and a USB keyboard too --, then raise the port
>     count on the XHCI controller. See the "p3" property in comment 7. The
>     maximum port count is 15 for a single XHCI controller.
> 
> (3) Assign bootindex 1 to the system disk, and bootindex 2 to the CD-ROM.

/usr/libexec/qemu-kvm -M q35 -cpu Opteron_G5 -m 4096 -global driver=cfi.pflash01,property=secure,value=on \

-drive file=/home/guest/OVMF_CODE.secboot.verbose.fd,if=pflash,format=raw,readonly=on,unit=0 \
-drive file=/home/guest/OVMF_VARS.fd,if=pflash,format=raw,unit=1,readonly=on \

 -boot menu=on,splash-time=12000 -global driver=cfi.pflash01,property=secure,value=on -monitor stdio \

-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pcie.0 \
-drive file=/home/RHEL7.5-Server-x86_64.iso,if=none,id=cd1,format=raw,snapshot=off,aio=native,cache=none \
-device scsi-hd,id=image1,drive=cd1,bootindex=2 \

-device nec-usb-xhci,id=xhci1,bus=pcie.0,addr=0x6 \

-device usb-storage,drive=drive-usb-1,id=usb-1,removable=on,bus=xhci1.0,port=1,bootindex=1 \
-drive file=/home/rhel7.5-system-disk.qcow2,if=none,id=drive-usb-1,media=disk,format=qcow2 \

-device usb-storage,drive=drive-usb-2,id=usb-2,removable=on,bus=xhci1.0,port=2 \
-drive file=/home/data-disk.qcow2,if=none,id=drive-usb-2,media=disk,format=qcow2 \

-device usb-tablet,bus=xhci1.0,port=3,id=usb-tablet1 \

-device usb-kbd,bus=xhci1.0,port=4,id=usb-keyboard \

-vnc :1 -debugcon file:/home/test/ovmf.log -global isa-debugcon.iobase=0x402 \

> 
> (4) Launch the VM for the first time. The system disk will be skipped (it is
>     empty) and the installer media will be booted.

Obtain the expected results

> 
> (5) In the installer, make sure that the logical volume that contains the
>     /home filesystem is in a volume group where the data disk is a physical
>     volume. The point is that at least a part of /home be placed on the data
>     disk.

1)create 40G system disk and 40G data disk via qemu-img 
2)create a Volume group,and it includes system disk and data disk.
3)assign all available space to system partition. e.g 

/home 50G
/boot 6G
/boot/efi 5G
/     10G
swap  5G
/var  4G

> 
> (6) Reboot as instructed by the installer. The reboot should work correctly
>     from the now-installed system disk. Check whether /home can be accessed.
> 

Obtain the expected results

> (7) Power off the VM gracefully. Assign bootindex 1 to the CD-ROM and
>     bootindex 2 to the system disk. Launch the VM. The installer should be
>     booted. At the first Anaconda dialog, force off the VM.
>
 
Obtain the expected results

> (8) Assign bootindex 1 to the system disk and bootindex 2 to the CD-ROM.
>     Launch the VM -- it should boot into the installed system again. Check
>     if /home can be accessed.

Obtain the expected results.


Anyway, According to comment 13. I got the expected result. so set this bug as verified.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0902