Bug 1274588 - Can't boot up guest os from passthrough usb device
Can't boot up guest os from passthrough usb device
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: SLOF (Show other bugs)
7.2
ppc64le Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Thomas Huth
Virtualization Bugs
:
Depends On: 1279954
Blocks: RHV4.1PPC RHEV4.0PPC
  Show dependency treegraph
 
Reported: 2015-10-23 01:24 EDT by Xujun Ma
Modified: 2017-07-03 23:11 EDT (History)
9 users (show)

See Also:
Fixed In Version: qemu-slof-20151103
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-04 00:31:47 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Xujun Ma 2015-10-23 01:24:23 EDT
Description of problem:
Can't boot up guest from passthrough usb device.

Version-Release number of selected component (if applicable):
host:
kernel-3.10.0-325.el7.ppc64le
qemu-kvm-rhev-2.3.0-31.el7.ppc64le
SLOF-20150313-5.gitc89b0df.el7.noarch
guest:
kernel-2.6.32-573.el6.ppc64

How reproducible:
100%

Steps to Reproduce:
1.boot up guest and install os with command
/usr/libexec/qemu-kvm \
 -m 16G -smp 16 -monitor stdio -qmp tcp::8888,server,nowait  -vnc :25\
 -usbdevice tablet \
 -netdev tap,id=tap0,script=/etc/qemu-ifup \
 -device virtio-net-pci,netdev=tap0,bootindex=3,id=net0,mac=24:be:05:11:92:11 \
 -device usb-ehci,id=usbctr \
 -device usb-host,hostbus=1,hostaddr=2,id=hostdev0 \
 -device virtio-scsi-pci\
 -device scsi-cd,id=scsi-cd1,drive=scsi-cd1-dr,bootindex=4 \
 -drive file=../iso/RHEL-6.7-20150702.0-Server-ppc64-dvd1.iso,if=none,id=scsi-cd1-dr,readonly=on,format=raw,cache=none \
2.boot up guest os from usb 

Actual results:
can't boot up guest os ,and it stop at slof

Expected results:
can boot up guest os 

Additional info:
Comment 2 Thomas Huth 2015-10-28 09:21:10 EDT
Is there an error message or unusual output from SLOF when the problem occurs? Could you please attach a console log (preferably text when starting with "-nographic", but screenshot is ok, too)?
Comment 3 Xujun Ma 2015-10-29 04:39:05 EDT
(In reply to Thomas Huth from comment #2)
> Is there an error message or unusual output from SLOF when the problem
> occurs? Could you please attach a console log (preferably text when starting
> with "-nographic", but screenshot is ok, too)?

yes ,it is .the whole output as follows:
[root@ibm-p8-rhevm-08 ~]# sh usb.sh 


SLOF **********************************************************************
QEMU Starting
 Build Date = Sep 18 2015 06:25:39
 FW Version = mockbuild@ release 20150313
 Press "s" to enter Open Firmware.

Press F12 for boot menu.

Populating /vdevice methods
Populating /vdevice/vty@71000000
Populating /vdevice/nvram@71000001
Populating /vdevice/v-scsi@71000002
       SCSI: Looking for devices
          8200000000000000 CD-ROM   : "QEMU     QEMU CD-ROM      2.3."
Populating /pci@800000020000000
                     00 1800 (D) : 8086 24cd    serial bus [ usb-ehci ]
                     00 1000 (D) : 1af4 1000    virtio [ net ]
                     00 0800 (D) : 106b 003f    serial bus [ usb-ohci ]
                     00 0000 (D) : 1234 1111    qemu vga
No NVRAM common partition, re-initializing...
Installing QEMU fb



Scanning USB 
  EHCI: Initializing
    USB Storage 
       SCSI: Looking for devices
USB-DISK: CSW signature invalid 
USB-DISK: CSW signature invalid 
  OHCI: initializing
    USB Keyboard 
    USB mouse 
    USB HUB 
No console specified using screen & keyboard
     
  Welcome to Open Firmware

  Copyright (c) 2004, 2011 IBM Corporation All rights reserved.
  This program and the accompanying materials are made available
  under the terms of the BSD License available at
  http://www.opensource.org/licenses/bsd-license.php


Trying to load:  from: /pci@800000020000000/usb@3/usb-host@1 ... 
E3405: No such device
Trying to load:  from: /pci@800000020000000/ethernet@2 ... 
E3403: Bad executable:   

E3406: Client application returned an error.

        ..`. ..     .......  ..           ......      .......
    ..`...`''.`'. .''``````..''.       .`''```''`.  `''``````
       .`` .:' ': `''.....  .''.       ''`     .''..''.......
         ``.':.';. ``````''`.''.      .''.      ''``''`````'`
         ``.':':`   .....`''.`'`...... `'`.....`''.`'`       
        .`.`'``   .'`'`````.  ``''''''  ``''`'''`. `'`       
  Type 'boot'  and press return  to  continue  booting  the system.
  Type 'reset-all'  and  press  return  to   reboot   the   system.


        ..`. ..     .......  ..           ......      .......
    ..`...`''.`'. .''``````..''.       .`''```''`.  `''``````
       .`` .:' ': `''.....  .''.       ''`     .''..''.......
         ``.':.';. ``````''`.''.      .''.      ''``''`````'`
         ``.':':`   .....`''.`'`...... `'`.....`''.`'`       
        .`.`'``   .'`'`````.  ``''''''  ``''`'''`. `'`       
  Type 'boot'  and press return  to  continue  booting  the system.
  Type 'reset-all'  and  press  return  to   reboot   the   system.


Ready! 
0 >
Comment 4 Thomas Huth 2015-12-01 18:14:42 EST
I just had a quick try on our P8 server and it seems to work with the USB disk that we've got plugged in there:

# lsusb -v

Bus 002 Device 002: ID 0781:5583 SanDisk Corp. 
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               3.00
  bDeviceClass            0 (Defined at Interface level)
  bDeviceSubClass         0 
  bDeviceProtocol         0 
  bMaxPacketSize0         9
  idVendor           0x0781 SanDisk Corp.
  idProduct          0x5583 
  bcdDevice            1.00
  iManufacturer           1 SanDisk
  iProduct                2 Ultra Fit
  iSerial                 3 4C530001090822110352
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength           44
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0x80
      (Bus Powered)
    MaxPower              224mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         8 Mass Storage
      bInterfaceSubClass      6 SCSI
      bInterfaceProtocol     80 Bulk-Only
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0400  1x 1024 bytes
        bInterval               0
        bMaxBurst              15
Binary Object Store Descriptor:
  bLength                 5
  bDescriptorType        15
  wTotalLength           22
  bNumDeviceCaps          2
  USB 2.0 Extension Device Capability:
    bLength                 7
    bDescriptorType        16
    bDevCapabilityType      2
    bmAttributes   0x00000002
      Link Power Management (LPM) Supported
  SuperSpeed USB Device Capability:
    bLength                10
    bDescriptorType        16
    bDevCapabilityType      3
    bmAttributes         0x00
    wSpeedsSupported   0x000e
      Device can operate at Full Speed (12Mbps)
      Device can operate at High Speed (480Mbps)
      Device can operate at SuperSpeed (5Gbps)
    bFunctionalitySupport   1
      Lowest fully-functional device speed is Full Speed (12Mbps)
    bU1DevExitLat          10 micro seconds
    bU2DevExitLat         256 micro seconds
Device Status:     0x0000
  (Bus Powered)

# rpm -qa | grep SLOF
SLOF-20150313-5.gitc89b0df.el7.noarch
# rpm -qa | grep qemu-kvm-rhev
qemu-kvm-rhev-2.3.0-31.el7_2.3.ppc64le
# uname -r
3.10.0-327.el7.ppc64le

/usr/libexec/qemu-kvm  -m 16G -smp 8 -nographic -usbdevice tablet  -netdev user,id=net0 -device virtio-net-pci,netdev=net0,bootindex=3,id=net0,mac=24:be:05:11:92:11  -device usb-ehci,id=usbctr  -device usb-host,hostbus=2,hostaddr=2,id=hostdev0  -device virtio-scsi-pci -device scsi-cd,id=scsi-cd1,drive=scsi-cd1-dr,bootindex=4  -drive file=/var/lib/libvirt/images/isos/RHEL-6.7-20150304.0-Server-ppc64-dvd1.iso,if=none,id=scsi-cd1-dr,readonly=on,format=raw,cache=none

SLOF **********************************************************************
QEMU Starting
 Build Date = Sep 18 2015 06:25:39
 FW Version = mockbuild@ release 20150313
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/vty@71000000
Populating /vdevice/nvram@71000001
Populating /pci@800000020000000
                     00 2000 (D) : 1af4 1004    virtio [ scsi ]
Populating /pci@800000020000000/scsi@4
       SCSI: Looking for devices
          100000000000000 CD-ROM   : "QEMU     QEMU CD-ROM      2.3."
                     00 1800 (D) : 8086 24cd    serial bus [ usb-ehci ]
                     00 1000 (D) : 1af4 1000    virtio [ net ]
                     00 0800 (D) : 106b 003f    serial bus [ usb-ohci ]
                     00 0000 (D) : 1234 1111    qemu vga
No NVRAM common partition, re-initializing...
Installing QEMU fb



Scanning USB 
  EHCI: Initializing
ptr 0x7E4F3019 desc_type 48
ptr 0x7E4F3026 desc_type 48
    USB Storage 
       SCSI: Looking for devices
          101000000000000 DISK     : "SanDisk Ultra Fit 1.00"
  OHCI: initializing
    USB Keyboard 
    USB mouse 
    USB HUB 
No console specified using screen & keyboard


... the only strange thing here are these "ptr 0x..." messages, but I guess these are only left-over debug strings.
Comment 5 Thomas Huth 2015-12-01 18:18:00 EST
Could you please provide the output of "lsusb -v" for that USB disk? Also, does it behave differently when use a different kind of USB host controller, i.e. "-device pci-ohci" instead of "-device usb-ehci" ? Or "-device nec-usb-xhci" ?
Comment 6 Thomas Huth 2015-12-02 05:07:21 EST
I've got to correct myself: Even though I do not get that "CSW signature invalid" message as you did, it seems like I can also not load a file in SLOF from the passthrough USB disk.
However, it also does not work in Linux in the guest when I start Linux from another device in the guest, I only get the following output in the dmesg log:

[    1.828196] usb 1-1: new full-speed USB device number 2 using ohci-pci
[    2.588163] usb 1-1: device descriptor read/64, error 18
[    3.438161] usb 1-1: device descriptor read/64, error 18
[    4.288203] usb 1-1: new full-speed USB device number 3 using ohci-pci
[    5.038220] usb 1-1: device descriptor read/64, error 18
[    5.888160] usb 1-1: device descriptor read/64, error 18
[    6.738160] usb 1-1: new full-speed USB device number 4 using ohci-pci
[    6.769459] usb 1-1: Invalid ep0 maxpacket: 9
[    7.508159] usb 1-1: new full-speed USB device number 5 using ohci-pci
[    7.539503] usb 1-1: Invalid ep0 maxpacket: 9
[    7.548173] usb usb1-port1: unable to enumerate USB device

Could you please check whether the passthrough USB disk is working in Linux for you when you start the guest OS from another disk?
Comment 7 Thomas Huth 2015-12-02 05:22:27 EST
Ok, those "device descriptor read/64, error 18" error messages indicate that the emulated host controller does not match the host's USB controller. It works in Linux if I use "-device nec-usb-xhci" - since my host also features an XHCI controller.

Could you please check what kind of USB controller is used by your host machine (e.g. with "lspci | grep -i USB")? And then use the same controller type for the guest and check whether the device is working in Linux in the guest?

Anyway, when I specify "-device nec-usb-xhci" (and thus the disk is later working in Linux), the SLOF firmware suddenly does not detect the USB disk anymore. So at least here is definitely a bug ...
Comment 8 Thomas Huth 2015-12-02 10:51:44 EST
Ok, it works for me when I use the SLOF version from upstream (with an additional fix for the FAT filesystem that was required for the USB disk that is attached to our system, see https://lists.ozlabs.org/pipermail/slof/2015-December/000448.html).
There have been a couple of XHCI related updates in SLOF between the version that we use in RHEL 7.2 and the current upstream version, so one of these must have fixed the remaining issue that I was seeing. We should get this with a rebase of SLOF into downstream, too, so I'm adding this bug to the list of bugs that are blocked by BZ 1279954.
Comment 9 Thomas Huth 2015-12-02 10:53:58 EST
FWIW, here's the console output of my successful boot from an USB passthrough disk:

# /usr/libexec/qemu-kvm  -m 4G -smp 2 -vga none -nographic -device nec-usb-xhci,id=usbctr -device usb-host,hostbus=2,hostaddr=2,id=hostdev0 -bios ~thuth/boot_rom.bin 


SLOF **********************************************************************
QEMU Starting
 Build Date = Dec  2 2015 15:47:26
 FW Version = git-b4c93802a5b2c72f
 Press "s" to enter Open Firmware.

Populating /vdevice methods
Populating /vdevice/vty@71000000
Populating /vdevice/nvram@71000001
Populating /vdevice/l-lan@71000002
Populating /vdevice/v-scsi@71000003
       SCSI: Looking for devices
          8200000000000000 CD-ROM   : "QEMU     QEMU CD-ROM      2.3."
Populating /pci@800000020000000
                     00 0000 (D) : 1033 0194    serial bus [ usb-xhci ]
No NVRAM common partition, re-initializing...
Scanning USB 
  XHCI: Initializing
ptr 0x7E4E8019 desc_type 48
ptr 0x7E4E8026 desc_type 48
    USB Storage 
       SCSI: Looking for devices
          101000000000000 DISK     : "SanDisk Ultra Fit 1.00"
Using default console: /vdevice/vty@71000000
     
  Welcome to Open Firmware

  Copyright (c) 2004, 2011 IBM Corporation All rights reserved.
  This program and the accompanying materials are made available
  under the terms of the BSD License available at
  http://www.opensource.org/licenses/bsd-license.php


        ..`. ..     .......  ..           ......      .......
    ..`...`''.`'. .''``````..''.       .`''```''`.  `''``````
       .`` .:' ': `''.....  .''.       ''`     .''..''.......
         ``.':.';. ``````''`.''.      .''.      ''``''`````'`
         ``.':':`   .....`''.`'`...... `'`.....`''.`'`       
        .`.`'``   .'`'`````.  ``''''''  ``''`'''`. `'`       
  Type 'boot'  and press return  to  continue  booting  the system.
  Type 'reset-all'  and  press  return  to   reboot   the   system.


Ready! 
0 > devalias  
disk : /pci@800000020000000/usb@0/storage@1/disk@101000000000000
usb0 : /pci@800000020000000/usb@0
scsi : /vdevice/v-scsi@71000003
cdrom : /vdevice/v-scsi@71000003/disk@8200000000000000
net : /vdevice/l-lan@71000002
nvram : /vdevice/nvram@71000001
hvterm : /vdevice/vty@71000000 ok
0 > boot disk:1,ppc64.img  
Trying to load:  from: /pci@800000020000000/usb@0/storage@1/disk@101000000000000:1,ppc64.img ...   Successfully loaded

zImage starting: loaded at 0x00400000 (sp: 0x031e0fb0)
Allocating 0x1925074 bytes for kernel ...
OF version = 'SLOF,thuth'
Trying to claim from 0x400000 to 0x31ecdfc (0x2decdfc) got ffffffff
gunzipping (0x03200000 <- 0x0040c000:0x00b3f6cb)...done 0x1680000 bytes
Attached initrd image at 0x00b40000-0x031df1e0
Allocating 0x269f1e0 bytes for initrd ...
Relocating initrd 0x4b26000 <- 0x00b40000 (0x269f1e0 bytes)
initrd head: 0xfd377a58

Linux/PowerPC load: 
Finalizing device tree... using OF tree (promptr=7dca04c4)
OF stdout device is: /vdevice/vty@71000000
Preparing to boot Linux version 4.2.3-300.fc23.ppc64 (mockbuild@ppc-builder2.qa.fedoraproject.org) (gcc version 5.1.1 20150618 (Red Hat 5.1.1-4) (GCC) ) #1 SMP Fri Oct 9 15:02:21 UTC 2015
...
Comment 10 Xujun Ma 2015-12-02 21:24:30 EST
I'm sorry to respond late,i need time to ask for a usb stick by creating ticket.

the issue seems to be solved,what else do i need to do?
Comment 11 Thomas Huth 2015-12-03 02:28:50 EST
(In reply to Xujun Ma from comment #10)
> the issue seems to be solved,what else do i need to do?

Could you maybe confirm that the host you were using has an XHCI controller as USB interface, not an EHCI? Then I think we can assume that this issue is solved (i.e. the controller type in the guest should match the controller type in the host).
Comment 12 Xujun Ma 2015-12-04 00:18:25 EST
(In reply to Thomas Huth from comment #11)
> (In reply to Xujun Ma from comment #10)
> > the issue seems to be solved,what else do i need to do?
> 
> Could you maybe confirm that the host you were using has an XHCI controller
> as USB interface, not an EHCI? Then I think we can assume that this issue is
> solved (i.e. the controller type in the guest should match the controller
> type in the host).

yes,my host only has an XHCI controller:Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller,and it will display error message "ptr 0x7E4F3019 desc_type 48" if use usb-ehci in guest,and it can't be found as a data disk in a booted guest,the dmesg displays info like "device descriptor read/64, error 18".

it will be normal if i use nec-usb-xhci controler in guest,the usb stick can be found and there are none of any error message,but it still can't boot os from the passthough usb stick.
Comment 14 Mike McCune 2016-03-28 18:28:36 EDT
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
Comment 15 Miroslav Rezanina 2016-05-06 07:50:47 EDT
Fix included in SLOF-20160223-1.gitdbbfda4.el7.
Comment 17 Xujun Ma 2016-06-27 08:30:54 EDT
Reproduced the issue on old version:
Version-Release number of selected component (if applicable):
host:kernel-3.10.0-327.18.2.el7.ppc64le
qemu-kvm-rhev-2.3.0-31.el7.ppc64le
SLOF-20150313-5.gitc89b0df.el7.noarch
guest:kernel-3.10.0-327.el7.ppc64le

Steps to Reproduce:
1.boot up guest and install os with command
/usr/libexec/qemu-kvm \
 -m 16G -smp 16 -monitor stdio -qmp tcp::8888,server,nowait  -vnc :25\
 -usbdevice tablet \
 -netdev tap,id=tap0,script=/etc/qemu-ifup \
 -device virtio-net-pci,netdev=tap0,bootindex=3,id=net0,mac=24:be:05:11:92:11 \
 -device usb-ehci,id=usbctr \
 -device usb-host,hostbus=1,hostaddr=2,id=hostdev0 \
 -device virtio-scsi-pci\
 -device scsi-cd,id=scsi-cd1,drive=scsi-cd1-dr,bootindex=4 \
 -drive file=RHEL-7.2-20151030.0-Server-ppc64-dvd1.iso,if=none,id=scsi-cd1-dr,readonly=on,format=raw,cache=none \
2.boot up guest os from usb 

Actual results:
can't boot up guest os ,and it stop at slof


Verified the issue on the latest build:
Version-Release number of selected component (if applicable):
qemu-kvm-rhev-2.6.0-5.el7.ppc64le
SLOF-20160223-4.gitdbbfda4.el7.noarch
host:kernel-3.10.0-327.18.2.el7.ppc64le
guest:kernel-3.10.0-327.el7.ppc64le


1.boot up guest and install os with command
/usr/libexec/qemu-kvm \
 -m 16G -smp 16 -monitor stdio -qmp tcp::8888,server,nowait  -vnc :25\
 -usbdevice tablet \
 -netdev tap,id=tap0,script=/etc/qemu-ifup \
 -device virtio-net-pci,netdev=tap0,bootindex=3,id=net0,mac=24:be:05:11:92:11 \
 -device usb-ehci,id=usbctr \
 -device usb-host,hostbus=1,hostaddr=2,id=hostdev0 \
 -device virtio-scsi-pci\
 -device scsi-cd,id=scsi-cd1,drive=scsi-cd1-dr,bootindex=4 \
 -drive file=RHEL-7.2-20151030.0-Server-ppc64-dvd1.iso,if=none,id=scsi-cd1-dr,readonly=on,format=raw,cache=none \
2.boot up guest os from usb 

Actual results:can boot up of form usb.the bug was fixed 

but there is another issue and it's different from usb data disk(Bug 1273717 ).
the issue is:
host will lost usb device after boot and quit guest twice.
the usb stick won't appear again after reboot host.
there are some message in host log:
[    0.882164] usb 1-2: new high-speed USB device number 2 using xhci_hcd
[   16.002226] usb 1-2: device descriptor read/64, error -110
[   31.232201] usb 1-2: device descriptor read/64, error -110
[   31.462161] usb 1-2: new high-speed USB device number 3 using xhci_hcd
[   46.582196] usb 1-2: device descriptor read/64, error -110
[   61.812196] usb 1-2: device descriptor read/64, error -110
[   62.042163] usb 1-2: new high-speed USB device number 4 using xhci_hcd
[   67.062291] usb 1-2: device descriptor read/8, error -110
[   72.192259] usb 1-2: device descriptor read/8, error -110
[   72.422160] usb 1-2: new high-speed USB device number 5 using xhci_hcd
[   77.442291] usb 1-2: device descriptor read/8, error -110
[   82.572292] usb 1-2: device descriptor read/8, error -110
[   82.682167] usb usb1-port2: unable to enumerate USB device
Comment 19 errata-xmlrpc 2016-11-04 00:31:47 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2355.html

Note You need to log in before you can comment on or make changes to this bug.