Bug 1354177
| Summary: | Booting from a passthrough usb stick fails when using the bootindex property | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Min Deng <mdeng> | ||||||||||
| Component: | qemu-kvm-rhev | Assignee: | Thomas Huth <thuth> | ||||||||||
| Status: | CLOSED ERRATA | QA Contact: | xianwang <xianwang> | ||||||||||
| Severity: | medium | Docs Contact: | |||||||||||
| Priority: | medium | ||||||||||||
| Version: | 7.4 | CC: | dgibson, knoel, lvivier, mdeng, michal.skrivanek, mrezanin, qzhang, thuth, virt-maint, xianwang | ||||||||||
| Target Milestone: | rc | ||||||||||||
| Target Release: | --- | ||||||||||||
| Hardware: | ppc64le | ||||||||||||
| OS: | Unspecified | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | qemu-kvm-rhev-2.9.0-1.el7 | Doc Type: | If docs needed, set a value | ||||||||||
| Doc Text: | Story Points: | --- | |||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2017-08-01 23:32:13 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Bug Depends On: | 1352765, 1366471 | ||||||||||||
| Bug Blocks: | |||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Min Deng
2016-07-10 10:48:53 UTC
Created attachment 1178049 [details]
cannotfound
SLOF-20160223-4.gitdbbfda4.el7.noarch Update - you can ignore step 3 and reboot it directly as the passthrough usb disk's bootindex=0. Can you please show the lsusb -v from the host, so we can see the details of the device which was passed through? Is it a plain USB storage device, or a USB hub with a storage device plugged into it? I suspect this may be due to SLOF not handling USB hubs properly. (In reply to David Gibson from comment #5) > Can you please show the lsusb -v from the host, so we can see the details > of the device which was passed through? Is it a plain USB storage device, > or a USB hub with a storage device plugged into it? > > I suspect this may be due to SLOF not handling USB hubs properly. I will upload the info you asked to the bug.For the USB storage device,I'm not very sure type of this usb on host side because it is remotely and only eng-ops guys can operate. Created attachment 1178668 [details]
lsusb-v
Created attachment 1178669 [details]
lsusb-t
Um.. according to that lsusb output, bus 1, device 7 is a HID device (keyboard or mouse) not a storage device. Was that on the same host as you used the commandline from the initial post? There seem to be two issues here. First issue is that SLOF currently can not handle devices that are attached to an XHCI USB hub - see BZ 1352765 for details (so I added that BZ as a dependency for this bug here). The passthrough storage device is placed on a hub here since the nec-usb-xhci controller only has four root hub ports by default, and you've specified three USB devices already (usb-mouse, usb-kbd and usb-tablet) before specifying the storage device. QEMU then adds a hub automatically inbetween so that it is possible to add more USB devices later. You can work around the problem with the USB hub by either specifying less other USB devices (e.g. omit the usb-mouse or usb-tablet), or by increasing the amount of root hub ports on the nec-usb-xhci device, e.g. by specifying the p3 property like this: ... -device nec-usb-xhci,id=controller0,p3=16 ... SLOF should then detect the USB storage device while scanning the USB bus. However, there is then a second problem left, SLOF does not boot automatically from the passthrough USB device yet. Seems like the "boot-device" firmware variable is pointing to the wrong location. Booting manually with "boot disk" seems to work fine, though: SLOF ********************************************************************** QEMU Starting Build Date = Jun 7 2016 03:06:01 FW Version = mockbuild@ release 20160223 Press "s" to enter Open Firmware. Populating /vdevice methods Populating /vdevice/v-scsi@1000 SCSI: Looking for devices Populating /vdevice/vty@71000000 Populating /vdevice/nvram@71000001 Populating /pci@800000020000000 00 0800 (D) : 1033 0194 serial bus [ usb-xhci ] 00 0000 (D) : 1af4 1000 virtio [ net ] No NVRAM common partition, re-initializing... Scanning USB XHCI: Initializing USB mouse USB Keyboard USB Storage SCSI: Looking for devices 104000000000000 DISK : "Lexar USB Flash Drive 1100" Using default console: /vdevice/vty@71000000 Welcome to Open Firmware Copyright (c) 2004, 2011 IBM Corporation All rights reserved. This program and the accompanying materials are made available under the terms of the BSD License available at http://www.opensource.org/licenses/bsd-license.php Trying to load: from: /pci@800000020000000/usb@1/usb-host@4 ... E3405: No such device E3407: Load failed ..`. .. ....... .. ...... ....... ..`...`''.`'. .''``````..''. .`''```''`. `''`````` .`` .:' ': `''..... .''. ''` .''..''....... ``.':.';. ``````''`.''. .''. ''``''`````'` ``.':':` .....`''.`'`...... `'`.....`''.`'` .`.`'`` .'`'`````. ``'''''' ``''`'''`. `'` Type 'boot' and press return to continue booting the system. Type 'reset-all' and press return to reboot the system. Ready! 0 > printenv boot-device Current: /pci@800000020000000/usb@1/usb-host@4 Default: ok 0 > devalias disk : /pci@800000020000000/usb@1/storage@4/disk@104000000000000 keyboard : /pci@800000020000000/usb@1/usb-keyboard@2 net : /pci@800000020000000/ethernet@0 usb0 : /pci@800000020000000/usb@1 nvram : /vdevice/nvram@71000001 hvterm : /vdevice/vty@71000000 scsi : /vdevice/v-scsi@1000 ok 0 > boot disk Trying to load: from: /pci@800000020000000/usb@1/storage@4/disk@104000000000000 ... Successfully loaded System has 0 Mbytes in RMA Config file read, 1024 bytes Welcome to Red Hat Enterprise Linux! Hit <TAB> for boot options Welcome to yaboot version 1.3.14 (Red Hat 1.3.14-43.el6) ... (In reply to David Gibson from comment #9) > Um.. according to that lsusb output, bus 1, device 7 is a HID device > (keyboard or mouse) not a storage device. Was that on the same host as you > used the commandline from the initial post? yes,I used the same host Patch has been merged upstream: http://git.qemu.org/?p=qemu.git;a=commitdiff;h=b99260ebbb5844da9e77fbcaa73b7b6980a68acf I have retest this scenario, but it does not pass.
Host infomation:
3.10.0-660.el7.ppc64le
qemu-kvm-rhev-2.9.0-2.el7.ppc64le
SLOF-20170303-2.git66d250e.el7.noarch
Guest:
RHEL-7.3-20161019.0-Server-ppc64le-dvd1.iso
Usb disk:
Bus 001 Device 002: ID 05dc:a81d Lexar Media, Inc.
bcdUSB 2.00
Port 1: Dev 2, If 0, Class=Mass Storage, Driver=usb-storage, 480M
step:
1.install RHEL7.3 os to usb disk with the following qemu cli:
/usr/libexec/qemu-kvm \
-name 'vm1' \
-sandbox off \
-machine pseries \
-nodefaults \
-device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04 \
-chardev socket,id=console0,path=/tmp/console0,server,nowait \
-device spapr-vty,chardev=console0 \
-device nec-usb-xhci,id=usb1,bus=pci.0,addr=06 \
-device virtio-scsi-pci,id=controller_scsi,bus=pci.0,addr=07 \
-drive id=drive_image1,if=none,format=raw,file=/home/xianwang/RHEL-7.3-20161019.0-Server-ppc64le-dvd1.iso \
-device scsi-cd,bus=controller_scsi.0,id=image1,drive=drive_image1,bootindex=1 \
-device usb-host,hostbus=001,hostaddr=002,id=hostdev,bootindex=0,bus=usb1.0 \
-m 4096 \
-smp 4,maxcpus=4 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=2 \
-device usb-kbd,id=usb-kbd,bus=usb1.0,port=3 \
-vnc :1 \
-qmp tcp:0:8881,server,nowait \
-vga std \
-monitor stdio \
-rtc base=utc,clock=host \
-boot once=c,menu=on \
-enable-kvm \
result:
can't find usb disk, so the os can't be installed to it, as attachment usb_disk.
there is already an rhel6 OS in this usb disk, but it is boot failed with the following message and it will reboot automatically.
[xianwang@ibm-p8-virt-02 ~]$ sudo nc -U /tmp/console0
00 2000 (D) : 1af4 1003 virtio [ serial ]
00 3000 (D) : 1033 0194 serial bus [ usb-xhci ]
00 3800 (D) : 1af4 1004 virtio [ scsi ]
Populating /pci@800000020000000/scsi@7
SCSI: Looking for devices
No NVRAM common partition, re-initializing...
Installing QEMU fb
Scanning USB
XHCI: Initializing
USB Storage
SCSI: Looking for devices
101000000000000 DISK : "Lexar USB Flash Drive 1100"
USB Keyboard
No console specified using screen & keyboard
Welcome to Open Firmware
Copyright (c) 2004, 2017 IBM Corporation All rights reserved.
This program and the accompanying materials are made available
under the terms of the BSD License available at
http://www.opensource.org/licenses/bsd-license.php
Trying to load: from: /pci@800000020000000/usb@6/storage@1/disk ... Successfully loaded
CF000012
CF000015ch
CF000020ne
CF000021t
Linux ppc64
#1 SMP Wed Apr 1pci 0000:00:06.0: can't enable device: BAR 0 [mem size 0x00004000] not assigned
rtas_flash: no firmware flash support
virtio-pci 0000:00:04.0: can't enable device: BAR 4 [mem size 0x00004000 pref] not assigned
virtio-pci 0000:00:07.0: can't enable device: BAR 4 [mem size 0x00004000 pref] not assigned
xhci_hcd 0000:00:06.0: can't enable device: BAR 0 [mem size 0x00004000] not assigned
Kernel panic - not syncing: Attempted to kill init!
Call Trace:
[c0000000fd8efb10] [c000000000013024] .show_stack+0x74/0x1c0 (unreliable)
[c0000000fd8efbc0] [c0000000005fa9c8] .panic+0xc4/0x208
[c0000000fd8efc50] [c00000000009c6a0] .do_exit+0x880/0x890
[c0000000fd8efd30] [c00000000009c700] .do_group_exit+0x50/0xf0
[c0000000fd8efdc0] [c00000000009c7b4] .SyS_exit_group+0x14/0x30
[c0000000fd8efe30] [c000000000008564] syscall_exit+0x0/0x40
Rebooting in 180 seconds..
Created attachment 1276983 [details]
usb_disk
I think we're facing a new problem here. Actually, on the virt-01 system, the Lexar stick is now back. I can successfully boot the guest from the stick, but after a while, I see some "usb 1-1: reset high-speed USB device number 2 using xhci_hcd" popping up in the host dmesg log, and the guest freezes. I have to reboot the host to get the USB stick working again. So could you maybe test again on the virt-01 system (the Lexar USB stick should be there again, with your RHEL-7.3 installation) to see whether you now can boot from USB stick there, too. If it hangs, please try to reboot the host and try again. If the boot works for you, I suggest that we mark this bug here as verified and open a new BZ for the new hangs that we're now facing. Thanks! (In reply to xianwang from comment #18) > xhci_hcd 0000:00:06.0: can't enable device: BAR 0 [mem size 0x00004000] not > assigned It works with upstream SLOF. This should be fixed by one of Thomas' upstream patches. It fails later, but it seems the image has been corrupted and fsck can't fix it because it is read-only. If the PCI BAR problem goes away with upstream SLOF, then please re-test with downstream version SLOF-20170303-3.git66d250e.el7 ... that should contain all current PCI fixes from upstream. NB: The PCI BAR problem is a separate issue, we should open a separate BZ for it instead of discussing it here if the problem persists. (In reply to Thomas Huth from comment #23) > If the PCI BAR problem goes away with upstream SLOF, then please re-test > with downstream version SLOF-20170303-3.git66d250e.el7 ... that should > contain all current PCI fixes from upstream. SLOF-20170303-3.git66d250e.el7 fixes the problem for me. Please, xianwang re-test with this SLOF version (I didn't test the bootindex part). The strange thing is the USB device is now write protected on the host: [ 221.544451] scsi 4:0:0:0: Direct-Access Lexar USB Flash Drive 1100 PQ: 0 ANSI: 4 [ 221.544668] scsi 4:0:0:0: alua: supports implicit and explicit TPGS [ 221.545103] scsi 4:0:0:0: alua: No target port descriptors found [ 221.545161] scsi 4:0:0:0: alua: not attached [ 221.545372] sd 4:0:0:0: Attached scsi generic sg22 type 0 [ 221.545421] sd 4:0:0:0: Embedded Enclosure Device [ 221.553092] sd 4:0:0:0: Wrong diagnostic page; asked for 1 got 0 [ 221.553171] sd 4:0:0:0: Failed to get diagnostic page 0xffffffea [ 221.553179] sd 4:0:0:0: Failed to bind enclosure -19 [ 221.553585] sd 4:0:0:0: [sdh] 62517248 512-byte logical blocks: (32.0 GB/29.8 GiB) [ 221.554167] sd 4:0:0:0: [sdh] Write Protect is on [ 221.554217] sd 4:0:0:0: [sdh] Mode Sense: 43 00 80 00 [ 221.554720] sd 4:0:0:0: [sdh] No Caching mode page found [ 221.554769] sd 4:0:0:0: [sdh] Assuming drive cache: write through [ 221.561494] sdh: sdh1 sdh2 sdh3 [ 221.564167] sd 4:0:0:0: [sdh] Attached SCSI removable disk If I "dd" the USB storage and boot from the image (read-write), it boots fine. (In reply to Laurent Vivier from comment #25) [...] > The strange thing is the USB device is now write protected on the host: Please try to reboot the host before doing the test to see whether it makes a difference ... I think we're facing some new USB problems here which might only trigger after a while ... This bug is verified for qemu-kvm-rhev-2.9.0-1.el7.
Bug reproduction:
Host:
3.10.0-663.el7.ppc64le
qemu-kvm-rhev-2.6.0-22.el7.ppc64le
SLOF-20170303-3.git66d250e.el7.noarch
1.Boot a guest with installing an OS to a passthrough usb disk
/usr/libexec/qemu-kvm \
-name 'vm1' \
-sandbox off \
-machine pseries \
-nodefaults \
-device virtio-serial-pci,id=virtio_serial_pci0,bus=pci.0,addr=04 \
-chardev socket,id=console0,path=/tmp/console0,server,nowait \
-device spapr-vty,chardev=console0 \
-device nec-usb-xhci,id=usb1,bus=pci.0,addr=06 \
-device virtio-scsi-pci,id=controller_scsi,bus=pci.0,addr=07 \
-drive id=drive_image1,if=none,format=raw,file=/home/xianwang/RHEL-7.3-20161019.0-Server-ppc64le-dvd1.iso \
-device scsi-cd,bus=controller_scsi.0,id=image1,drive=drive_image1,bootindex=1 \
-device usb-host,hostbus=001,hostaddr=002,id=hostdev,bootindex=0,bus=usb1.0 \
-m 4096 \
-smp 4,maxcpus=4 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=2 \
-device usb-kbd,id=usb-kbd,bus=usb1.0,port=3 \
-vnc :1 \
-qmp tcp:0:8881,server,nowait \
-vga std \
-monitor stdio \
-rtc base=utc,clock=host \
-boot order=cdn,menu=on,strict=on \
-enable-kvm \
2.OS is installed successfully.
3.Quit qemu process.
4.Boot up this guest again from usb disk with the same qemu cli as step 1.
result:
can not find this usb device and boot failed, the console output information is as following:
Trying to load: from: /pci@800000020000000/usb@6/usb-host@1 ...
E3405: No such device
E3407: Load failed
Type 'boot' and press return to continue booting the system.
Type 'reset-all' and press return to reboot the system.
Ready!
0 >
Bug vefiry:
Host:
3.10.0-663.el7.ppc64le
qemu-kvm-rhev-2.9.0-1.el7.ppc64le
SLOF-20170303-3.git66d250e.el7.noarch
the steps are same as bug reproduction
result:
boot guest succeed, the console info is as following:
Trying to load: from: /pci@800000020000000/usb@6/storage@1/disk ... Successfully loaded
CF000012
CF000015ch
Linux ppc64le
#1 SMP Wed Oct 1
So, this bug is verified.
(In reply to Thomas Huth from comment #26) > (In reply to Laurent Vivier from comment #25) > [...] > > The strange thing is the USB device is now write protected on the host: > > Please try to reboot the host before doing the test to see whether it makes > a difference ... I think we're facing some new USB problems here which might > only trigger after a while ... After a full power cycle it's always write protected. I've tried with "hdparm -r0" to turn off write protection but it doesn't work. I think the storage has been damaged: [ 438.634670] sd 5:0:0:0: Wrong diagnostic page; asked for 1 got 0 [ 438.634785] sd 5:0:0:0: Failed to get diagnostic page 0xffffffea [ 438.634795] sd 5:0:0:0: [sdg] 62517248 512-byte logical blocks: (32.0 GB/29.8 GiB) [ 438.635032] sd 5:0:0:0: Failed to bind enclosure -19 We have the same message from the petitboot shell. (In reply to Thomas Huth from comment #21) > I think we're facing a new problem here. Actually, on the virt-01 system, > the Lexar stick is now back. I can successfully boot the guest from the > stick, but after a while, I see some "usb 1-1: reset high-speed USB device > number 2 using xhci_hcd" popping up in the host dmesg log, and the guest > freezes. I have to reboot the host to get the USB stick working again. > > So could you maybe test again on the virt-01 system (the Lexar USB stick > should be there again, with your RHEL-7.3 installation) to see whether you > now can boot from USB stick there, too. If it hangs, please try to reboot > the host and try again. If the boot works for you, I suggest that we mark > this bug here as verified and open a new BZ for the new hangs that we're now > facing. Thanks! Yes, now, there is some "usb 1-1: reset high-speed USB device number 2 using xhci_hcd" in host dmesg info, but I personally think the result is as following, the usb is 2.0 version that with low speed and the controller assigned to it is usb-xhci both guest and host which is for high speed usb device, i.e, the reason for this is that usb controller and device are not match, but I am not sure, so, I am not sure whether this is a bug. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 |