Bug 1381039

Summary: USB devices aren't found or enumerated, results in boot failure from USB stick installation media, using USB 3.1 Gen 2 (Type-C, HP USB Boost, Thunderbolt)
Product: [Fedora] Fedora Reporter: Chris Murphy <bugzilla>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: gansalmon, ichavero, itamar, jonathan, kernel-maint, knutjbj, kparal, madhu.chinakonda, mchehab, pschindl, robatino
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: RejectedBlocker
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-24 16:18:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
blkid lsmod (cell photo)
none
lsmod, rare case
none
dmesg, rare case
none
lspci, rare case
none
lspci after unplug and plug, rare case
none
journal acpi=off
none
rdsosreport.txt none

Description Chris Murphy 2016-10-02 16:51:00 UTC
Description of problem:

When booting from Fedora installation media, GRUB, kernel and initramfs are found and loaded, but startup fails and ends at a dracut shell. The cause is the kernel isn't seeing the device.


Version-Release number of selected component (if applicable):
4.8.0-0.rc7.git0.1.fc25.x86_64


How reproducible:
Always.
Rarely, unplugging the stick then plugging it back in, while at the dracut shell, results in enumeration.


Steps to Reproduce:
1. Create USB stick media using dd, Fedora-Workstation-Live-x86_64-25-20160930.n.0.iso
2. Boot
3.

Actual results:

GRUB, kernel, initramfs load; startup fails when initramfs is looking for rootfs

Expected results:

Should boot.

Additional info:

Hardware: HP Spectre 13-v021nr (2016) with 8/11/2016 firmware

Is probably not a regression, happens with Fedora-Workstation-Live-x86_64-24-1.2.iso (kernel 4.5.5) and ubuntu-16.04.1-desktop-amd64.iso (kernel 4.4.0).

Comment 1 Chris Murphy 2016-10-02 16:55:51 UTC
Created attachment 1206641 [details]
blkid lsmod (cell photo)

Shows blkid and lsmod output. blkid only sees the nvme device, not the USB stick that got us to this point. lsmod shows missing uas and usb_storage modules.

Unplug, plug, does not result in play (USB stick still missing). This is the most common outcome.

Comment 2 Chris Murphy 2016-10-02 16:58:54 UTC
Created attachment 1206642 [details]
lsmod, rare case

This boot still fails, but shows uas and usb_storage modules. The USB stick does not enumerate during startup, I still end up at dracut shell. If I unplug the stick then plug it back in, enumeration happens.

Comment 3 Chris Murphy 2016-10-02 17:10:40 UTC
Created attachment 1206644 [details]
dmesg, rare case

This dmesg goes with "lsmod, rare case" in comment 2. They are the same boot.

4.710740 there's a bunch of usb related items suggesting it understands it's got a USB bus, but still doesn't see the stick.

5.282785 it sees the stick seems to be using the correct xhci_hcd driver but it still isn't enumerated. 

348.737213 is when I disconnect the USB stick, and 362.716545 is when I plug it back in, and enumeration happens. There's no change in lsmod.

Comment 4 Chris Murphy 2016-10-02 17:16:20 UTC
Created attachment 1206647 [details]
lspci, rare case

Goes with boot described in comment 2. This lspci -vvnn is taken when enumeration hasn't happened yet; before unplug plug and play of the stick.

Comment 5 Chris Murphy 2016-10-02 17:17:45 UTC
Created attachment 1206650 [details]
lspci after unplug and plug, rare case

Goes with boot described in comment 2. This lspci -vvnn is taken after unplugging the stick and plugging it back in. diff shows various state differences between the two lspci's.

Comment 6 Chris Murphy 2016-10-02 18:52:59 UTC
Created attachment 1206652 [details]
journal acpi=off

Still fails to complete startup, but different details. Enumeration happens but takes 20+ minutes to get to anaconda, and then no graphical boot, there is a call trace that might explain that, along with the suggestion to use pci=biosirq.

pci=biosirq by itself results in original failure
acpi=off pci=biosirq together has same results as acpi=off without pci=biosirq

Everything after 1900s can be ignored; I yanked the USB stick to attach one that's read writable in order to extract the journal.

Comment 7 Chris Murphy 2016-10-02 19:03:52 UTC
https://fedoraproject.org/wiki/Common_kernel_problems

These have no effect, original failure is the result:
usbcore.autosuspend=-1
floppy.allowed_drive_mask=0
pci=nomsi,nommconf
pcie_aspm=off
rdblacklist=ahci
libata.dma=0

Comment 8 Chris Murphy 2016-10-02 19:13:27 UTC
Without quiet but with pci=noacpi, nothing happens after GRUB. No penguins, no kernel boot text. Just an underscore in the upper left corner and a black screen.

Comment 9 Chris Murphy 2016-10-02 19:15:20 UTC
Nominating as final blocker:
"In rawhide bugs, if the report is something that would prevent someone from installing the next release (crashes during boot, doesn't find hard disks etc), mark the bug as blocking 'F9Blocker' (bug 235706)."
https://fedoraproject.org/wiki/Common_kernel_problems

Comment 10 Petr Schindler 2016-10-03 17:23:16 UTC
Discussed at 2016-10-03 blocker review meeting: [1]. 

This bug was rejected as Final blocker: so far we only know of a single affected system, and it's not a regression; this isn't a broad enough impact for a hardware-specific bug to be counted as a violation of the criteria. this decision can be revisited if more systems are found to be affected

[1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2016-10-03/

Comment 11 Kamil Páral 2016-10-04 11:18:21 UTC
After seeing bug 1380149, I wonder, Chris, is there a USB2 port on that laptop that you could try instead of USB3? (or maybe try with a USB2 hub, not sure if that makes a difference, though)

Comment 12 Chris Murphy 2016-10-04 18:29:00 UTC
Bug 1380149 has errors I'm not seeing. This HP Spectre has three USB-C ports only. Connected to that port is an HP supplied USB-C to USB-3 adapter, into which I've connected three different USB 2 sticks. I also have a USB 3 hub I've plugged in which have USB 3 drives. The same problem happens in all combinations, they simply fail to enumerate.

Since the problem doesn't happen on Windows 10, I suspect it's a kernel limitation, either HP is doing something different than other systems, or Intel hasn't gotten support for this hardware in the upstream kernel. If linux-next builds don't work within 3 weeks I'm sending this laptop back to HP, I don't want a brick.

Comment 13 Chris Murphy 2016-10-05 17:09:45 UTC
HP specs say: 1 USB 3.1 Gen 1 (Type-C™, HP USB Boost); 2 USB 3.1 Gen 2 (Type-C™, HP USB Boost, Thunderbolt).

The single Gen 1 port is also the power port. This means I've been using the Gen 2 ports, which are also Thunderbolt. I *can* boot from the Gen 1 port. This problem only happens on either of the other two ports. So this could be either a USB Gen 2 related bug, or maybe Thunderbolt/PCIe bug, seeing as this Gen 1 port is not Thunderbolt.

Anyway, I have a graphical installer. I just can't charge while doing the installation.

Comment 14 Chris Murphy 2016-10-24 21:54:22 UTC
Created attachment 1213605 [details]
rdsosreport.txt

Fedora-Workstation-netinst-x86_64-25-20161023.n.0.iso
systemd.log_level=debug rd.debug

Comment 15 Chris Murphy 2017-01-02 23:14:14 UTC
Affects kernels 4.8.15-300.fc25.x86_64 through mainline 4.10-rc1. Upstream threads:
http://www.spinics.net/lists/linux-usb/msg151275.html
http://www.spinics.net/lists/linux-pci/msg57110.html

Comment 16 Chris Murphy 2018-04-24 16:18:42 UTC
Fixed upstream in 4.16.x.