Bug 1575870

Summary: After resume Bluetooth stopping after few seconds and high kworker proces / interrupt gpe6D
Product: [Fedora] Fedora Reporter: Bart Ratgers <bart>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED WORKSFORME QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 28CC: airlied, bskeggs, ewk, hdegoede, ichavero, itamar, jarodwilson, jglisse, john.j5live, jonathan, josef, kernel-maint, linville, mchehab, mjg59, steved
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-28 10:36:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dmidecode file none

Description Bart Ratgers 2018-05-08 06:54:18 UTC
Created attachment 1433022 [details]
dmidecode file

Description of problem:

After updating to Fedora 28 and new kernel version 4.16.5-300 & 4.16.6-302, my Bluetooth stopping after a few seconds. It seems that is related to an interrupt storm on interrupt gpe6D. The strange thing is that before suspending/resuming interrupt gpe6D is disabled.

Within 1 minute more than 400k interrupts is handled bij the kernel and BLuetooth is slowly dying. 

/sys/firmware/acpi/interrupts/gpe6D:  497754     STS enabled      unmasked

When I'm switching back to an old kernel 4.16.4-200, everything works as expected. 

Version-Release number of selected component (if applicable):


How reproducible:
On my HP Spectre x360 Convertible 13-ae0xx update to fedora 28


Steps to Reproduce:
1. Attach a Bluetooth mouse 
2. Suspend the machine 
3. Resume the machine 

Actual results:

- Bluetooth dying after a few seconds, the first ten seconds, the mouse works
- Very high load on a CPU due busy process of kworker/0:0
- Interrupt storm on interrupt gpe6D
  /sys/firmware/acpi/interrupts/gpe6D:  497754     STS enabled      unmasked

Expected results:

- Working Bluetooth mouse
- Normal CPU load
- Interrupt gpe6D disabled
  /sys/firmware/acpi/interrupts/gpe6D:       0         disabled     unmasked


Additional info:

Comment 1 Hans de Goede 2018-05-08 07:20:58 UTC
Thank you for the bug report, please try booting with "btusb.enable_autosuspend=0" added to the kernel commandline and let us know if that helps.

After adding this to the kernel commandline, please check that it is disabled by doing:

cat /sys/module/btusb/parameters/enable_autosuspend

This should result in "N" (for no) being printed.

Comment 2 Bart Ratgers 2018-05-08 20:46:03 UTC
Thanks, that works. After disabling secure boot I can change the kernel parameter. With the kernel parameter i can suspend and resume the laptop with a working Bluetooth mouse and without and interrupt storm. 
Even interrupt gpe6D is disabled. 

Is this a work around or a definitive solution?

Comment 3 Hans de Goede 2018-05-09 08:22:19 UTC
(In reply to Bart Ratgers from comment #2)
> Thanks, that works. After disabling secure boot I can change the kernel
> parameter. With the kernel parameter i can suspend and resume the laptop
> with a working Bluetooth mouse and without and interrupt storm. 
> Even interrupt gpe6D is disabled. 
> 
> Is this a work around or a definitive solution?

Could be either, also depends a bit on which type of bluetooth controller your HP spectre X360 has, can you do "lsusb" and "dmesg | grep -i bluetooth" and post
the output of both command here please ?

If your bluetooth controller is a qca rome bluetooth controller then this is somewhat expected, it seems some vendors hook these up in such a way that they don't survive a suspend/resume without some extra handling. In that case the permananent solution is to add a quirk to the kernel to automatically do "btusb.enable_autosuspend=0" for your model laptop.

But the whole gpe6D story is weird...

Can you file a new kernel bug for this here:
https://bugzilla.kernel.org/enter_bug.cgi?product=ACPI

With all the info we've so far and close the old kernel bug as a duplicate of the new one (you cannot change the product after filing the bug) ?

Comment 4 Bart Ratgers 2018-05-09 15:33:10 UTC
Output of


lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp. 
Bus 001 Device 002: ID 05c8:0815 Cheng Uei Precision Industry Co., Ltd (Foxlink) 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

lspci
00:00.0 Host bridge: Intel Corporation Xeon E3-1200 v6/7th Gen Core Processor Host Bridge/DRAM Registers (rev 08)
00:02.0 VGA compatible controller: Intel Corporation UHD Graphics 620 (rev 07)
00:04.0 Signal processing controller: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem (rev 08)
00:13.0 Non-VGA unclassified device: Intel Corporation Sunrise Point-LP Integrated Sensor Hub (rev 21)
00:14.0 USB controller: Intel Corporation Sunrise Point-LP USB 3.0 xHCI Controller (rev 21)
00:14.2 Signal processing controller: Intel Corporation Sunrise Point-LP Thermal subsystem (rev 21)
00:15.0 Signal processing controller: Intel Corporation Sunrise Point-LP Serial IO I2C Controller #0 (rev 21)
00:16.0 Communication controller: Intel Corporation Sunrise Point-LP CSME HECI #1 (rev 21)
00:1c.0 PCI bridge: Intel Corporation Sunrise Point-LP PCI Express Root Port #1 (rev f1)
00:1c.1 PCI bridge: Intel Corporation Device 9d11 (rev f1)
00:1c.4 PCI bridge: Intel Corporation Sunrise Point-LP PCI Express Root Port #5 (rev f1)
00:1d.0 PCI bridge: Intel Corporation Sunrise Point-LP PCI Express Root Port #9 (rev f1)
00:1e.0 Signal processing controller: Intel Corporation Sunrise Point-LP Serial IO UART Controller #0 (rev 21)
00:1e.2 Signal processing controller: Intel Corporation Sunrise Point-LP Serial IO SPI Controller #0 (rev 21)
00:1f.0 ISA bridge: Intel Corporation Intel(R) 100 Series Chipset Family LPC Controller/eSPI Controller - 9D4E (rev 21)
00:1f.2 Memory controller: Intel Corporation Sunrise Point-LP PMC (rev 21)
00:1f.3 Audio device: Intel Corporation Sunrise Point-LP HD Audio (rev 21)
00:1f.4 SMBus: Intel Corporation Sunrise Point-LP SMBus (rev 21)
01:00.0 Unassigned class [ff00]: Realtek Semiconductor Co., Ltd. RTS525A PCI Express Card Reader (rev 01)
02:00.0 Network controller: Intel Corporation Wireless 8265 / 8275 (rev 78)
6e:00.0 Non-Volatile memory controller: Toshiba America Info Systems XG4 NVMe SSD Controller (rev 01)


dmesg | grep -i bluetooth
[   12.939785] Bluetooth: Core ver 2.22
[   12.939816] Bluetooth: HCI device and connection manager initialized
[   12.939819] Bluetooth: HCI socket layer initialized
[   12.939821] Bluetooth: L2CAP socket layer initialized
[   12.939830] Bluetooth: SCO socket layer initialized
[   12.993688] Bluetooth: hci0: Bootloader revision 0.0 build 26 week 38 2015
[   12.994693] Bluetooth: hci0: Device revision is 16
[   12.994695] Bluetooth: hci0: Secure boot is enabled
[   12.994696] Bluetooth: hci0: OTP lock is enabled
[   12.994697] Bluetooth: hci0: API lock is enabled
[   12.994698] Bluetooth: hci0: Debug lock is disabled
[   12.994699] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[   12.998512] Bluetooth: hci0: Found device firmware: intel/ibt-12-16.sfi
[   13.580798] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   13.580800] Bluetooth: BNEP filters: protocol multicast
[   13.580804] Bluetooth: BNEP socket layer initialized
[   14.430037] Bluetooth: hci0: Waiting for firmware download to complete
[   14.430673] Bluetooth: hci0: Firmware loaded in 1403623 usecs
[   14.430740] Bluetooth: hci0: Waiting for device to boot
[   14.442711] Bluetooth: hci0: Device booted in 11734 usecs
[   14.449784] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-12-16.ddc
[   14.452679] Bluetooth: hci0: Applying Intel DDC parameters completed
[   32.117887] Bluetooth: RFCOMM TTY layer initialized
[   32.117896] Bluetooth: RFCOMM socket layer initialized
[   32.117977] Bluetooth: RFCOMM ver 1.11
[   40.460845] hid-generic 0005:046D:B017.0006: input,hidraw1: BLUETOOTH HID v0.17 Keyboard [MX Master] on 90:61:AE:A1:45:D6
[   91.439511] Bluetooth: hci0: Bootloader revision 0.0 build 26 week 38 2015
[   91.440531] Bluetooth: hci0: Device revision is 16
[   91.440533] Bluetooth: hci0: Secure boot is enabled
[   91.440533] Bluetooth: hci0: OTP lock is enabled
[   91.440534] Bluetooth: hci0: API lock is enabled
[   91.440535] Bluetooth: hci0: Debug lock is disabled
[   91.440536] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[   91.440539] Bluetooth: hci0: Found device firmware: intel/ibt-12-16.sfi
[   92.954988] Bluetooth: hci0: Waiting for firmware download to complete
[   92.955507] Bluetooth: hci0: Firmware loaded in 1481565 usecs
[   92.955566] Bluetooth: hci0: Waiting for device to boot
[   92.967734] Bluetooth: hci0: Device booted in 11938 usecs
[   92.967737] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-12-16.ddc
[   92.979795] Bluetooth: hci0: Applying Intel DDC parameters completed
[  103.700564] hid-generic 0005:046D:B017.0007: input,hidraw1: BLUETOOTH HID v0.17 Keyboard [MX Master] on 90:61:AE:A1:45:D6
[ 1161.143665] Bluetooth: hci0: Bootloader revision 0.0 build 26 week 38 2015
[ 1161.144971] Bluetooth: hci0: Device revision is 16
[ 1161.144972] Bluetooth: hci0: Secure boot is enabled
[ 1161.144973] Bluetooth: hci0: OTP lock is enabled
[ 1161.144974] Bluetooth: hci0: API lock is enabled
[ 1161.144974] Bluetooth: hci0: Debug lock is disabled
[ 1161.144975] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[ 1161.145236] Bluetooth: hci0: Found device firmware: intel/ibt-12-16.sfi
[ 1162.761868] Bluetooth: hci0: Waiting for firmware download to complete
[ 1162.762862] Bluetooth: hci0: Firmware loaded in 1581849 usecs
[ 1162.762913] Bluetooth: hci0: Waiting for device to boot
[ 1162.774912] Bluetooth: hci0: Device booted in 11725 usecs
[ 1162.775031] Bluetooth: hci0: Found Intel DDC parameters: intel/ibt-12-16.ddc
[ 1162.786863] Bluetooth: hci0: Applying Intel DDC parameters completed
[ 1173.407531] hid-generic 0005:046D:B017.0008: input,hidraw1: BLUETOOTH HID v0.17 Keyboard [MX Master] on 90:61:AE:A1:45:D6

Comment 5 Bart Ratgers 2018-05-09 20:00:03 UTC
I reassign the existing bug to ACPI, but I'm not sure which component. I've choosing config-interrupts for now, but I'm not sure. So the reference in this bug is still valid. See: https://bugzilla.kernel.org/show_bug.cgi?id=199649

Comment 6 Hans de Goede 2018-05-10 08:17:49 UTC
Hmm, so you've an Intel wifi/bluetooth combo card rather then a Qualcom one which are usually the cards needing a quirk for this.

And you say this only happens after a suspend/resume right, so on first boot everything is fine?

Thank you for updating the kernel bug (I did not know you can simply change the Product), I think that for now we need to see if the kernel bug gets some input and we can come up with a proper fix that way.

Comment 7 Hans de Goede 2018-05-10 08:18:48 UTC
One thing worth trying is the latest 4.17 kernel: https://koji.fedoraproject.org/koji/buildinfo?buildID=1080118
Install instructions here: https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

Comment 8 Bart Ratgers 2018-05-10 10:04:51 UTC
Things became more weird. After updating to kernel version kernel-core-4.16.7-300.fc28.x86_64 and remove the kernel parameter. I can suspending en resuming without any problem. But interrupt gpe6D is still enabled after resuming (disabled during initial start). Only a few thousand interrupts i noticed on gpe 6D. And the Bluetooth device is stille working

Then I boot the system with kernel kernel-core-4.16.6-302.fc28.x86_64 and removing the kernel parameter "btusb.enable_autosuspend=0". I saw the same result: gpe6D is enabled, only a few thousand interrupts and the Bluetooth device is still working. 

I'm strugling with a few questions:
- What is the causes the interrupt storm on interrupt gpe6D?
- Why is interrupt gpe6D enabled after suspending and resuming?

I try to go back to kernel version 4.16.5-300 (the initial version of fedora 28) to see if that version change the behavior of the ACPI of the machine.

Comment 9 Hans de Goede 2018-05-10 10:11:36 UTC
Thanks for the update, I'm also in the Cc of the buzilla.kernel.org bug (and reading along there).

This seems to be something which is best handled upstream for now, if a bug fix comes out of that which we should backport, feel free to re-open this bug (or file a new one if that is more appropriate) closing this with a resolution of "upstream" for now.

Comment 10 Hans de Goede 2018-05-10 10:12:45 UTC
p.s.

As mentioned before please give 4.17-rc4 a try, I'm sure upstream will be interested in the results of that too.

Comment 11 Bart Ratgers 2018-05-15 07:36:05 UTC
I've tested the 4.17-rc4 kernel, and indeed the problem seems to bee resolved. Even better the ACPI error that I received during the boot process are also disappeared. 

Even when i suspend and resume the laptop, the gpe6d stays disabled and the mouse keeps working. 

/sys/firmware/acpi/interrupts/gpe6D:    1253         disabled     unmasked



I've also tested with all the 'old' 4.16 kernels and contrary to what I said last week Thursday the problem with an interrupt storm still exist, but happened at random. 

So the solution for my looks like the new 4.17-rc4 kernel. I will keep testing the next week.

Comment 12 Bart Ratgers 2018-05-15 14:29:34 UTC
Sadly to say, but even with the new kernel the problem still exist. But again a different behavior. 

The new situation is that when I boot up the system (initial boot), than interrupt gpe6D is disabled during the whole time. 

After I suspend and resume the laptop, then the interrupt gpe6D switching continues between enabled / disabled. 

I wrote a small script to perform a date and grep . -r /sys/firmware/acpi/interrupts/. When I move the mouse, then the interrupt is disabled and when i stop moving, then the interrupt is enabled again after a few seconds. 

16:23:12   /sys/firmware/acpi/interrupts/gpe6D: 2797 EN enabled unmasked
16:23:18   /sys/firmware/acpi/interrupts/gpe6D: 2797 EN enabled unmasked
16:23:19   /sys/firmware/acpi/interrupts/gpe6D: 2797 EN enabled unmasked
16:23:20   /sys/firmware/acpi/interrupts/gpe6D: 2797 EN enabled unmasked
16:23:21   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:22   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:23   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:24   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:25   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:26   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:27   /sys/firmware/acpi/interrupts/gpe6D: 2876 disabled unmasked
16:23:28   /sys/firmware/acpi/interrupts/gpe6D: 2876 EN enabled unmasked
16:23:30   /sys/firmware/acpi/interrupts/gpe6D: 2876 EN enabled unmasked
16:23:31   /sys/firmware/acpi/interrupts/gpe6D: 2876 EN enabled unmasked
16:23:32   /sys/firmware/acpi/interrupts/gpe6D: 2876 EN enabled unmasked
16:23:33   /sys/firmware/acpi/interrupts/gpe6D: 2936 disabled unmasked
16:23:34   /sys/firmware/acpi/interrupts/gpe6D: 2936 disabled unmasked
16:23:35   /sys/firmware/acpi/interrupts/gpe6D: 2936 disabled unmasked
16:23:36   /sys/firmware/acpi/interrupts/gpe6D: 2936 disabled unmasked
16:23:37   /sys/firmware/acpi/interrupts/gpe6D: 2936 disabled unmasked
16:23:38   /sys/firmware/acpi/interrupts/gpe6D: 2936 disabled unmasked
16:23:39   /sys/firmware/acpi/interrupts/gpe6D: 2936 EN enabled unmasked
16:23:40   /sys/firmware/acpi/interrupts/gpe6D: 2936 EN enabled unmasked

Sometimes, directly after resuming this phenomenon caused an interrupt storm on interrupt gpe6D. The reason, I don't know yet. But when such a storm occurred, then I lose my blue tooth connection.

Comment 13 Bart Ratgers 2018-05-15 14:38:56 UTC
Can this issues caused by the Intel firmware. Last year I've a sort of problem with disconnecting Bluetooth devices, see: https://bugzilla.kernel.org/show_bug.cgi?id=194815

Comment 14 Justin M. Forbes 2018-07-23 14:54:50 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 28 kernel bugs.

Fedora 28 has now been rebased to 4.17.7-200.fc28.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you experience different issues, please open a new bug report for those.

Comment 15 Bart Ratgers 2018-07-28 10:36:41 UTC
I tested with the new 4.17.7-200.fc28.x86_64 kernel and the problems seems to be resolved. No interrupt storm after resuming and a working Logitech MX Master mouse.

I close this bug.

Comment 16 Hans de Goede 2018-07-29 09:26:38 UTC
(In reply to Bart Ratgers from comment #15)
> I tested with the new 4.17.7-200.fc28.x86_64 kernel and the problems seems
> to be resolved. No interrupt storm after resuming and a working Logitech MX
> Master mouse.
> 
> I close this bug.

Thank you for the update and good to hear that this is fixed now.