Thank you for reporting this, the btusb code has the following comment for QCA devices:

        if (id->driver_info & BTUSB_QCA_ROME) {
                data->setup_on_usb = btusb_setup_qca;
                hdev->set_bdaddr = btusb_set_bdaddr_ath3012;

                /* QCA Rome devices lose their updated firmware over suspend,
                 * but the USB hub doesn't notice any status change.
                 * Explicitly request a device reset on resume.
                 */
                set_bit(BTUSB_RESET_RESUME, &data->flags);
        }

I guess the "lose their updated firmware over suspend" also applies to them being runtime-suspended. Since reloading the firmware on a runtime-resume will introduce an unacceptable latency for for example bt keyboard key-presses, the solution seems to be to simple disable runtime-pm for QCA devices. So what you are doing with btusb.enable_autosuspend=n but then automatically for all QCA devices (and for QCA devices only).

I will write a patch for this and submit it upstream, once upstream has merged it I will add it to the Fedora kernels.

Ok, slight change of plan, I've come up with a nice and clean way to fix this, but before submitting this upstream I would like to know that the patch actually works :)

So I've kicked of a scratch kernel-build with the patch added:
https://koji.fedoraproject.org/koji/taskinfo?taskID=23983840

Once this is done building, can you give this a try please and confirm that you have working bluetooth with this, without the btusb.enable_autosuspend=n kernel cmdline option.

I can confirm booing that scratch build with no 'btusb.enable_autosuspend' specified and bluetooth works. ;)

(In reply to Kevin Fenzi from comment #3)
> I can confirm booing that scratch build with no 'btusb.enable_autosuspend'
> specified and bluetooth works. ;)

Great, thank you for testing. I've submitted the patch upstream and added it to the Fedora kernel pkgs for F28+, to be picked up by the next official build.

So to clarify, this means this hardware gets no power savings from bluetooth autosuspend right?

(In reply to Kevin Fenzi from comment #5)
> So to clarify, this means this hardware gets no power savings from bluetooth
> autosuspend right?

Correct. I'm afraid there is not much we can do about that in this case.

So upstream has been making some changes surrounding suspend/resume handling for QCA devices and I'm going to have to respin my patch to apply on top of the new code for 4.16.

Kevin, can you run lsusb on the machine in question please?

Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 005: ID 0cf3:e300 Qualcomm Atheros Communications 
Bus 001 Device 004: ID 06cb:0081 Synaptics, Inc. 
Bus 001 Device 003: ID 1050:0407 Yubico.com Yubikey 4 OTP+U2F+CCID
Bus 001 Device 002: ID 5986:210d Acer, Inc 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Thanks, here is a scratch-build (still building atm) with a different version of the fix for this, this new version should go upstream for 4.16, so if you've some time to test this then that would be great:

https://koji.fedoraproject.org/koji/taskinfo?taskID=24071811

Bluetooth works ok for me with this kernel...

Ok, so the chromeos people complained about just blacklisting runtime-suspend for all QCA devices, since this turns out to not be a chipset bug, but a platform bug, so the plan is to move the quirks for not doing runtime-suspend for these over the DMI matching.

Kevin, can you provide "dmidecode" output please (note that will have serial numbers in there which you may want to scrub) ?

This is long fixed in current Fedora kernels BTW, but that fix is going to get reverted and replaced with a DMI based blacklist (instead of blacklisting by usb-ids).

Created attachment 1397332 [details]
dmidecode from yoga 920

Here's the dmidecode.

Thanks. Here is a new (rawhide) kernel scratchbuild using the new approach
(note still building atm) can you give this a test-run if you've some time and test that it does not regress things please ?  :

https://koji.fedoraproject.org/koji/taskinfo?taskID=25162667

p.s.

Can you also please do:

sudo cat /sys/kernel/debug/usb/devices

And paste the output here? Upstream wants this in the commit message.

Seems to work ok, I do see this in dmesg: 

[Mon Feb 19 09:58:25 2018] Bluetooth: hci0: last event is not cmd complete (0x0f)
[Mon Feb 19 09:58:27 2018] Bluetooth: hci0: SCO packet for unknown connection handle 9
[Mon Feb 19 09:58:27 2018] Bluetooth: hci0: SCO packet for unknown connection handle 0
...bunch more of those... 
[Mon Feb 19 09:59:29 2018] Bluetooth: hci0: last event is not cmd complete (0x0f)
[Mon Feb 19 09:59:45 2018] Bluetooth: hci0: last event is not cmd complete (0x0f)
[Mon Feb 19 10:00:01 2018] Bluetooth: hci0: last event is not cmd complete (0x0f)
[Mon Feb 19 10:00:18 2018] Bluetooth: hci0: last event is not cmd complete (0x0f)

Will attach the usb devices output.

Created attachment 1397973 [details]
usb-devices

Thank you for the quick test, those errors seem to be unrelated. Maybe something new with 4.16 in general?

Could be indeed.

Hi Kevin,

So the ChromeOS people think your initial issues may have been caused by the initial reset-resume workaround for QCA devices being broken in general and that maybe your laptop can work without any quirk at all...

So I'm sorry to have to ask you this, but here is another scratch-build which completely removes all quirks for your laptop. Note if this works it will save about 0.4W of power which should result in a nice battery-life gain (*)

Here is the scratch-build (still building atm):
https://koji.fedoraproject.org/koji/taskinfo?taskID=25345663

Regards,

Hans

*) You may be able to get the same gain through "rfkill block bluetooth" if that causes your BIOS to completely drop the USB device from the USB bus, at least some Dell-s do that.

After debugging some selinux issues it does indeed appear to work. ;)

Cool, so what is the output of:

cat /sys/bus/usb/devices/*/power/control

?

cat /sys/bus/usb/devices/*/power/control

auto
on
on
auto
auto
auto
auto
auto

Hmm, I wonder what those 2 "on-s" are, can you do:

ls /sys/bus/usb/devices/*/power/control

And:

lsusb -t

?

 ls /sys/bus/usb/devices/*/power/control

/sys/bus/usb/devices/1-1/power/control  /sys/bus/usb/devices/usb1/power/control
/sys/bus/usb/devices/1-3/power/control  /sys/bus/usb/devices/usb2/power/control
/sys/bus/usb/devices/1-5/power/control  /sys/bus/usb/devices/usb3/power/control
/sys/bus/usb/devices/1-8/power/control  /sys/bus/usb/devices/usb4/power/control

lsusb -t
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 10000M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/2p, 480M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/6p, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/12p, 480M
    |__ Port 1: Dev 2, If 1, Class=Video, Driver=uvcvideo, 480M
    |__ Port 1: Dev 2, If 0, Class=Video, Driver=uvcvideo, 480M
    |__ Port 3: Dev 6, If 2, Class=Chip/SmartCard, Driver=, 12M
    |__ Port 3: Dev 6, If 0, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 3: Dev 6, If 1, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 5: Dev 4, If 0, Class=Vendor Specific Class, Driver=, 12M
    |__ Port 8: Dev 5, If 1, Class=Wireless, Driver=btusb, 12M
    |__ Port 8: Dev 5, If 0, Class=Wireless, Driver=btusb, 12M

Ok, so the 2 usb devices which are not at "auto" are:

1) Your yubikey
2) The fingerprint reader

According to: https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/udev/gen_autosuspend_rules.py#164

Enabling autosuspend is safe for older versions of the yubikey, so I guess it should probably be safe for the newer one too ?

And autosuspend is ok to use with fingerprint readers in general.

Can you try doing a manual "echo auto > ..." for the 2 /sys/bus/usb/devices/*/power/control files which are currently "on" ? This should give you some battery savings (around 0.4W for Haswell, likely less on newer machines like yours as I expect the USB controller to be more energy efficient there).

If you can run with them set to auto for a couple of days and then report back, then I will add these to the list of usb-ids to whitelist for autosuspend which I'm working on.

###

As for the original bluetooth issue, thank you for testing. I've submitted yet another patch upstream removing the Yoga 920 from the new DMI based reset-resume quirk list.

I'm just trying to know if this issue is related, as I don't know what happens exactly when I connect my "Creative Aurvana" headset. The GNOME Bluetooth panel turns off (rfkill soft block) and these messages appear in the journal:

Mar 19 08:37:00 anass-galago kernel: Bluetooth: hci0: last event is not cmd complete (0x0f)
Mar 19 08:37:17 anass-galago bluetoothd[15044]: Disconnected from D-Bus. Exiting.
Mar 19 08:37:17 anass-galago bluetoothd[15044]: Endpoint unregistered: sender=:1.84 path=/MediaEndpoint/A2DPSource
Mar 19 08:37:17 anass-galago bluetoothd[15044]: Endpoint unregistered: sender=:1.84 path=/MediaEndpoint/A2DPSink
Mar 19 08:37:17 anass-galago bluetoothd[15044]: Stopping SDP server
Mar 19 08:37:17 anass-galago bluetoothd[15044]: Exit
Mar 19 08:37:17 anass-galago NetworkManager[1042]: <info>  [1521441437.6696] device (F0:EE:10:64:06:69): state change: disconnected -> unmanaged (reason 'removed', sys-iface-state: 'removed')
Mar 19 08:37:17 anass-galago dbus-daemon[1282]: [session uid=42 pid=1282] Activating service name='ca.desrt.dconf' requested by ':1.13' (uid=42 pid=1301 comm="/usr/bin/gnome-shell " label="system_u:system_r:xdm_t:s0-s0:c0.c1023")
Mar 19 08:37:17 anass-galago audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=bluetooth comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Mar 19 08:37:17 anass-galago gnome-shell[1301]: Error setting property 'Powered' on interface org.bluez.Adapter1: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name :1.137 was not provided by any .service files (g-dbus-error-quark, 2)
Mar 19 08:37:17 anass-galago gnome-shell[4138]: Error setting property 'Powered' on interface org.bluez.Adapter1: GDBus.Error:org.freedesktop.DBus.Error.ServiceUnknown: The name :1.137 was not provided by any .service files (g-dbus-error-quark, 2)
Mar 19 08:37:17 anass-galago dbus-daemon[1282]: [session uid=42 pid=1282] Successfully activated service 'ca.desrt.dconf'

I'm failing to know if the problem is from the kernel, or from somewhere up in the stack.

Anass, are you running F28 nightly composes or rawhide? If you're not running those then you do not have usb autosuspend for bluetooth enabled and this bug does not apply to you.

If you are running F28+, then try adding:

btusb.enable_autosuspend=n

To your kernel commandline, if that fixes things then you are affected by this bug, if that does not fix things then you've a different bug and should file a new bug.

(In reply to Hans de Goede from comment #27)
> Anass, are you running F28 nightly composes or rawhide? If you're not
> running those then you do not have usb autosuspend for bluetooth enabled and
> this bug does not apply to you.

Thank you for your prompt response, I'm running F28 rawhide, that's why I thought this bug might be related.

the relevant software version:

alsa-plugins-pulseaudio-1.1.5-2.fc28.i686
alsa-plugins-pulseaudio-1.1.5-2.fc28.x86_64
bluez-5.48-4.fc28.x86_64
bluez-cups-5.48-4.fc28.x86_64
bluez-libs-5.48-4.fc28.x86_64
bluez-libs-devel-5.48-4.fc28.x86_64
bluez-obexd-5.48-4.fc28.x86_64
gnome-bluetooth-3.28.0-1.fc28.x86_64
gnome-bluetooth-libs-3.28.0-1.fc28.x86_64
kernel-4.15.9-300.fc27.x86_64
kernel-4.16.0-0.rc4.git0.1.fc28.x86_64
kernel-4.16.0-0.rc5.git0.2.fc28.x86_64
kernel-core-4.15.9-300.fc27.x86_64
kernel-core-4.16.0-0.rc4.git0.1.fc28.x86_64
kernel-core-4.16.0-0.rc5.git0.2.fc28.x86_64
kernel-debug-devel-4.15.9-300.fc27.x86_64
kernel-debug-devel-4.16.0-0.rc4.git0.1.fc28.x86_64
kernel-debug-devel-4.16.0-0.rc5.git0.2.fc28.x86_64
kernel-devel-4.15.9-300.fc27.x86_64
kernel-devel-4.16.0-0.rc4.git0.1.fc28.x86_64
kernel-devel-4.16.0-0.rc5.git0.2.fc28.x86_64
kernel-headers-4.16.0-0.rc5.git0.2.fc28.x86_64
kernel-modules-4.15.9-300.fc27.x86_64
kernel-modules-4.16.0-0.rc4.git0.1.fc28.x86_64
kernel-modules-4.16.0-0.rc5.git0.2.fc28.x86_64
kernel-tools-4.16.0-0.rc5.git0.1.fc28.x86_64
kernel-tools-libs-4.16.0-0.rc5.git0.1.fc28.x86_64
kf5-bluez-qt-5.44.0-1.fc28.x86_64
NetworkManager-1.10.6-1.fc28.x86_64
NetworkManager-bluetooth-1.10.6-1.fc28.x86_64
pulseaudio-11.1-16.fc28.x86_64
pulseaudio-gdm-hooks-11.1-16.fc28.x86_64
pulseaudio-libs-11.1-16.fc28.i686
pulseaudio-libs-11.1-16.fc28.x86_64
pulseaudio-libs-glib2-11.1-16.fc28.x86_64
pulseaudio-module-bluetooth-11.1-16.fc28.x86_64
pulseaudio-module-gconf-11.1-16.fc28.x86_64
pulseaudio-module-x11-11.1-16.fc28.x86_64
pulseaudio-utils-11.1-16.fc28.x86_64

> If you are running F28+, then try adding:
> 
> btusb.enable_autosuspend=n
> 
> To your kernel commandline, if that fixes things then you are affected by
> this bug, if that does not fix things then you've a different bug and should
> file a new bug.

I tried it but the same issue happened, it's just like someone issues "systemctl stop bluetooth" as soon as the connection with my headset is established.

logs can be found here:
https://paste.fedoraproject.org/paste/tozGJTmjRjFW8ML5ulSouw

(In reply to Anass Ahmed from comment #28)
> > If you are running F28+, then try adding:
> > 
> > btusb.enable_autosuspend=n
> > 
> > To your kernel commandline, if that fixes things then you are affected by
> > this bug, if that does not fix things then you've a different bug and should
> > file a new bug.
> 
> I tried it but the same issue happened, it's just like someone issues
> "systemctl stop bluetooth" as soon as the connection with my headset is
> established.

Hmm, this sounds like a userspace bug to me, what does:

systemctl status bluetooth.service

Say after this has happened? Is bluez still running?

Eitherway please file a new bug against "bluez" for this and if "systemctl status bluetooth.service" yields any interesting results, include those.

(In reply to Hans de Goede from comment #29)
> Hmm, this sounds like a userspace bug to me, what does:
> 
> systemctl status bluetooth.service
> 
> Say after this has happened? Is bluez still running?
> 
> Eitherway please file a new bug against "bluez" for this and if "systemctl
> status bluetooth.service" yields any interesting results, include those.

Oh, I did, sir, it's all here in Bug 1558042.

But I didn't know where to put it, so I reported it against GNOME Bluetooth.

Thanks again.

Laptop: Dell XPS13 9360
Bluetooth: Qualcomm Atheros QCA6174 (rev 32); wifi & bluetooth

Summary: 'btusb.enable_autosuspend=n' greatly improves Bluetooth reliability in Fedora 28 beta on a Dell XPS 13 with a "Killer Wireless AC-1535" (Atheros) card


After upgrading to Fedora 28 pre-beta on my XPS13 a few weeks ago, I was having issues with my Logitech MX ERGO trackball staying connected. Bluetooth would disappear completely, and it would not show up in the system.

Thankfully, I remembered the blog post about better power saving making its way to Fedora 28 and I found this issue.

After adding 'btusb.enable_autosuspend=n' to grub (and rebooting), Bluetooth works as expected... and keeps working as expected. I've tested it for at least a week and a half, and things are *much* better.


I have noticed Bluetooth disappearing twice since, but now only during a complete suspend and resume of the entire laptop — walking away from the laptop for a few minutes no longer causes the Bluetooth stack to fall down completely. Autosuspend seems to greatly increase the likelihood of things breaking.


Is the patch in the most recent kernel? I've upgraded since having these problems and your fix might now be included. 

Currently, I'm on kernel-4.16.0-0.rc6.git0.2.fc28.x86_64.

Thanks!

(In reply to Garrett LeSage from comment #31)
> Laptop: Dell XPS13 9360
> Bluetooth: Qualcomm Atheros QCA6174 (rev 32); wifi & bluetooth

Thank you for reporting this, can you do:

sudo dmidecode > dmi.log

And attach the generated dmi.log file here please ?

I need info from that file to add a quirk for your model laptop. I will get back to you with a test kernel once I've prepared a patch.

Note that file will also contain the laptops serial and I guess also its assettag so you may want to make the attachment private.

Created attachment 1419997 [details]
dmidecode from XPS13 9360

(redacted serial numbers)

Hi Garrett,

I've prepared a patch to fix this and started a kernel scratch-build with the patch included:

https://koji.fedoraproject.org/koji/taskinfo?taskID=26307651

Note this is still building atm, once the build has finished, please test this, you can find generic testing instructions for kernel scratch-builds here:

https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt

As noted in the instructions you should remove the kernel commandline option you added to test. Also please test if bluetooth survives a normal suspend/resume with the patched kernel (it should).

Once you done testing please let me know if this fixed the need to manually disable runtime_pm as well as bluetooth being broken after putting the laptop to sleep. Once I've your feedback I will submit the patch upstream.

Regards,

Hans

Note the scratch-build is ready for testing now, grab it while it is hot as koji removes the files of scratchbuilds in a couple of days.

Garett,

If you can make some time to test this, that would be great. Once you've confirmed the fix I can submit it upstream so that this will be fixed for all XPS 13 9360 owners.

I've saved a copy of the kernel rpms here, before koji cleans them up:

https://fedorapeople.org/~jwrdegoede/rhbz1514836/

Thanks & Regards,

Hans

Garrett, ping ?  It would be really nice to get test-results from you so that I can submit the patch fixing this upstream.

Oops! Sorry; I was focused on editing videos for RH Summit for the past couple weeks. 

I'll test these on my Dell ASAP. Thanks!

So far, so good. 

I've tried suspending and resuming the Dell several times, and everything works, including Bluetooth.

I'll keep trying it out tonight and the next few days and will let you know if I encounter any issues.

Thanks for the testing. I've submitted the patch fixing this upstream.

Jeremy I've added you to the Cc while submitting the patch upstream, perhaps you can add it to the Fedora kernels for now?

Bluetooth has been flawless all night. I purposefully let my laptop sit around unattended for a while and also suspended / resumed at least half a dozen more times too. No issues; everything's great.

Thanks so much for all your hard work!

I do know there are two (at least) varieties of the XPS 13 9360 - one has the QCA chip and the other has an Intel 8265 chip. I've got the Intel variety and I wonder if your patch will have unintended consequences on it? It does seem like the Intel bluetooth is already getting reset since I see it downloading the firmware again on resume, but I haven't looked closely at the code yet. I'll do some testing later tonight. If all goes well I'll see about adding it to the 4.16.5 build which should be tomorrow.

(In reply to Jeremy Cline from comment #42)
> I do know there are two (at least) varieties of the XPS 13 9360 - one has
> the QCA chip and the other has an Intel 8265 chip. I've got the Intel
> variety and I wonder if your patch will have unintended consequences on it?
> It does seem like the Intel bluetooth is already getting reset since I see
> it downloading the firmware again on resume, but I haven't looked closely at
> the code yet. I'll do some testing later tonight. If all goes well I'll see
> about adding it to the 4.16.5 build which should be tomorrow.

Ah, good point, one change which this will probably cause is higher idle power-consumption, since this also disables runtime-pm for the bluetooth module (which in turn will keep the USB controller alive all the time).

I will submit a patch upstream to limit the quirk to only apply to QCA_ROME devices.

kernel-4.16.5-200.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-09afae3bb9

kernel-4.16.5-300.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-a9d6bb6a8e

Hi Garrett,

I've included Hans' patches in the 4.16.5 kernel which should arrive in the updates-testing repositories soon. Can you test it out and provide karma on the update?

Thanks!

kernel-4.16.5-300.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-a9d6bb6a8e

It's working perfectly here and I've just added karma on Bodhi reflecting it.

Thanks again, all!

kernel-4.16.5-200.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-09afae3bb9

kernel-4.16.5-200.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

kernel-4.16.5-300.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.