Created attachment 1915347 [details] Kernel logs, most relevant towards the end My Asus ExpertBook B7 Flip contains an "Intel 5G Solution 5000" data modem, which is actually a "Fibocom FM350-FL" going by what's printed on the board. This modem is partially handled by the "mtk_t7xx" kernel module, being detected as "generic MBIM [14C3:4D75]" firmware version "81600.0000.00.29.20.01_GC C62". I say "partially" because it does eventually provide an internet connection after some fiddling, but the journald logs are rather filled with red lines about this module and device. A particularly serious issue is that it completely prevents the system from suspending. I attach the logs from an attempt to suspend. The only way I've found to overcome this issue is to virtually remove this modem from the PCIe bus by issuing `echo 1 > /sys/devices/pci0000:00/0000:00:1c.5/0000:58:00.0/remove` before suspending. This action doesn't appear reversible, and while the system suspends successfully, after resuming the modem won't work (and in my experience the Intel CPU is also often prevented from reaching PC10 state, at least after a deep suspend). I am running Linux version 5.19.12-300.fc37.x86_64. This computer is new, but I've encountered this issue in all of Fedora 36, Fedora 37 (which I'm using now and which the logs come from) and Fedora Rawhide from a few days ago. I do not suspect this issue would NOT occur on older versions. To reproduce the issue, attempt to suspend the system using any supported method, with /sys/power/mem_sleep set to either s2idle or deep. All the modules in my system come with Fedora and were enabled automatically by the system. I am attaching kernel logs, stripped of lines containing my MAC addresses.
I believe this bug affects me as well. I've spent several hours over the last couple of days digging into why my machine tries to suspend but immediately wakes up. After digging through journalctl I came across this bug report (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990700) where @ljlbox posted a link to this report, and to the best of my knowledge it comes down to a bug with the Fibocom/mtk_t7xx module. Specifications: Levovo ThinkPad P1 Gen 5 with optional Fibocom FM350-GL 5G SUB6 Modem Fedora 36: Relatively fresh install, though I have been messing with the sleep options trying to get it to stay suspended Gnome Wayland, but tested with KDE Plasma X11 and the problem is identical Symptoms: Suspend blanks the screen, but within 1 second the keyboard lights up again then the screen turns back on to the lock screen. The same applies when closing the lid. Some excepts from journalctl: kernel: mtk_t7xx 0000:08:00.0: [PM] SAP suspend error: -110 kernel: mtk_t7xx 0000:08:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -110 kernel: mtk_t7xx 0000:08:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x160 returns -110 kernel: mtk_t7xx 0000:08:00.0: PM: failed to suspend async: error -110 kernel: PM: Some devices failed to suspend, or early wake event detected lspci for device: 08:00.0 Wireless controller [0d40]: MEDIATEK Corp. Device 4d75 (rev 01) Subsystem: Device 1cf8:3502 Flags: bus master, fast devsel, latency 0, IRQ 16, IOMMU group 19 Memory at 625c800000 (64-bit, prefetchable) [size=32K] Memory at b4800000 (64-bit, non-prefetchable) [size=8M] Memory at 625c000000 (64-bit, prefetchable) [size=8M] Capabilities: [80] Express Endpoint, MSI 00 Capabilities: [d0] MSI-X: Enable+ Count=34 Masked- Capabilities: [e0] MSI: Enable- Count=1/32 Maskable+ 64bit+ Capabilities: [f8] Power Management version 3 Capabilities: [100] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?> Capabilities: [108] Latency Tolerance Reporting Capabilities: [110] L1 PM Substates Capabilities: [200] Advanced Error Reporting Capabilities: [300] Secondary PCI Express Kernel driver in use: mtk_t7xx Kernel modules: mtk_t7xx I attempted using the echo 1 > (device)/remove, but while the device was disabled, it did not seem to resolve the suspend issue for me. I consider myself an intermediate Linux user, mainly focused on running servers and messing with software. I very rarely get into kernel modules/builds, but am willing to do some testing if provided the instructions. If I can provide any additional diagnostic information, please ping me.
same issue for me: Fedora 36, 6.0.5-200.fc36.x86_64 on a gen10 thinkpad x1 carbon . The MediaTek PCIe 5G WWAN modem will not let the system suspend mtk_t7xx 0000:08:00.0: [PM] SAP suspend error: -110 mtk_t7xx 0000:08:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -110 mtk_t7xx 0000:08:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x160 returns -110 mtk_t7xx 0000:08:00.0: PM: failed to suspend async: error -110 PM: Some devices failed to suspend, or early wake event detected
It might be worth noting that the https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990700 issue that @bugzilla point out contains patches (or at least references to patches that are claimed to solve the issues by some parties). It looks like the Ubuntu people have decided to use an ad-hoc patch for the involved kernel module for now.
Just an FYI: the issue persists, for me, with kernel 6.0.6-300.fc37.x86_64 which was recently added to Fedora 37. Upon attempting to suspend with this version, I get xhci_hcd 0000:00:14.0: PM: pci_pm_suspend(): hcd_pci_suspend+0x0/0x30 returns -16 xhci_hcd 0000:00:14.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x160 returns -16 xhci_hcd 0000:00:14.0: PM: failed to suspend async: error -16 mtk_t7xx 0000:58:00.0: [PM] SAP suspend error: -110 mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -110 mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x160 returns -110 mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -110 PM: Some devices failed to suspend, or early wake event detected pcieport 10000:e0:1d.0: can't derive routing for PCI INT A nvme 10000:e1:00.0: PCI INT A: no GSI nvme nvme0: Shutdown timeout set to 10 seconds nvme nvme0: 8/0/0 default/read/poll queues PM: resume devices took 0.268 seconds For @bugzilla my workaround still works (but the device doesn't come back, despite the "post" line). It consists in creating a file /usr/lib/systemd/system-sleep/broadband-modem containing #!/usr/bin/sh case $1 in pre) modprobe -r mtk_t7xx && echo 1 > /sys/bus/pci/devices/0000:58:00.0/remove ;; post) echo 1 > /sys/bus/pci/rescan && modprobe mtk_t7xx ;; esac
Appreciate the workaround, it didn't seem to do it for me. My device is 0000:08:00.0 which I located under /sys/bus/pci/devices/ but echo 1 >, whether manually or via your script (with and without escaping colons) still produces the same `mtk_t7xx 0000:08:00.0: [PM] SAP suspend error: -110`. For now I'm just fully shutting down, and have considered temporarily, physically disconnecting the card to see if that helps. I don't need it yet, I just bought the laptop with it in case I do need it in the future.
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1990700 This seems to have been fixed for ubuntu; what is the redhat equivalent of linux-oem, and how long does it usually take for those fixes to come to redhat/fedora?
Ubuntu seems to have fixed this by cherry-picking: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d20ef656f9942950d67e29418512a6929682ad60 which was reverted upstream because apparently it was merged without proper review and which has not shown up upstream again since :| I have started a test build of the Fedora kernel (6.1.0 based from rawhide) with this patch added, this is currently building here: https://koji.fedoraproject.org/koji/taskinfo?taskID=95395499 This should be done building in a couple of hours (once it is done the link will show rpms to downloads). Please install this test kernel and let me know if it fixes the suspend-resume issue. For some generic instructions for installing a kernel directly from koji (the Fedora buildsystem) see here: https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt
Unfortunately, although the system appeared to suspend on the first try, logs still indicated misbehavior by the Fibocom modem, and on the second try, it failed to suspend entirely. The Fibocom was also no longer available to use. This was on my Fedora 33 on an Asus Expertbook B7 Flip 11th gen, running Linux version 6.1.0-65.bz2131366.fc37.x86_64 (mockbuild.fedoraproject.org) (gcc (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4), GNU ld version 2.38-25.fc37) #1 SMP PREEMPT_DYNAMIC Thu Dec 15 12:44:09 UTC 2022 Here are journal snippets from both attempts: 1) Dec 15 21:16:47 ljlbook systemd-logind[984]: Power key pressed short. Dec 15 21:16:47 ljlbook ModemManager[2020]: <info> [sleep-monitor] system is about to suspend Dec 15 21:16:47 ljlbook ModemManager[2020]: [/dev/wwan0mbim0] channel destroyed Dec 15 21:16:47 ljlbook NetworkManager[1217]: <info> [1671135407.8627] manager: sleep: sleep requested (sleeping: no enabled: yes) Dec 15 21:16:47 ljlbook NetworkManager[1217]: <info> [1671135407.8630] device (eno2): state change: unavailable -> unmanaged (reason 'sleeping', sys-iface-state: 'managed') [...] Dec 15 21:16:48 ljlbook kernel: PM: suspend entry (deep) Dec 15 21:16:48 ljlbook wpa_supplicant[1324]: wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:16:48 ljlbook kernel: Filesystems sync: 0.018 seconds Dec 15 21:16:48 ljlbook wpa_supplicant[1324]: wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:16:48 ljlbook wpa_supplicant[1324]: nl80211: deinit ifname=wlo1 disabled_11b_rates=0 Dec 15 21:16:57 ljlbook kernel: Freezing user space processes ... (elapsed 0.001 seconds) done. Dec 15 21:16:57 ljlbook kernel: OOM killer disabled. Dec 15 21:16:57 ljlbook kernel: Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. Dec 15 21:16:57 ljlbook kernel: printk: Suspending console(s) (use no_console_suspend to debug) Dec 15 21:16:57 ljlbook kernel: e1000e: EEE TX LPI TIMER: 00000011 Dec 15 21:16:57 ljlbook kernel: PM: suspend devices took 0.328 seconds Dec 15 21:16:57 ljlbook kernel: ACPI: EC: interrupt blocked Dec 15 21:16:57 ljlbook kernel: ACPI: PM: Preparing to enter system sleep state S3 Dec 15 21:16:57 ljlbook kernel: ACPI: EC: event blocked Dec 15 21:16:57 ljlbook kernel: ACPI: EC: EC stopped Dec 15 21:16:57 ljlbook kernel: ACPI: PM: Saving platform NVS memory Dec 15 21:16:57 ljlbook kernel: Disabling non-boot CPUs ... Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 1 is now offline Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 2 is now offline Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 3 is now offline Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 4 is now offline Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 5 is now offline Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 6 is now offline Dec 15 21:16:57 ljlbook kernel: IRQ149: set affinity failed(-22). Dec 15 21:16:57 ljlbook kernel: smpboot: CPU 7 is now offline Dec 15 21:16:57 ljlbook kernel: [Firmware Bug]: TSC ADJUST differs: CPU0 0 --> -681990681. Restoring Dec 15 21:16:57 ljlbook kernel: ACPI: PM: Low-level resume complete Dec 15 21:16:57 ljlbook kernel: ACPI: EC: EC started Dec 15 21:16:57 ljlbook kernel: ACPI: PM: Restoring platform NVS memory Dec 15 21:16:57 ljlbook kernel: Enabling non-boot CPUs ... Dec 15 21:16:57 ljlbook kernel: x86: Booting SMP configuration: Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 1 APIC 0x2 Dec 15 21:16:57 ljlbook kernel: CPU1 is up Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 2 APIC 0x4 Dec 15 21:16:57 ljlbook kernel: CPU2 is up Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 3 APIC 0x6 Dec 15 21:16:57 ljlbook kernel: CPU3 is up Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 4 APIC 0x1 Dec 15 21:16:57 ljlbook kernel: CPU4 is up Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 5 APIC 0x3 Dec 15 21:16:57 ljlbook kernel: CPU5 is up Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 6 APIC 0x5 Dec 15 21:16:57 ljlbook kernel: CPU6 is up Dec 15 21:16:57 ljlbook kernel: smpboot: Booting Node 0 Processor 7 APIC 0x7 Dec 15 21:16:57 ljlbook kernel: CPU7 is up Dec 15 21:16:57 ljlbook kernel: ACPI: PM: Waking up from system sleep state S3 Dec 15 21:16:57 ljlbook kernel: ACPI: EC: interrupt unblocked Dec 15 21:16:57 ljlbook kernel: ACPI: EC: event unblocked Dec 15 21:16:57 ljlbook kernel: pcieport 10000:e0:1d.0: can't derive routing for PCI INT A Dec 15 21:16:57 ljlbook kernel: nvme 10000:e1:00.0: PCI INT A: no GSI Dec 15 21:16:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM configuration timed out Dec 15 21:16:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_resume+0x0/0xe0 returns -110 Dec 15 21:16:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to resume async: error -110 Dec 15 21:16:57 ljlbook kernel: nvme nvme0: Shutdown timeout set to 10 seconds Dec 15 21:16:57 ljlbook kernel: nvme nvme0: 8/0/0 default/read/poll queues Dec 15 21:16:57 ljlbook kernel: xhci_hcd 0000:00:0d.0: xHC error in resume, USBSTS 0x401, Reinit Dec 15 21:16:57 ljlbook kernel: usb usb1: root hub lost power or was reset Dec 15 21:16:57 ljlbook kernel: usb usb2: root hub lost power or was reset Dec 15 21:16:57 ljlbook kernel: usb 3-5: reset high-speed USB device number 2 using xhci_hcd Dec 15 21:16:57 ljlbook kernel: usb 3-10: reset full-speed USB device number 5 using xhci_hcd [...] Dec 15 21:16:57 ljlbook systemd-resolved[866]: Clock change detected. Flushing caches. Dec 15 21:16:57 ljlbook kernel: PM: suspend exit [...] Dec 15 21:16:57 ljlbook systemd[1]: systemd-suspend.service: Deactivated successfully. Dec 15 21:16:57 ljlbook systemd[1]: Finished systemd-suspend.service - System Suspend. Dec 15 21:16:57 ljlbook audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Dec 15 21:16:57 ljlbook audit[1]: SERVICE_STOP pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' Dec 15 21:16:57 ljlbook systemd[1]: Stopped target sleep.target - Sleep. Dec 15 21:16:57 ljlbook systemd[1]: Reached target suspend.target - Suspend. Dec 15 21:16:57 ljlbook systemd-logind[984]: Operation 'sleep' finished. Dec 15 21:16:57 ljlbook systemd[1]: Stopped target suspend.target - Suspend. Dec 15 21:16:57 ljlbook ModemManager[2020]: <info> [sleep-monitor] system is resuming 2) Dec 15 21:18:51 ljlbook systemd-logind[984]: Power key pressed short. Dec 15 21:18:51 ljlbook ModemManager[2020]: <info> [sleep-monitor] system is about to suspend Dec 15 21:18:51 ljlbook NetworkManager[1217]: <info> [1671135531.8900] manager: sleep: sleep requested (sleeping: no enabled: yes) [...] Dec 15 21:18:51 ljlbook ModemManager[2020]: [/dev/wwan0mbim0] channel destroyed [...] [...] Dec 15 21:18:51 ljlbook NetworkManager[1217]: <info> [1671135531.9668] device (wwan0mbim0): state change: unavailable -> unmanaged (reason 'sleeping', sys-iface-state: 'managed') Dec 15 21:18:51 ljlbook NetworkManager[1217]: <info> [1671135531.9669] manager: NetworkManager state is now ASLEEP Dec 15 21:18:51 ljlbook NetworkManager[1217]: <info> [1671135531.9674] device (wlo1): state change: activated -> deactivating (reason 'sleeping', sys-iface-state: 'managed') Dec 15 21:18:51 ljlbook systemd[1]: Starting NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service... Dec 15 21:18:51 ljlbook systemd[1]: Started NetworkManager-dispatcher.service - Network Manager Script Dispatcher Service. Dec 15 21:18:51 ljlbook audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' [...] [...] [...] [...] Dec 15 21:18:52 ljlbook systemd[1]: Reached target sleep.target - Sleep. Dec 15 21:18:52 ljlbook wpa_supplicant[1324]: p2p-dev-wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:18:52 ljlbook wpa_supplicant[1324]: p2p-dev-wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:18:52 ljlbook wpa_supplicant[1324]: nl80211: deinit ifname=p2p-dev-wlo1 disabled_11b_rates=0 Dec 15 21:18:52 ljlbook systemd[1]: Starting systemd-suspend.service - System Suspend... Dec 15 21:18:52 ljlbook wpa_supplicant[1324]: wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:18:52 ljlbook systemd-sleep[4548]: Entering sleep state 'suspend'... Dec 15 21:18:52 ljlbook kernel: PM: suspend entry (deep) Dec 15 21:18:52 ljlbook kernel: Filesystems sync: 0.012 seconds Dec 15 21:18:52 ljlbook wpa_supplicant[1324]: wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:18:52 ljlbook wpa_supplicant[1324]: nl80211: deinit ifname=wlo1 disabled_11b_rates=0 Dec 15 21:18:52 ljlbook kernel: Freezing user space processes ... (elapsed 0.001 seconds) done. Dec 15 21:18:52 ljlbook kernel: OOM killer disabled. Dec 15 21:18:52 ljlbook kernel: Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. Dec 15 21:18:52 ljlbook kernel: printk: Suspending console(s) (use no_console_suspend to debug) Dec 15 21:18:52 ljlbook kernel: mtk_t7xx 0000:58:00.0: [PM] Exiting suspend, modem in invalid state Dec 15 21:18:52 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -14 Dec 15 21:18:52 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -14 Dec 15 21:18:52 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -14 Dec 15 21:18:52 ljlbook kernel: PM: Some devices failed to suspend, or early wake event detected Dec 15 21:18:52 ljlbook kernel: pcieport 10000:e0:1d.0: can't derive routing for PCI INT A Dec 15 21:18:52 ljlbook kernel: nvme 10000:e1:00.0: PCI INT A: no GSI Dec 15 21:18:52 ljlbook kernel: nvme nvme0: Shutdown timeout set to 10 seconds Dec 15 21:18:52 ljlbook kernel: nvme nvme0: 8/0/0 default/read/poll queues Dec 15 21:18:52 ljlbook kernel: PM: resume devices took 0.204 seconds Dec 15 21:18:52 ljlbook kernel: OOM killer enabled. Dec 15 21:18:52 ljlbook kernel: Restarting tasks ... done. Dec 15 21:18:52 ljlbook kernel: random: crng reseeded on system resumption Dec 15 21:18:52 ljlbook bluetoothd[1208]: Controller resume with wake event 0x0 Dec 15 21:18:52 ljlbook kernel: PM: suspend exit Dec 15 21:18:52 ljlbook kernel: PM: suspend entry (s2idle) Dec 15 21:18:52 ljlbook kernel: Filesystems sync: 0.015 seconds Dec 15 21:18:52 ljlbook tlp[4598]: Warning: systemd-rfkill.service is not masked, radio device switching may not work as configured. Dec 15 21:18:52 ljlbook tlp[4598]: >>> Invoke 'systemctl mask systemd-rfkill.service' to correct this. Dec 15 21:18:52 ljlbook tlp[4598]: Warning: systemd-rfkill.socket is not masked, radio device switching may not work as configured. Dec 15 21:18:53 ljlbook kernel: Freezing user space processes ... (elapsed 0.001 seconds) done. Dec 15 21:18:53 ljlbook kernel: OOM killer disabled. Dec 15 21:18:53 ljlbook kernel: Freezing remaining freezable tasks ... (elapsed 0.000 seconds) done. Dec 15 21:18:53 ljlbook kernel: printk: Suspending console(s) (use no_console_suspend to debug) Dec 15 21:18:53 ljlbook kernel: mtk_t7xx 0000:58:00.0: [PM] Exiting suspend, modem in invalid state Dec 15 21:18:53 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -14 Dec 15 21:18:53 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -14 Dec 15 21:18:53 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -14 Dec 15 21:18:53 ljlbook kernel: PM: Some devices failed to suspend, or early wake event detected Dec 15 21:18:53 ljlbook kernel: pcieport 10000:e0:1d.0: can't derive routing for PCI INT A Dec 15 21:18:53 ljlbook kernel: nvme 10000:e1:00.0: PCI INT A: no GSI Dec 15 21:18:53 ljlbook kernel: nvme nvme0: Shutdown timeout set to 10 seconds Dec 15 21:18:53 ljlbook kernel: nvme nvme0: 8/0/0 default/read/poll queues Dec 15 21:18:53 ljlbook kernel: PM: resume devices took 0.200 seconds Dec 15 21:18:53 ljlbook kernel: OOM killer enabled. Dec 15 21:18:53 ljlbook kernel: Restarting tasks ... done. Dec 15 21:18:53 ljlbook kernel: random: crng reseeded on system resumption Dec 15 21:18:52 ljlbook tlp[4598]: >>> Invoke 'systemctl mask systemd-rfkill.socket' to correct this. Dec 15 21:18:53 ljlbook systemd-sleep[4548]: Failed to put system to sleep. System resumed again: Bad address Dec 15 21:18:53 ljlbook bluetoothd[1208]: Controller resume with wake event 0x0 Dec 15 21:18:53 ljlbook kernel: PM: suspend exit [...] Dec 15 21:18:53 ljlbook systemd[1]: systemd-suspend.service: Main process exited, code=exited, status=1/FAILURE Dec 15 21:18:53 ljlbook systemd[1]: systemd-suspend.service: Failed with result 'exit-code'. Dec 15 21:18:53 ljlbook systemd[1]: Failed to start systemd-suspend.service - System Suspend. Dec 15 21:18:53 ljlbook systemd[1]: Dependency failed for suspend.target - Suspend. Dec 15 21:18:53 ljlbook systemd[1]: suspend.target: Job suspend.target/start failed with result 'dependency'. Dec 15 21:18:53 ljlbook audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=systemd-suspend comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed' Dec 15 21:18:53 ljlbook systemd-logind[984]: Operation 'sleep' finished. Dec 15 21:18:53 ljlbook systemd[1]: Stopped target sleep.target - Sleep. Dec 15 21:18:53 ljlbook NetworkManager[1217]: <info> [1671135533.3233] manager: sleep: wake requested (sleeping: yes enabled: yes) Dec 15 21:18:53 ljlbook ModemManager[2020]: <info> [sleep-monitor] system is resuming [...] Dec 15 21:18:54 ljlbook ModemManager[2020]: opening device... Dec 15 21:18:54 ljlbook ModemManager[2020]: [/dev/wwan0mbim0] Couldn't find descriptors file, possibly not using cdc_mbim Dec 15 21:18:54 ljlbook ModemManager[2020]: [/dev/wwan0mbim0] Fallback to default max control message size: 4096 Dec 15 21:18:54 ljlbook kernel: mtk_t7xx 0000:58:00.0: Write error on AT port, -19 Dec 15 21:18:54 ljlbook kernel: mtk_t7xx 0000:58:00.0: Write error on MBIM port, -19 Dec 15 21:18:54 ljlbook kernel: mtk_t7xx 0000:58:00.0: Write error on AT port, -19 Dec 15 21:18:54 ljlbook kernel: mtk_t7xx 0000:58:00.0: Write error on MBIM port, -19 Dec 15 21:18:55 ljlbook ModemManager[2020]: <info> [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:14.3': not supported by any plugin Dec 15 21:18:55 ljlbook ModemManager[2020]: <info> [base-manager] couldn't check support for device '/sys/devices/pci0000:00/0000:00:1f.6': not supported by any plugin
Sorry, I mistakenly wrote Fedora 33. Of course it's Fedora 37.
> Unfortunately, although the system appeared to suspend on the first try, logs still indicated misbehavior by the Fibocom modem, and on the second try, it failed to suspend entirely. The Fibocom was also no longer available to use. Ok thanks for trying. I believe it is best to just let upstream sort this out then, sorry.
It did succeed on the first try, in terms of suspending, it just left the modem in a bad state that prevented further suspending. That was S3 sleep, by the way. I rebooted (unsure if that reset the modem, maybe I should have powered off) and tried again with s2idle sleep, and failed: Dec 15 21:40:55 ljlbook systemd-logind[1056]: Power key pressed short. Dec 15 21:40:55 ljlbook ModemManager[1123]: <info> [sleep-monitor] system is about to suspend Dec 15 21:40:55 ljlbook NetworkManager[1228]: <info> [1671136855.7439] manager: sleep: sleep requested (sleeping: no enabled: yes) [...] Dec 15 21:40:56 ljlbook NetworkManager[1228]: <info> [1671136856.0961] manager: NetworkManager state is now ASLEEP [...] Dec 15 21:40:56 ljlbook systemd[1]: Reached target sleep.target - Sleep. Dec 15 21:40:56 ljlbook wpa_supplicant[1325]: p2p-dev-wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:40:56 ljlbook wpa_supplicant[1325]: p2p-dev-wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:40:56 ljlbook wpa_supplicant[1325]: nl80211: deinit ifname=p2p-dev-wlo1 disabled_11b_rates=0 Dec 15 21:40:56 ljlbook systemd[1]: Starting systemd-suspend.service - System Suspend... Dec 15 21:40:56 ljlbook wpa_supplicant[1325]: wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:40:56 ljlbook wpa_supplicant[1325]: wlo1: CTRL-EVENT-DSCP-POLICY clear_all Dec 15 21:40:56 ljlbook wpa_supplicant[1325]: nl80211: deinit ifname=wlo1 disabled_11b_rates=0 Dec 15 21:40:56 ljlbook systemd-sleep[5290]: Entering sleep state 'suspend'... Dec 15 21:40:56 ljlbook kernel: PM: suspend entry (s2idle) Dec 15 21:40:56 ljlbook kernel: Filesystems sync: 0.088 seconds Dec 15 21:40:57 ljlbook kernel: Freezing user space processes ... (elapsed 0.003 seconds) done. Dec 15 21:40:57 ljlbook kernel: OOM killer disabled. Dec 15 21:40:57 ljlbook kernel: Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. Dec 15 21:40:57 ljlbook kernel: printk: Suspending console(s) (use no_console_suspend to debug) Dec 15 21:40:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: [PM] Exiting suspend, modem in invalid state Dec 15 21:40:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -14 Dec 15 21:40:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -14 Dec 15 21:40:57 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -14 Dec 15 21:40:57 ljlbook kernel: PM: Some devices failed to suspend, or early wake event detected Dec 15 21:40:57 ljlbook kernel: PM: resume devices took 0.567 seconds Dec 15 21:40:57 ljlbook kernel: OOM killer enabled. Dec 15 21:40:57 ljlbook kernel: Restarting tasks ... done. Dec 15 21:40:57 ljlbook kernel: random: crng reseeded on system resumption Dec 15 21:40:57 ljlbook bluetoothd[1214]: Controller resume with wake event 0x0 Dec 15 21:40:57 ljlbook kernel: PM: suspend exit Dec 15 21:40:57 ljlbook kernel: PM: suspend entry (s2idle) Dec 15 21:40:57 ljlbook kernel: Filesystems sync: 0.017 seconds Dec 15 21:40:58 ljlbook kernel: Freezing user space processes ... (elapsed 0.003 seconds) done. Dec 15 21:40:58 ljlbook kernel: OOM killer disabled. Dec 15 21:40:58 ljlbook kernel: Freezing remaining freezable tasks ... (elapsed 0.063 seconds) done. Dec 15 21:40:58 ljlbook kernel: printk: Suspending console(s) (use no_console_suspend to debug) Dec 15 21:40:58 ljlbook kernel: mtk_t7xx 0000:58:00.0: [PM] Exiting suspend, modem in invalid state Dec 15 21:40:58 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -14 Dec 15 21:40:58 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -14 Dec 15 21:40:58 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -14 Dec 15 21:40:58 ljlbook kernel: PM: Some devices failed to suspend, or early wake event detected Dec 15 21:40:58 ljlbook kernel: PM: resume devices took 0.205 seconds Dec 15 21:40:58 ljlbook kernel: OOM killer enabled. Dec 15 21:40:58 ljlbook kernel: Restarting tasks ... done. Dec 15 21:40:58 ljlbook kernel: random: crng reseeded on system resumption Dec 15 21:40:58 ljlbook bluetoothd[1214]: Controller resume with wake event 0x0 Dec 15 21:40:58 ljlbook systemd-sleep[5290]: Failed to put system to sleep. System resumed again: Bad address Dec 15 21:40:58 ljlbook kernel: PM: suspend exit
(In reply to Hans de Goede from comment #7) > Ubuntu seems to have fixed this by cherry-picking: > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/ > ?id=d20ef656f9942950d67e29418512a6929682ad60 > > which was reverted upstream because apparently it was merged without proper > review and which has not shown up upstream again since :| > > I have started a test build of the Fedora kernel (6.1.0 based from rawhide) > with this patch added, this is currently building here: > https://koji.fedoraproject.org/koji/taskinfo?taskID=95395499 > > This should be done building in a couple of hours (once it is done the link > will show rpms to downloads). Please install this test kernel and let me > know if it fixes the suspend-resume issue. > > For some generic instructions for installing a kernel directly from koji > (the Fedora buildsystem) see here: > https://fedorapeople.org/~jwrdegoede/kernel-test-instructions.txt I...believe this is working for me. It appears to be suspending correctly. I no longer see any module errors (mtk_t7xx) in journalctl. Assuming this works and sticks, what should I expect or plan for in terms of future upgrades? I'm assuming someday I'll upgrade and get a new kernel version that may or may not have this fix in place. @ljlbox Just wondering, did you remove your workaround script you mentioned in /usr/lib/systemd/system-sleep/broadband-modem ? I backed up and copied that elsewhere before rebooting, perhaps that's the main difference. Or perhaps we have slightly different version numbers or something. journalctl log: 15:29:55 systemd[1]: Starting systemd-suspend.service - System Suspend... 15:29:55 systemd-sleep[6269]: Entering sleep state 'suspend'... 15:29:55 kernel: PM: suspend entry (s2idle) 15:29:55 kernel: Filesystems sync: 0.025 seconds # Suspended for about 1:30 15:31:19 kernel: Freezing user space processes ... (elapsed 0.001 seconds) done. 15:31:19 kernel: OOM killer disabled. 15:31:19 kernel: Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. 15:31:19 kernel: printk: Suspending console(s) (use no_console_suspend to debug) 15:31:19 kernel: PM: suspend devices took 1.554 seconds 15:31:19 kernel: ACPI: EC: interrupt blocked 15:31:19 kernel: ACPI: EC: interrupt unblocked 15:31:19 kernel: i915 0000:00:02.0: [drm] GuC firmware i915/adlp_guc_70.bin version 70.5.1 15:31:19 kernel: i915 0000:00:02.0: [drm] HuC firmware i915/tgl_huc.bin version 7.9.3 15:31:19 kernel: i915 0000:00:02.0: [drm] HuC authenticated 15:31:19 kernel: i915 0000:00:02.0: [drm] GuC submission enabled 15:31:19 kernel: i915 0000:00:02.0: [drm] GuC SLPC enabled 15:31:19 kernel: i915 0000:00:02.0: [drm] GuC RC: enabled 15:31:19 kernel: nvme nvme0: Shutdown timeout set to 10 seconds 15:31:19 kernel: nvme nvme0: 20/0/0 default/read/poll queues 15:31:19 kernel: PM: resume devices took 0.634 seconds I'll keep an eye on this and report back if anything changes.
kernel-6.4.15-200.fc38.x86_64 This kernel appears to behave exactly like described above. It causes the system to spontaneously attempt to suspend randomly every within 5-60 seconds but the driver prevents suspend from succeeding it comes back to the gdm lockscreen a second later. kernel-6.5.2-300.fc38.x86_64 The random spontaneous suspend attempts seem to be gone but intentional suspend attempts are still broken.
Bug seems to be still there even in latest Fedora kernel (Linux 6.5.11-300.fc39.x86_64) [22720.337247] printk: Suspending console(s) (use no_console_suspend to debug) [22721.968655] mtk_t7xx 0000:08:00.0: [PM] SAP suspend error: -110 [22721.968722] mtk_t7xx 0000:08:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -110 [22721.968794] mtk_t7xx 0000:08:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -110 [22721.968826] mtk_t7xx 0000:08:00.0: PM: failed to suspend async: error -110 [22721.969183] PM: Some devices failed to suspend, or early wake event detected It's a really annoying bug: The laptop must be always shut down or it gets hot and drains battery. The only workaround I know of is to disable modem in BIOS (and thus not to use wwan network anymore)
This message is a reminder that Fedora Linux 37 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '37'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 37 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
Per comment 14 this is still happening in F39, updating version.
With kernel 6.6.2-201.fc39.x86_64, just released in fedora upgrade repo, I was able to enter suspend state, and the SAP error went away from logs. I will test more in the coming days and report here
Unfortunately I can't report the same as Mario. I am now on 6.6.6-200.fc39.x86_64 and I still get dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM configuration timed out dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: [PM] Exiting suspend, modem in invalid state dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -14 dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -14 dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -14 dic 18 01:55:01 ljlbook kernel: PM: Some devices failed to suspend, or early wake event detected ... dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: [PM] Exiting suspend, modem in invalid state dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: pci_pm_suspend(): t7xx_pci_pm_suspend+0x0/0x20 [mtk_t7xx] returns -14 dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: dpm_run_callback(): pci_pm_suspend+0x0/0x170 returns -14 dic 18 01:55:01 ljlbook kernel: e1000e: EEE TX LPI TIMER: 00000011 dic 18 01:55:01 ljlbook kernel: mtk_t7xx 0000:58:00.0: PM: failed to suspend async: error -14 dic 18 01:55:01 ljlbook kernel: PM: Some devices failed to suspend, or early wake event detected dic 18 01:55:01 ljlbook kernel: PM: resume devices took 0.262 seconds and the suspend fails.