Description of problem: This kernel oops has happened when booting F30 since 5.1.9 and rawhide 5.3.0-rc0 to 5.3.0-rc1. abrt showed that this oops happened 131 times in five minutes with kernel-5.3.0-0.rc1.git3.1.fc31.x86_64 while booting and the ISP connection was not yet available according to the router. The oops usually happened once on most boots. The ethernet NIC is a Realtek RTL8168. lspci 01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15) ethtool -i enp1s0 driver: r8169 version: firmware-version: rtl8168h-2_0.0.2 02/26/15 expansion-rom-version: bus-info: 0000:01:00.0 supports-statistics: yes supports-test: no supports-eeprom-access: no supports-register-dump: yes supports-priv-flags: no Additional info: reporter: libreport-2.10.1 NETDEV WATCHDOG: enp1s0 (r8169): transmit queue 0 timed out WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:442 dev_watchdog+0x2ac/0x2e0 Modules linked in: rfcomm ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables bnep sunrpc vfat fat edac_mce_amd kvm_amd ccp kvm btusb btrtl uvcvideo iwlmvm btbcm mac80211 btintel irqbypass bluetooth crct10dif_pclmul snd_hda_codec_realtek videobuf2_vmalloc crc32_pclmul snd_hda_codec_generic libarc4 ledtrig_audio wdat_wdt snd_hda_codec_hdmi videobuf2_memops videobuf2_v4l2 iwlwifi snd_hda_intel videobuf2_common ghash_clmulni_intel snd_hda_codec videodev snd_hda_core snd_hwdep snd_seq hp_wmi ecdh_generic snd_seq_device joydev sparse_keymap mc ecc wmi_bmof fam15h_power k10temp cfg80211 snd_pcm snd_timer rfkill snd sp5100_tco soundcore i2c_piix4 i2c_scmi hp_wireless acpi_cpufreq amdgpu amd_iommu_v2 gpu_sched i2c_algo_bit crc32c_intel ttm serio_raw drm_kms_helper uas drm usb_storage r8169 wmi video hid_multitouch CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.3.0-0.rc1.git3.1.fc31.x86_64 #1 Hardware name: HP HP Laptop 15-bw0xx/8332, BIOS F.51 05/31/2019 RIP: 0010:dev_watchdog+0x2ac/0x2e0 Code: 6d e9 59 ff ff ff 4c 89 f7 c6 05 ef c9 f5 00 01 e8 99 79 fa ff 44 89 e9 4c 89 f6 48 c7 c7 b8 31 40 93 48 89 c2 e8 3c c9 75 ff <0f> 0b e9 3b ff ff ff e8 e8 46 7f ff 85 c0 75 c0 48 c7 c2 98 0d 30 RSP: 0018:ffffb9854007ce30 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff91e5aa858a00 RCX: 0000000000000006 RDX: 0000000000000007 RSI: ffff91e5b5fbc038 RDI: ffff91e5b6fd9d80 RBP: ffff91e5b54d44a0 R08: 000000f0d5b3da06 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff91e5b54d4508 R13: 0000000000000000 R14: ffff91e5b54d4000 R15: ffff91e5aa858a80 FS: 0000000000000000(0000) GS:ffff91e5b6e00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f5cddb36000 CR3: 00000001b6c5e000 CR4: 00000000001406e0 Call Trace: <IRQ> ? qdisc_put_unlocked+0x30/0x30 ? qdisc_put_unlocked+0x30/0x30 call_timer_fn+0xaf/0x2c0 run_timer_softirq+0x1b4/0x5e0 __do_softirq+0xe1/0x45d irq_exit+0xf7/0x100 smp_apic_timer_interrupt+0xa4/0x230 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:native_safe_halt+0xe/0x10 Code: 90 90 90 90 90 90 90 90 90 90 90 90 e9 07 00 00 00 0f 00 2d c6 77 4e 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d b6 77 4e 00 fb f4 <c3> 90 0f 1f 44 00 00 41 54 55 53 e8 82 7b 66 ff 65 8b 2d 7b 12 4f RSP: 0018:ffffb9854015fe10 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 RAX: ffff91e5b5fbb300 RBX: 0000000000000001 RCX: 0000000000000000 RDX: ffff91e5b5fbb300 RSI: 0000000000000006 RDI: ffff91e5b5fbb300 RBP: ffff91e5b4f23000 R08: 000000f0d5ad8390 R09: 0000000000000000 R10: 0000000000000000 R11: 0000000000000000 R12: ffff91e5b4f23064 R13: 0000000000000001 R14: 0000000000000001 R15: ffff91e5b5fbb300 acpi_safe_halt+0x20/0x30 acpi_idle_enter+0xf3/0x2b0 cpuidle_enter_state+0x7c/0x450 cpuidle_enter+0x29/0x40 do_idle+0x1ec/0x280 cpu_startup_entry+0x19/0x20 start_secondary+0x16f/0x1c0 secondary_startup_64+0xa4/0xb0 irq event stamp: 20649523 hardirqs last enabled at (20649522): [<ffffffff92170d9b>] console_unlock+0x46b/0x5d0 hardirqs last disabled at (20649523): [<ffffffff9200383a>] trace_hardirqs_off_thunk+0x1a/0x20 softirqs last enabled at (20649506): [<ffffffff920f1d28>] irq_enter+0x68/0x70 softirqs last disabled at (20649507): [<ffffffff920f1e27>] irq_exit+0xf7/0x100 Potential duplicate: bug 1575969
Created attachment 1594138 [details] File: dmesg
"This kernel oops has happened when booting F30 since 5.1.9 and rawhide 5.3.0-rc0 to 5.3.0-rc1." Means it didn't happen before 5.1.9? What was the last kernel version you used w/o this issue? "The oops usually happened once on most boots." It should never happen. Same as previous question, what was the last kernel version with this oops never happening? What doesn't look normal in the log: - Autoneg seems to take more time than normal (until link is shown as up at 9.85s) - Link breaks twice (at 15.51s and 19.74s) Does the same issue happen if you try with different cable and different link partner?
(In reply to Heiner Kallweit from comment #2) > "This kernel oops has happened when booting F30 since 5.1.9 and rawhide > 5.3.0-rc0 to 5.3.0-rc1." > Means it didn't happen before 5.1.9? What was the last kernel version you > used w/o this issue? > This oops at net/sched/sch_generic.c:442 appeared first with 5.3.0-rc0. The ethernet connection has stopped on a few boots while using 5.3.0 usually 5-15 minutes after booting. I wasn't able to connect to the router when that happened. The connection worked each time I rebooted, but not if I restarted the router alone. I was using an F30 KDE Plasma spin installation with 5.0.9 in the image which I did on June 6 to another usb flash drive. I updated to 5.1.7 after the installation was complete. A similar oops at net/sched/sch_generic.c:460 occurred 47 times from 5.1.9 on June 12 to 5.1.19 on July 22. The F30 oops had one qdisc_put_unlocked call at the top of the stack trace while the rawhide one with 5.3.0 had two qdisc_put_unlocked there. Jun 12 22:43:58 kernel: NETDEV WATCHDOG: enp1s0 (r8169): transmit queue 0 timed out Jun 12 22:43:58 kernel: WARNING: CPU: 3 PID: 2125 at net/sched/sch_generic.c:460 dev_watchdog+0x1e9/0x1f0 Jun 12 22:43:58 kernel: Modules linked in: ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables sunrpc vfat fat dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua edac_mce_amd snd_hda_codec_realtek snd_hda_codec_generic arc4 ledtrig_audio iwlmvm ccp snd_hda_codec_hdmi snd_hda_intel kvm mac80211 snd_hda_codec uvcvideo snd_hda_core videobuf2_vmalloc snd_hwdep videobuf2_memops btusb videobuf2_v4l2 iwlwifi snd_seq btrtl btbcm snd_seq_device videobuf2_common snd_pcm irqbypass btintel videodev bluetooth crct10dif_pclmul cfg80211 media crc32_pclmul ghash_clmulni_intel snd_timer wdat_wdt ecdh_generic hp_wmi snd sparse_keymap rfkill joydev wmi_bmof soundcore sp5100_tco fam15h_power k10temp i2c_piix4 i2c_scmi Jun 12 22:43:58 kernel: hp_wireless pcc_cpufreq acpi_cpufreq amdgpu i2c_algo_bit chash gpu_sched amd_iommu_v2 ttm crc32c_intel drm_kms_helper serio_raw uas drm usb_storage r8169 wmi video hid_multitouch Jun 12 22:43:58 kernel: CPU: 3 PID: 2125 Comm: Compositor Not tainted 5.1.9-300.fc30.x86_64 #1 Jun 12 22:43:58 kernel: Hardware name: HP HP Laptop 15-bw0xx/8332, BIOS F.10 05/11/2017 Jun 12 22:43:58 kernel: RIP: 0010:dev_watchdog+0x1e9/0x1f0 Jun 12 22:43:58 kernel: Code: 48 63 55 e0 eb 93 4c 89 ef c6 05 02 43 ad 00 01 e8 0c 72 fb ff 44 89 e1 4c 89 ee 48 c7 c7 b0 6f 18 be 48 89 c2 e8 d6 1d 87 ff <0f> 0b eb bf 0f 1f 00 0f 1f 44 00 00 48 c7 47 08 00 00 00 00 48 c7 Jun 12 22:43:58 kernel: RSP: 0000:ffff99dff7383e88 EFLAGS: 00010282 Jun 12 22:43:58 kernel: RAX: 0000000000000000 RBX: ffff99dfee37da00 RCX: 0000000000000006 Jun 12 22:43:58 kernel: RDX: 0000000000000007 RSI: 0000000000000086 RDI: ffff99dff73968c0 Jun 12 22:43:58 kernel: RBP: ffff99dff5f36480 R08: 0000000000000001 R09: 00000000000003fd Jun 12 22:43:58 kernel: R10: ffffffffbe9d3fa8 R11: 0000000000000003 R12: 0000000000000000 Jun 12 22:43:58 kernel: R13: ffff99dff5f36000 R14: 0000000000000000 R15: ffff99dff7383ed8 Jun 12 22:43:58 kernel: FS: 00007f43a2b0d700(0000) GS:ffff99dff7380000(0000) knlGS:0000000000000000 Jun 12 22:43:58 kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jun 12 22:43:58 kernel: CR2: 00007f18f00be000 CR3: 0000000195db0000 CR4: 00000000001406e0 Jun 12 22:43:58 kernel: Call Trace: Jun 12 22:43:58 kernel: <IRQ> Jun 12 22:43:58 kernel: ? qdisc_put_unlocked+0x30/0x30 Jun 12 22:43:58 kernel: call_timer_fn+0x2b/0x130 Jun 12 22:43:58 kernel: run_timer_softirq+0x3bd/0x440 Jun 12 22:43:58 kernel: ? timerqueue_add+0x56/0x90 Jun 12 22:43:58 kernel: ? enqueue_hrtimer+0x36/0x90 Jun 12 22:43:58 kernel: __do_softirq+0xed/0x30e Jun 12 22:43:58 kernel: irq_exit+0xf1/0x100 Jun 12 22:43:58 kernel: smp_apic_timer_interrupt+0x76/0x140 Jun 12 22:43:58 kernel: apic_timer_interrupt+0xf/0x20 Jun 12 22:43:58 kernel: </IRQ> Jun 12 22:43:58 kernel: RIP: 0033:0x7f43bb3fa628 Jun 12 22:43:58 kernel: Code: 5a d0 f3 0f 5c 54 24 20 0f 54 e2 f3 0f 5a e4 66 0f 2f ec 0f 86 ce 90 8f fd f2 44 0f 10 85 10 03 00 00 f2 0f 10 0d 30 f0 dd 00 <66> 0f 6f b4 24 b0 00 00 00 66 0f 6f ac 24 c0 00 00 00 66 44 0f 6f Jun 12 22:43:58 kernel: RSP: 002b:00007f43a2b0c0c0 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13 Jun 12 22:43:58 kernel: RAX: 0000000000000000 RBX: 00007f43a2b0c330 RCX: 00000000bdc83c00 Jun 12 22:43:58 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 Jun 12 22:43:58 kernel: RBP: 00007f4387897c00 R08: 0000000000000000 R09: 0000000000000001 Jun 12 22:43:58 kernel: R10: 0000000083cac300 R11: 0000000000000000 R12: 0000000000000000 Jun 12 22:43:58 kernel: R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000000 Jun 12 22:43:58 kernel: ---[ end trace a4fb700a1a70112c ]--- The problem probably happened before 5.1.9. The journal only goes back to June 11, so I don't know which kernel version didn't have the problem. Similar traces were reported since March at https://bugzilla.redhat.com/show_bug.cgi?id=1692075 where some reported the output of ethtool -k which for me was the following ethtool -k enp1s0 Features for enp1s0: rx-checksumming: on tx-checksumming: off tx-checksum-ipv4: off tx-checksum-ip-generic: off [fixed] tx-checksum-ipv6: off tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: off tx-scatter-gather: off tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: off tx-tcp-segmentation: off tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off tx-tcp6-segmentation: off generic-segmentation-offload: off [requested on] generic-receive-offload: on large-receive-offload: off [fixed] rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off [fixed] receive-hashing: off [fixed] highdma: on [fixed] rx-vlan-filter: off [fixed] vlan-challenged: off [fixed] tx-lockless: off [fixed] netns-local: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] tx-gre-segmentation: off [fixed] tx-gre-csum-segmentation: off [fixed] tx-ipxip4-segmentation: off [fixed] tx-ipxip6-segmentation: off [fixed] tx-udp_tnl-segmentation: off [fixed] tx-udp_tnl-csum-segmentation: off [fixed] tx-gso-partial: off [fixed] tx-sctp-segmentation: off [fixed] tx-esp-segmentation: off [fixed] tx-udp-segmentation: off [fixed] fcoe-mtu: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off rx-all: off tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off [fixed] esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: off [fixed] tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] > "The oops usually happened once on most boots." > It should never happen. Same as previous question, what was the last kernel > version with this oops never happening? > I should clarify that the oopses usually happened only once during most boots in which they occurred in F30 from 5.1.9-5.1.19. The journal lists 364 boots during that time. > What doesn't look normal in the log: > - Autoneg seems to take more time than normal (until link is shown as up at > 9.85s) > - Link breaks twice (at 15.51s and 19.74s) > I think that I plugged in the router a few seconds after booting the computer, so the router was probably booting for that period of 10-20 seconds. After that the router showed an error indicating that the connection to the ISPs servers couldn't be made as has happened frequently before, so I restarted the router. > Does the same issue happen if you try with different cable and different > link partner? I've tried connecting with a different cable, and another oops with the same trace happened about 10 minutes afterwards. I'll try to connect to another router by ethernet and connect that to the router connected to the ISP. Thanks.
What we call oops here actually is a very generic tx timeout and it can have a thousand reasons, even dependent on the respective RTL8168 chip version. I have a test system with the same chip version as your system, and it does not have this problem. Instead of using another router you could also try to place a switch between your router and your system. To check whether it's a regression you can test whether LTS kernel versions 4.14 and 4.19 are fine.
In addition you could check whether the issue occurs with the r8168 vendor driver too. Most distributions offer it as optional package, or you can compile it from the sources provided by Realtek.
Heiner, I've often seen that when the Bell Home Hub 3000 / Sagemcom Fast5566 router from the ISP which I've been connecting to with the HP laptop with the RTL8168 stopped responding, there were NetworkManager DHCP request timeouts in the journal. Those DHCP timeouts might be involved in some of the oopses I've seen. I connected an Asus RT-N56U to the laptop by ethernet, and then I used the RT-N56U's ethernet switch to connect to the Bell Home Hub 3000. I put the latest stable OpenWrt 18.0.4 running the 4.14.131 kernel on the Asus RT-N56U https://openwrt.org/releases/18.06/notes-18.06.4 The Bell Home Hub 3000 seems to be running a custom OpenWrt build made by Sagemcom with a 3.4.11-rt19 kernel from 2012, and its latest firmware update was done in April 2019. The internet connection worked fine for the most part. I saw the same oops at sch_generic.c:442 after about 50 minutes during the fifth boot with this network configuration. I wasn't using the system at the time the timeout happened, and the journal doesn't show anything else then but the same timeout. I think that the Bell Home Hub 3000 has stopped responding at times which might lead to the timeouts partly due to its outdated firmware. I haven't found a r8168 driver in the Fedora repositories. Do you happen to know if there's an rpm for Fedora with the r8168? The r8168 driver at https://www.realtek.com/en/component/zoo/category/network-interface-controllers-10-100-1000m-gigabit-ethernet-pci-express-software has "GBE Ethernet LINUX driver r8168 for kernel up to 4.15" so it might not be applicable to the current Fedora kernels. Thanks.
The r8168 sources provided by Realtek can be compiled also on kernel versions newer than 4.15.
(In reply to Heiner Kallweit from comment #7) > The r8168 sources provided by Realtek can be compiled also on kernel > versions newer than 4.15. I'm unsure if the r8168 driver from Realtek's site is intended for use with the 5.3 branch. I haven't compiled a kernel module before. I've continued to see these transmit queue timeouts up to and including 5.3.0-0.rc4.git0. The ethernet connection has often stopped at times when the timeout wasn't shown in the journal. These connection lockups often happen after downloading more than about 150 MB of a file. I've found that running removing r8169 and adding it back restored the connection in each of 17 times that I tried it using sudo modprobe -r r8169 sudo modprobe r8169 The r8169 driver seemed to lock up when transferring data and remained in that state until I removed and restarted it or rebooted. The RTL8168 card and the Bell Home Hub 3000 router switch speeds are up to 1 Gbit/s, so I think they should be able to support the up to 100 Mbit/s download speed of the VDSL2 with pair bonding WAN connection. Thanks.
Also you could try disabling EEE (using ethtool).
Description of problem: These time outs of the transmit queue happen when making internet connections with a Realtek RTL8168 ethernet NIC. The r8169 driver appears to be getting stuck in the qdisc_put_unlocked function, and so the transmit queue remains in a locked state. The connection can't send any more data afterwards. Removing and adding the r8169 driver restores the connection using sudo modprobe -r r8169 sudo modprobe r8169 abrt showed time outs with this trace happened 18 times between 5.3.0-rc2.git0 and 5.3.0-rc5.git0, though time outs like them happened since 5.1.9 and probably before that. I reported transmit queue time outs in qdisc_put_unlocked with similar traces using r8169 at https://bugzilla.redhat.com/show_bug.cgi?id=1733837 I ran ethtool --show-eee enp1s0 which showed "EEE status: enabled - active". I shut off EEE a few times using sudo ethtool --set-eee enp1s0 eee off as suggested at https://bugzilla.redhat.com/show_bug.cgi?id=1733837#c9 These connection problems still happened with EEE off. Version-Release number of selected component: kernel-core-5.3.0-0.rc2.git0.1.fc31 Additional info: reporter: libreport-2.10.1 cmdline: BOOT_IMAGE=(hd1,gpt2)/vmlinuz-5.3.0-0.rc2.git0.1.fc31.x86_64 root=/dev/mapper/fedora_localhost--live-root ro resume=/dev/mapper/fedora_localhost--live-swap rd.lvm.lv=fedora_localhost-live/root rd.lvm.lv=fedora_localhost-live/swap rhgb quiet crash_function: qdisc_put_unlocked kernel: 5.3.0-0.rc2.git0.1.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: NETDEV WATCHDOG: enp1s0 (r8169): transmit queue 0 timed out WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:442 dev_watchdog+0x22d/0x240 Modules linked in: usblp rfcomm ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables bnep sunrpc vfat fat uvcvideo videobuf2_vmalloc snd_hda_codec_realtek videobuf2_memops videobuf2_v4l2 snd_hda_codec_generic iwlmvm ledtrig_audio videobuf2_common snd_hda_codec_hdmi mac80211 snd_hda_intel videodev snd_hda_codec edac_mce_amd snd_hda_core mc kvm_amd btusb libarc4 snd_hwdep ccp btrtl snd_seq btbcm kvm btintel bluetooth iwlwifi irqbypass snd_seq_device snd_pcm cfg80211 ecdh_generic crct10dif_pclmul ecc crc32_pclmul hp_wmi snd_timer sparse_keymap rfkill wdat_wdt ghash_clmulni_intel sp5100_tco snd soundcore joydev k10temp fam15h_power i2c_piix4 wmi_bmof hp_wireless i2c_scmi acpi_cpufreq amdgpu amd_iommu_v2 gpu_sched i2c_algo_bit ttm drm_kms_helper crc32c_intel drm serio_raw uas usb_storage r8169 wmi video hid_multitouch CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.3.0-0.rc2.git0.1.fc31.x86_64 #1 Hardware name: HP HP Laptop 15-bw0xx/8332, BIOS F.51 05/31/2019 RIP: 0010:dev_watchdog+0x22d/0x240 Code: 85 c0 75 e5 eb 9f 4c 89 ef c6 05 08 b5 ca 00 01 e8 78 25 fb ff 44 89 e1 4c 89 ee 48 c7 c7 e0 5c 1b 83 48 89 c2 e8 31 b3 84 ff <0f> 0b eb 80 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 RSP: 0018:ffffa99f8007ce78 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff8e1b2d589200 RCX: 0000000000000000 RDX: 0000000000040400 RSI: 00000000000000f6 RDI: 0000000000000300 RBP: ffff8e1b335e8480 R08: ffff8e1b37297908 R09: 0000000000000462 R10: 00000000000183a4 R11: 0000000000000003 R12: 0000000000000000 R13: ffff8e1b335e8000 R14: ffff8e1b335e8480 R15: ffffffff83405100 FS: 0000000000000000(0000) GS:ffff8e1b37280000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f85f50c9000 CR3: 00000001b67be000 CR4: 00000000001406e0 Call Trace: <IRQ> ? qdisc_put_unlocked+0x30/0x30 call_timer_fn+0x2d/0x130 run_timer_softirq+0x17d/0x4a0 ? tick_sched_handle+0x22/0x60 ? tick_sched_timer+0x38/0x80 ? tick_sched_do_timer+0x70/0x70 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:cpuidle_enter_state+0xc4/0x420 Code: e8 c1 fd 92 ff 80 7c 24 0f 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 39 03 00 00 31 ff e8 e3 ec 98 ff fb 66 0f 1f 44 00 00 <45> 85 e4 0f 89 cf 01 00 00 c7 45 10 00 00 00 00 48 83 c4 18 44 89 RSP: 0018:ffffa99f8015fe68 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: ffff8e1b372a9480 RBX: ffffffff834dc020 RCX: 000000000000001f RDX: 0000000000000000 RSI: 00000000334bcdc2 RDI: 0000000000000000 RBP: ffff8e1b2e61ac00 R08: 00000d7bc6411275 R09: 0000000000000e90 R10: ffff8e1b372a82c4 R11: ffff8e1b372a82a4 R12: 0000000000000002 R13: 00000d7bc6411275 R14: 0000000000000002 R15: ffff8e1b36385dc0 ? cpuidle_enter_state+0x9f/0x420 cpuidle_enter+0x29/0x40 do_idle+0x1dd/0x260 cpu_startup_entry+0x19/0x20 start_secondary+0x163/0x1b0 secondary_startup_64+0xa4/0xb0
This error is a completely generic timeout, and the root cause doesn't have to be in the network driver. What you could do is check older kernel versions and bisect from the last known good kernel version.
(In reply to Heiner Kallweit from comment #11) > This error is a completely generic timeout, and the root cause doesn't have > to be in the network driver. What you could do is check older kernel > versions and bisect from the last known good kernel version. I tried to reproduce these connection lockups in 5.0.0-1.fc31 and 5.1.0-1.fc31 for about 30 minutes each, but I didn't see them. The connection errors often take many hours of normal usage to appear so I'm unsure if 5.0.0-1.fc31 and 5.1.0-1.fc31 aren't affected by them. The Fedora documentation on bisecting the kernel noted that it can be a time-consuming task https://docs.fedoraproject.org/en-US/quick-docs/kernel/troubleshooting/index.html#_bisecting_the_kernel Finding which kernels are unaffected by this problem could be time-consuming also. I observed the connection problems in 5.2.0-1.fc31, 5.2.0-0.rc1.git0.1.fc31, 5.2.0-0.rc0.git6.1.fc31, 5.1.9-1.fc30 to 5.1.19-1.fc30, and 5.3.0-0.rc0.git3.fc31 to 5.3.0-0.rc6.git9.fc31. Similar reports with qdisc_put_unlocked at the top of the trace stack included those with other ethernet drivers e1000 "Intel(R) PRO/1000 Network Driver" https://bugzilla.redhat.com/show_bug.cgi?id=1692075#c9 r8152 "Realtek RTL8152/RTL8153 Based USB Ethernet Adapters" https://bugzilla.redhat.com/show_bug.cgi?id=1692075#c82 atl1c "Qualcomm Atheros 100/1000M Ethernet Network Driver" https://bugzilla.redhat.com/show_bug.cgi?id=1692075#c83 I guess that this error isn't specific to r8169, but it might be related to ethernet. I haven't seen these connection problems when using Wifi on the same system with the Intel Wireless-AC 3168NGW and iwlwifi driver, though other unrelated errors have appeared as I reported at https://bugzilla.redhat.com/show_bug.cgi?id=1733723 I decompressed 5.3.0-rc6 using sudo /usr/src/kernels/5.3.0-0.rc6.git0.1.fc31.x86_64/scripts/extract-vmlinux vmlinuz-5.3.0-0.rc6.git0.1.fc31.x86_64 > vmlinux-5.3.0-0.rc6.git0.1.fc31.x86_64 I ran gdb vmlinux-5.3.0-0.rc6.git0.1.fc31.x86_64 based on documentation at https://www.kernel.org/doc/html/latest/admin-guide/bug-hunting.html Since the traces all have qdisc_put_unlocked+0x30/0x30 at the top of the stack, I ran (gdb) set listsize 15 (gdb) l *qdisc_put_unlocked+0x30/0x30 0xffffffff8189a191 is in qdisc_put_unlocked (net/sched/sch_generic.c:990). 983 984 /* Version of qdisc_put() that is called with rtnl mutex unlocked. 985 * Intended to be used as optimization, this function only takes rtnl lock if 986 * qdisc reference counter reached zero. 987 */ 988 989 void qdisc_put_unlocked(struct Qdisc *qdisc) 990 { 991 if (qdisc->flags & TCQ_F_BUILTIN || 992 !refcount_dec_and_rtnl_lock(&qdisc->refcnt)) 993 return; 994 995 qdisc_destroy(qdisc); 996 rtnl_unlock(); 997 } If the timeouts happened when qdisc_put_unlocked got stuck at its start, then qdisc_put_unlocked wouldn't return or reach rtnl_unlock() which I suppose unlocks something network related. The connection might've gotten locked up due to that problem which could've been a deadlock. The connection not being able to send any new data until the r8169 driver or connection were restarted might be consistent with that situation. I resolved the connection lockup once by disconnecting and connecting the ethernet connection in the NetworkManager Plasma applet, though that hasn't worked at other times. Thanks.
*** Bug 1760641 has been marked as a duplicate of this bug. ***
Description of problem: It appears an issue related to the wireless driver rtw88 Version-Release number of selected component: kernel-core-5.3.5-300.fc31 Additional info: reporter: libreport-2.11.0 cmdline: BOOT_IMAGE=(hd0,gpt7)/vmlinuz-5.3.5-300.fc31.x86_64 root=UUID=0072cabc-a086-496a-ab8e-7ceb802e02ee ro resume=UUID=28862a1c-b043-43d0-b4a8-42e3648afd4e rhgb quiet psmouse.synaptics_intertouch=1 crash_function: call_timer_fn kernel: 5.3.5-300.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: WARNING: CPU: 1 PID: 2040 at drivers/net/wireless/realtek/rtw88/tx.c:156 rtw_tx_report_purge_timer+0x20/0x50 [rtw88] Modules linked in: uinput rfcomm xt_CHECKSUM xt_MASQUERADE nf_nat_tftp nf_conntrack_tftp tun bridge stp llc nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cmac bnep sunrpc vfat fat edac_mce_amd kvm_amd rtwpci ccp rtw88 kvm snd_hda_codec_realtek snd_hda_codec_generic irqbypass ledtrig_audio snd_hda_codec_hdmi uvcvideo mac80211 snd_hda_intel videobuf2_vmalloc crct10dif_pclmul snd_hda_codec crc32_pclmul videobuf2_memops btusb videobuf2_v4l2 btrtl snd_hda_core btbcm videobuf2_common snd_hwdep ghash_clmulni_intel snd_seq btintel videodev bluetooth hp_wmi snd_seq_device joydev mc snd_pcm wmi_bmof sparse_keymap cfg80211 ecdh_generic snd_timer k10temp ecc sp5100_tco snd i2c_piix4 rfkill rtsx_pci_ms ucsi_acpi libarc4 memstick soundcore typec_ucsi typec hp_accel i2c_scmi lis3lv02d input_polldev hp_wireless acpi_cpufreq lz4 lz4_compress zram ip_tables amdgpu rtsx_pci_sdmmc mmc_core hid_multitouch amd_iommu_v2 gpu_sched i2c_algo_bit ttm drm_kms_helper crc32c_intel drm serio_raw rtsx_pci wmi video pinctrl_amd i2c_hid fuse CPU: 1 PID: 2040 Comm: gdbus Not tainted 5.3.5-300.fc31.x86_64 #1 Hardware name: HP HP ENVY x360 Convertible 15-cp0xxx/8497, BIOS F.42 05/13/2019 RIP: 0010:rtw_tx_report_purge_timer+0x20/0x50 [rtw88] Code: 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 00 00 8b 47 f0 85 c0 75 01 c3 41 54 55 53 48 89 fb 48 c7 c7 d0 15 ef c0 e8 be 83 24 e0 <0f> 0b 4c 8d 63 d8 4c 89 e7 e8 b2 d2 b4 e0 48 8d 7b e0 48 89 c5 e8 RSP: 0000:ffffb0e7c003ee80 EFLAGS: 00010282 RAX: 0000000000000000 RBX: ffff9844c8d5d5a8 RCX: 0000000000000006 RDX: 0000000000000007 RSI: 0000000000000086 RDI: ffff9844cfc57900 RBP: 0000000000000000 R08: 0000000000000001 R09: 00000000000004b9 R10: 0000000000019c78 R11: 0000000000000003 R12: ffff9844c8d5d5a8 R13: ffffffffc0e972f0 R14: ffff9844c8d5d5a8 R15: ffffffffa2405100 FS: 00007f260632c700(0000) GS:ffff9844cfc40000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f25fc0ac028 CR3: 00000003ae2c0000 CR4: 00000000003406e0 Call Trace: <IRQ> call_timer_fn+0x2d/0x130 run_timer_softirq+0x17d/0x4a0 ? timerqueue_add+0x56/0x90 ? enqueue_hrtimer+0x36/0x90 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0033:0x7f26150e9461 Code: d2 45 31 e4 48 85 ff 74 40 42 0f b6 04 27 49 83 c4 01 3c 61 74 f3 3c 6d 74 ef 3c 28 74 1b 3c 7b 74 17 3c 29 74 1b 3c 7d 74 17 <85> d2 75 db 4c 89 e0 41 5c c3 0f 1f 44 00 00 83 c2 01 eb ec 0f 1f RSP: 002b:00007f260632b760 EFLAGS: 00000297 ORIG_RAX: ffffffffffffff13 RAX: 0000000000000073 RBX: 00007f260632b7d0 RCX: 0000000000000000 RDX: 0000000000000002 RSI: 00007f25fc0a9ee0 RDI: 00007f25fc0a9ee0 RBP: 00007f25fc0a9ee0 R08: 00007f260632b9f0 R09: 000000000000000a R10: 00007f25fc07e500 R11: 0000000000000000 R12: 0000000000000005 R13: 00007f25fc083b90 R14: 00007f260632b8c0 R15: 00007f25fc0a9ee0
Description of problem: disconnecting f31-kde-beta-rc1_9 resulted into this problem many times Version-Release number of selected component: kernel-core-5.3.7-301.fc31 Additional info: reporter: libreport-2.10.1 cmdline: BOOT_IMAGE=(hd0,gpt2)/vmlinuz-5.3.7-301.fc31.x86_64 root=/dev/mapper/EncFS-root ro rd.lvm.lv=EncFS/root rd.luks.uuid=luks-9ba2a1f2-1e12-4917-a758-879cbef15f35 rhgb quiet crash_function: call_timer_fn kernel: 5.3.7-301.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: WARNING: CPU: 0 PID: 0 at drivers/net/wireless/realtek/rtw88/tx.c:156 rtw_tx_report_purge_timer+0x20/0x50 [rtw88] Modules linked in: tun ccm rfcomm ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cmac bnep sunrpc vfat fat intel_rapl_msr intel_rapl_common snd_hda_codec_hdmi snd_hda_codec_realtek x86_pkg_temp_thermal snd_hda_codec_generic intel_powerclamp coretemp ledtrig_audio snd_hda_intel kvm_intel snd_hda_codec iTCO_wdt iTCO_vendor_support mei_hdcp snd_hda_core kvm pktcdvd rtwpci rtw88 snd_hwdep snd_seq irqbypass snd_seq_device intel_cstate btusb eeepc_wmi snd_pcm asus_wmi btrtl intel_uncore btbcm mac80211 intel_rapl_perf sparse_keymap wmi_bmof snd_timer btintel snd mxm_wmi bluetooth cfg80211 joydev soundcore i2c_i801 ecdh_generic ecc rfkill mei_me libarc4 mei ie31200_edac acpi_pad fuse ip_tables chacha_x86_64 chacha_generic poly1305_x86_64 poly1305_generic chacha20poly1305 dm_crypt dm_integrity async_xor xor async_tx i915 i2c_algo_bit crct10dif_pclmul crc32_pclmul drm_kms_helper crc32c_intel drm e1000e ghash_clmulni_intel nvme nvme_core wmi video CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.3.7-301.fc31.x86_64 #1 Hardware name: System manufacturer System Product Name/ROG STRIX Z370-E GAMING, BIOS 2201 05/27/2019 RIP: 0010:rtw_tx_report_purge_timer+0x20/0x50 [rtw88] Code: 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 00 00 8b 47 f0 85 c0 75 01 c3 41 54 55 53 48 89 fb 48 c7 c7 d0 25 ac c0 e8 5e 73 67 ed <0f> 0b 4c 8d 63 d8 4c 89 e7 e8 b2 d2 f7 ed 48 8d 7b e0 48 89 c5 e8 RSP: 0018:ffffb8d640003e80 EFLAGS: 00010282 RAX: 0000000000000000 RBX: ffff9476221c55a8 RCX: 0000000000000006 RDX: 0000000000000007 RSI: 0000000000000086 RDI: ffff947626817900 RBP: 0000000000000000 R08: 0000000000000001 R09: 00000000000004f7 R10: 000000000001af04 R11: 0000000000000003 R12: ffff9476221c55a8 R13: ffffffffc0a682f0 R14: ffff9476221c55a8 R15: ffffffffaf405100 FS: 0000000000000000(0000) GS:ffff947626800000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000177f9b07e1e0 CR3: 000000028040a002 CR4: 00000000003606f0 Call Trace: <IRQ> call_timer_fn+0x2d/0x130 run_timer_softirq+0x180/0x480 ? timerqueue_add+0x56/0x90 ? enqueue_hrtimer+0x36/0x90 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:cpuidle_enter_state+0xc4/0x420 Code: e8 a1 ce 92 ff 80 7c 24 0f 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 39 03 00 00 31 ff e8 03 bd 98 ff fb 66 0f 1f 44 00 00 <45> 85 e4 0f 89 cf 01 00 00 c7 45 10 00 00 00 00 48 83 c4 18 44 89 RSP: 0018:ffffffffaf403e58 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: ffff947626828880 RBX: ffffffffaf4d6520 RCX: 000000000000001f RDX: 0000000000000000 RSI: 0000000022a1cda2 RDI: 0000000000000000 RBP: ffff947626832e00 R08: 00000c623e3c4a7a R09: 000000000000172b R10: ffff9476268276c4 R11: ffff9476268276a4 R12: 0000000000000003 R13: 00000c623e3c4a7a R14: 0000000000000003 R15: ffffffffaf413780 ? cpuidle_enter_state+0x9f/0x420 cpuidle_enter+0x29/0x40 do_idle+0x1cf/0x250 cpu_startup_entry+0x19/0x20 start_kernel+0x54f/0x56c secondary_startup_64+0xa4/0xb0
*** Bug 1767910 has been marked as a duplicate of this bug. ***
*** Bug 1768014 has been marked as a duplicate of this bug. ***
Description of problem: Log in session Version-Release number of selected component: kernel-core-5.3.7-301.fc31 Additional info: reporter: libreport-2.10.1 cmdline: BOOT_IMAGE=(hd0,gpt7)/vmlinuz-5.3.7-301.fc31.x86_64 root=UUID=0072cabc-a086-496a-ab8e-7ceb802e02ee ro resume=UUID=28862a1c-b043-43d0-b4a8-42e3648afd4e rhgb quiet psmouse.synaptics_intertouch=1 crash_function: call_timer_fn kernel: 5.3.7-301.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: WARNING: CPU: 0 PID: 0 at drivers/net/wireless/realtek/rtw88/tx.c:156 rtw_tx_report_purge_timer+0x20/0x50 [rtw88] Modules linked in: ccm uinput rfcomm xt_CHECKSUM xt_MASQUERADE nf_nat_tftp nf_conntrack_tftp tun bridge stp llc nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cmac bnep sunrpc vfat fat edac_mce_amd kvm_amd snd_hda_codec_realtek snd_hda_codec_hdmi ccp snd_hda_codec_generic rtwpci ledtrig_audio rtw88 kvm snd_hda_intel uvcvideo irqbypass snd_hda_codec videobuf2_vmalloc mac80211 snd_hda_core btusb crct10dif_pclmul videobuf2_memops btrtl videobuf2_v4l2 crc32_pclmul btbcm snd_hwdep btintel videobuf2_common snd_seq ghash_clmulni_intel bluetooth snd_seq_device videodev cfg80211 snd_pcm mc hp_wmi ecdh_generic ecc k10temp sparse_keymap wmi_bmof sp5100_tco snd_timer joydev rtsx_pci_ms i2c_piix4 rfkill snd memstick libarc4 ucsi_acpi typec_ucsi soundcore typec hp_accel i2c_scmi lis3lv02d hp_wireless input_polldev acpi_cpufreq lz4 lz4_compress zram ip_tables amdgpu rtsx_pci_sdmmc mmc_core hid_multitouch amd_iommu_v2 gpu_sched i2c_algo_bit ttm drm_kms_helper crc32c_intel drm serio_raw rtsx_pci wmi video i2c_hid pinctrl_amd fuse CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.3.7-301.fc31.x86_64 #1 Hardware name: HP HP ENVY x360 Convertible 15-cp0xxx/8497, BIOS F.42 05/13/2019 RIP: 0010:rtw_tx_report_purge_timer+0x20/0x50 [rtw88] Code: 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 00 00 8b 47 f0 85 c0 75 01 c3 41 54 55 53 48 89 fb 48 c7 c7 d0 85 ee c0 e8 5e 13 25 e3 <0f> 0b 4c 8d 63 d8 4c 89 e7 e8 b2 72 b5 e3 48 8d 7b e0 48 89 c5 e8 RSP: 0018:ffffb487c0003e80 EFLAGS: 00010282 RAX: 0000000000000000 RBX: ffff950bb733d5a8 RCX: 0000000000000000 RDX: 0000000000040400 RSI: 00000000000000f6 RDI: 0000000000000300 RBP: 0000000000000000 R08: ffff950bcfc17908 R09: 000000000000057c R10: 000000000001d404 R11: 0000000000000003 R12: ffff950bb733d5a8 R13: ffffffffc0e8e2f0 R14: ffff950bb733d5a8 R15: ffffffffa5405100 FS: 0000000000000000(0000) GS:ffff950bcfc00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007fac0df4e010 CR3: 000000011a1da000 CR4: 00000000003406f0 Call Trace: <IRQ> call_timer_fn+0x2d/0x130 run_timer_softirq+0x180/0x480 ? tick_sched_handle+0x22/0x60 ? tick_sched_timer+0x38/0x80 ? tick_sched_do_timer+0x70/0x70 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:cpuidle_enter_state+0xc4/0x420 Code: e8 a1 ce 92 ff 80 7c 24 0f 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 39 03 00 00 31 ff e8 03 bd 98 ff fb 66 0f 1f 44 00 00 <45> 85 e4 0f 89 cf 01 00 00 c7 45 10 00 00 00 00 48 83 c4 18 44 89 RSP: 0018:ffffffffa5403e58 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: ffff950bcfc28880 RBX: ffffffffa54dc080 RCX: 000000000000001f RDX: 0000000000000000 RSI: 000000004010d7fa RDI: 0000000000000000 RBP: ffff950bc6113c00 R08: 000017dededdaa31 R09: 0000000000004f15 R10: ffff950bcfc276c4 R11: ffff950bcfc276a4 R12: 0000000000000002 R13: 000017dededdaa31 R14: 0000000000000002 R15: ffffffffa5413780 ? cpuidle_enter_state+0x9f/0x420 cpuidle_enter+0x29/0x40 do_idle+0x1cf/0x250 cpu_startup_entry+0x19/0x20 start_kernel+0x54f/0x56c secondary_startup_64+0xa4/0xb0
*** Bug 1768283 has been marked as a duplicate of this bug. ***
Description of problem: appeared during a burst of high network traffic. Version-Release number of selected component: kernel-core-5.3.8-300.fc31 Additional info: reporter: libreport-2.10.1 cmdline: BOOT_IMAGE=(hd1,gpt2)/vmlinuz-5.3.8-300.fc31.x86_64 root=/dev/mapper/fedora-root ro resume=/dev/mapper/fedora-swap rd.lvm.lv=fedora/root rd.luks.uuid=luks-a99fc9de-32dd-43f7-9133-699710a861ef rd.lvm.lv=fedora/swap rhgb quiet crash_function: qdisc_put_unlocked kernel: 5.3.8-300.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: NETDEV WATCHDOG: eno2 (e1000e): transmit queue 0 timed out WARNING: CPU: 8 PID: 0 at net/sched/sch_generic.c:447 dev_watchdog+0x22d/0x240 Modules linked in: uinput rfcomm xt_CHECKSUM nf_nat_tftp nf_conntrack_tftp tun bridge stp llc xt_MASQUERADE nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cmac bnep lm75 sunrpc vfat fat sof_pci_dev snd_sof_intel_hda_common snd_sof_intel_hda snd_sof_intel_byt snd_sof_intel_ipc snd_sof snd_sof_xtensa_dsp snd_soc_skl snd_hda_codec_hdmi snd_soc_hdac_hda snd_hda_ext_core snd_soc_skl_ipc snd_soc_sst_ipc snd_soc_sst_dsp snd_soc_acpi_intel_match snd_soc_acpi snd_soc_core snd_hda_codec_realtek intel_rapl_msr snd_compress raid0 snd_hda_codec_generic iwlmvm intel_rapl_common ac97_bus ledtrig_audio snd_pcm_dmaengine snd_hda_intel iTCO_wdt mac80211 iTCO_vendor_support mei_hdcp mei_wdt snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core coretemp snd_hwdep libarc4 kvm_intel snd_seq snd_seq_device kvm btusb iwlwifi snd_pcm btrtl irqbypass btbcm btintel intel_cstate snd_timer snd intel_uncore rtsx_usb_ms bluetooth intel_wmi_thunderbolt wmi_bmof i2c_i801 intel_rapl_perf soundcore cfg80211 pl2303 memstick ecdh_generic idma64 ecc mei_me intel_lpss_pci rfkill mei intel_lpss ie31200_edac acpi_pad acpi_tad ip_tables dm_crypt mmc_block rtsx_usb_sdmmc mmc_core rtsx_usb mcs7830 i915 i2c_algo_bit drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel drm e1000e nvme ghash_clmulni_intel nvme_core wmi video ax88179_178a usbnet mii fuse i2c_dev CPU: 8 PID: 0 Comm: swapper/8 Not tainted 5.3.8-300.fc31.x86_64 #1 Hardware name: LENOVO 30C6S5K700/3138, BIOS M1VKT48A 09/09/2019 RIP: 0010:dev_watchdog+0x22d/0x240 Code: 85 c0 75 e5 eb 9f 4c 89 ef c6 05 fa 35 ca 00 01 e8 18 25 fb ff 44 89 e1 4c 89 ee 48 c7 c7 48 ac 1f 8c 48 89 c2 e8 01 34 84 ff <0f> 0b eb 80 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 RSP: 0018:ffffb666002e0e78 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff8ac290ac6400 RCX: 000000000000083f RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 000000000000083f RBP: ffff8ac28d624480 R08: ffff8ac29c217908 R09: 00000000000004d9 R10: 000000000001c218 R11: 0000000000000003 R12: 0000000000000000 R13: ffff8ac28d624000 R14: ffff8ac28d624480 R15: ffffffff8c405100 FS: 0000000000000000(0000) GS:ffff8ac29c200000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f77bdfe7b38 CR3: 000000054f40a005 CR4: 00000000003606e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Call Trace: <IRQ> ? qdisc_put_unlocked+0x30/0x30 call_timer_fn+0x2d/0x130 run_timer_softirq+0x180/0x480 ? tick_sched_handle+0x22/0x60 ? tick_sched_timer+0x38/0x80 ? tick_sched_do_timer+0x70/0x70 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:cpuidle_enter_state+0xc4/0x420 Code: e8 b1 cd 92 ff 80 7c 24 0f 00 74 17 9c 58 0f 1f 44 00 00 f6 c4 02 0f 85 39 03 00 00 31 ff e8 13 bc 98 ff fb 66 0f 1f 44 00 00 <45> 85 e4 0f 89 cf 01 00 00 c7 45 10 00 00 00 00 48 83 c4 18 44 89 RSP: 0018:ffffb6660015fe68 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: ffff8ac29c228880 RBX: ffffffff8c4d6520 RCX: 000000000000001f RDX: 0000000000000000 RSI: 000000002819a025 RDI: 0000000000000000 RBP: ffff8ac29c232e00 R08: 0000084fb12c2f60 R09: 0000000000000cb2 R10: ffff8ac29c2276c4 R11: ffff8ac29c2276a4 R12: 0000000000000008 R13: 0000084fb12c2f60 R14: 0000000000000008 R15: ffff8ac2987f8000 ? cpuidle_enter_state+0x9f/0x420 cpuidle_enter+0x29/0x40 do_idle+0x1cf/0x250 cpu_startup_entry+0x19/0x20 start_secondary+0x163/0x1b0 secondary_startup_64+0xa4/0xb0
Description of problem: happened after a dnf update, upon reboot. Version-Release number of selected component: kernel-core-5.3.9-300.fc31 Additional info: reporter: libreport-2.10.1 cmdline: BOOT_IMAGE=(hd1,gpt2)/vmlinuz-5.3.9-300.fc31.x86_64 root=/dev/mapper/fedora_localhost--live-root ro resume=/dev/mapper/fedora_localhost--live-swap rd.lvm.lv=fedora_localhost-live/root rd.luks.uuid=luks-b1a94e40-2992-4a9e-a9f4-7c533a7edeac rd.lvm.lv=fedora_localhost-live/swap rhgb quiet crash_function: qdisc_put_unlocked kernel: 5.3.9-300.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: NETDEV WATCHDOG: enp0s3 (e1000): transmit queue 0 timed out WARNING: CPU: 0 PID: 0 at net/sched/sch_generic.c:447 dev_watchdog+0x22d/0x240 Modules linked in: ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc vfat fat snd_intel8x0 snd_ac97_codec ac97_bus snd_seq pktcdvd snd_seq_device snd_pcm joydev snd_timer snd soundcore i2c_piix4 vboxguest edac_mce_amd ip_tables dm_crypt crct10dif_pclmul crc32_pclmul crc32c_intel vmwgfx drm_kms_helper ttm ghash_clmulni_intel drm serio_raw e1000 ata_generic pata_acpi video fuse CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.3.9-300.fc31.x86_64 #1 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 RIP: 0010:dev_watchdog+0x22d/0x240 Code: 85 c0 75 e5 eb 9f 4c 89 ef c6 05 3a 36 ca 00 01 e8 18 25 fb ff 44 89 e1 4c 89 ee 48 c7 c7 e0 ad 1f 85 48 89 c2 e8 01 34 84 ff <0f> 0b eb 80 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 RSP: 0018:ffffbdaf40003e78 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff9c69536cd200 RCX: 0000000000000006 RDX: 0000000000000007 RSI: 0000000000000092 RDI: ffff9c695ba17900 RBP: ffff9c69595e8480 R08: 0000000000000001 R09: 000000000000025a R10: 000000000000cd38 R11: 0000000000000003 R12: 0000000000000000 R13: ffff9c69595e8000 R14: ffff9c69595e8480 R15: ffffffff85405100 FS: 0000000000000000(0000) GS:ffff9c695ba00000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055fc3b992000 CR3: 00000000c0f84000 CR4: 00000000000406f0 Call Trace: <IRQ> ? qdisc_put_unlocked+0x30/0x30 call_timer_fn+0x2d/0x130 run_timer_softirq+0x180/0x480 ? enqueue_hrtimer+0x36/0x90 ? kvm_sched_clock_read+0xd/0x20 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:native_safe_halt+0xe/0x10 Code: 90 90 90 90 90 90 90 90 90 90 90 90 e9 07 00 00 00 0f 00 2d c6 38 42 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d b6 38 42 00 fb f4 <c3> 90 0f 1f 44 00 00 41 54 55 53 e8 f2 51 77 ff 65 8b 2d bb b2 62 RSP: 0018:ffffffff85403ea0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: ffffffff849e4ca0 RBX: 0000000000000000 RCX: 0000000000000000 RDX: 0000000000000000 RSI: 0000000000000086 RDI: 0000000000000086 RBP: 0000000000000000 R08: 000000218deb9786 R09: 0000000000028880 R10: 00000000000003f7 R11: 0000000000000390 R12: ffffffff85413780 R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff85413780 ? __sched_text_end+0x4/0x4 default_idle+0x1a/0x140 do_idle+0x1ed/0x250 cpu_startup_entry+0x19/0x20 start_kernel+0x54f/0x56c secondary_startup_64+0xa4/0xb0
*** Bug 1773036 has been marked as a duplicate of this bug. ***
*** Bug 1776054 has been marked as a duplicate of this bug. ***
*** Bug 1776866 has been marked as a duplicate of this bug. ***
*** Bug 1779174 has been marked as a duplicate of this bug. ***
*** Bug 1779287 has been marked as a duplicate of this bug. ***
*** Bug 1782586 has been marked as a duplicate of this bug. ***
*** Bug 1786575 has been marked as a duplicate of this bug. ***
*** Bug 1786765 has been marked as a duplicate of this bug. ***
Description of problem: Fedora 31 Workstation, fresh installation in a Oracle VirtualBox VM. sudo dnf upgrade had been started and was running. That's all what I can say about the circumstances Version-Release number of selected component: kernel-core-5.3.7-301.fc31 Additional info: reporter: libreport-2.10.1 cmdline: BOOT_IMAGE=(hd0,msdos1)/boot/vmlinuz-5.3.7-301.fc31.x86_64 root=UUID=1ffc0ceb-36e3-4151-a079-0fbe37527d11 ro resume=UUID=b5ef1499-a628-4900-8b80-68363fab0afb rhgb quiet crash_function: qdisc_put_unlocked kernel: 5.3.7-301.fc31.x86_64 runlevel: N 5 type: Kerneloops Truncated backtrace: NETDEV WATCHDOG: enp0s3 (e1000): transmit queue 0 timed out WARNING: CPU: 1 PID: 0 at net/sched/sch_generic.c:447 dev_watchdog+0x22d/0x240 Modules linked in: nls_utf8 isofs nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter sunrpc snd_intel8x0 snd_ac97_codec ac97_bus snd_seq snd_seq_device snd_pcm snd_timer joydev snd soundcore edac_mce_amd vboxguest i2c_piix4 crct10dif_pclmul crc32_pclmul ghash_clmulni_intel ip_tables vboxvideo drm_vram_helper ttm drm_kms_helper drm crc32c_intel serio_raw e1000 ata_generic pata_acpi video fuse CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.3.7-301.fc31.x86_64 #1 Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 RIP: 0010:dev_watchdog+0x22d/0x240 Code: 85 c0 75 e5 eb 9f 4c 89 ef c6 05 3a 36 ca 00 01 e8 18 25 fb ff 44 89 e1 4c 89 ee 48 c7 c7 90 aa 1f 9e 48 89 c2 e8 01 34 84 ff <0f> 0b eb 80 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f 44 RSP: 0018:ffffa83a400b4e78 EFLAGS: 00010286 RAX: 0000000000000000 RBX: ffff8bd7d3abc000 RCX: 000000000000083f RDX: 0000000000000000 RSI: 00000000000000f6 RDI: 000000000000083f RBP: ffff8bd7d9060480 R08: ffff8bd7dba97908 R09: 0000000000000225 R10: 000000000000ba3c R11: 0000000000000003 R12: 0000000000000000 R13: ffff8bd7d9060000 R14: ffff8bd7d9060480 R15: ffffffff9e405100 FS: 0000000000000000(0000) GS:ffff8bd7dba80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f6197dfd800 CR3: 000000002940a000 CR4: 00000000000406e0 Call Trace: <IRQ> ? qdisc_put_unlocked+0x30/0x30 call_timer_fn+0x2d/0x130 run_timer_softirq+0x180/0x480 ? tick_sched_handle+0x22/0x60 ? tick_sched_timer+0x38/0x80 ? kvm_sched_clock_read+0xd/0x20 __do_softirq+0xee/0x2ff irq_exit+0xe9/0xf0 smp_apic_timer_interrupt+0x76/0x130 apic_timer_interrupt+0xf/0x20 </IRQ> RIP: 0010:native_safe_halt+0xe/0x10 Code: 90 90 90 90 90 90 90 90 90 90 90 90 e9 07 00 00 00 0f 00 2d 16 3a 42 00 f4 c3 66 90 e9 07 00 00 00 0f 00 2d 06 3a 42 00 fb f4 <c3> 90 0f 1f 44 00 00 41 54 55 53 e8 e2 54 77 ff 65 8b 2d 0b b4 62 RSP: 0018:ffffa83a4006feb0 EFLAGS: 00000246 ORIG_RAX: ffffffffffffff13 RAX: ffffffff9d9e4b50 RBX: 0000000000000001 RCX: 0000000000000000 RDX: 0000000000000001 RSI: 0000000000000001 RDI: ffff8bd7dba9c520 RBP: 0000000000000001 R08: 00000066a1728aaf R09: 0000000000000000 R10: ffff8bd7dba97888 R11: 0000000000000000 R12: ffff8bd7db221f40 R13: 0000000000000000 R14: 0000000000000000 R15: ffff8bd7db221f40 ? __sched_text_end+0x4/0x4 default_idle+0x1a/0x140 do_idle+0x1ed/0x250 cpu_startup_entry+0x19/0x20 start_secondary+0x163/0x1b0 secondary_startup_64+0xa4/0xb0
*** Bug 1787923 has been marked as a duplicate of this bug. ***
Would like to report some observation regarding this bug (well, my duplicate, to be precise): After installing the r8168 kernel module of Realtek webpage for one single installed kernel, the wifi worked without a problem in that configuration. Interestingly, it afterwards also worked without any problem when booting to other kernels still using the r8169 module!
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There are a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 31 kernel bugs. Fedora 31 has now been rebased to 5.5.7-200.fc31. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 32, and are still experiencing this issue, please change the version to Fedora 32. If you experience different issues, please open a new bug report for those.
(In reply to Justin M. Forbes from comment #33) > *********** MASS BUG UPDATE ************** > > We apologize for the inconvenience. There are a large number of bugs to go > through and several of them have gone stale. Due to this, we are doing a > mass bug update across all of the Fedora 31 kernel bugs. > > Fedora 31 has now been rebased to 5.5.7-200.fc31. Please test this kernel > update (or newer) and let us know if you issue has been resolved or if it is > still present with the newer kernel. > > If you have moved on to Fedora 32, and are still experiencing this issue, > please change the version to Fedora 32. > > If you experience different issues, please open a new bug report for those. I haven't seen this ethernet transmit queue timeout in qdisc_put_unlocked since I updated from 5.3.18 to 5.4.5. I guess that this problem was fixed in the 5.4 branch. Thanks.
Does that indicate that the patch that fixed the problem remains unknown? Also, if any of the bugs reported for RHEL have been marked as a duplicate of this bug, they should be reopened or examined if the patch has been backported to the EL kernels.