I have been testing the 5.3 kernel builds on F30, and have run into some strange crashes. They all follow the same pattern: first a WARNING a few minutes after boot: Oct 03 04:39:51 laptop kernel: WARNING: CPU: 5 PID: 198 at fs/ext4/inode.c:3941 ext4_set_page_dirty+0x3e/0x50 Oct 03 04:39:51 laptop kernel: Modules linked in: squashfs zstd_decompress loop ccm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ppdev parport fuse sunrpc vfat fat snd_hda_codec_hdmi snd_hda_codec_realtek iwlmvm snd_hda_codec_generic ledtrig_audio x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_intel mac80211 snd_hda_codec kvm_intel snd_hda_core snd_hwdep libarc4 snd_seq iwlwifi kvm snd_seq_device irqbypass intel_cstate snd_pcm iTCO_wdt intel_uncore asus_nb_wmi mei_hdcp iTCO_vendor_support intel_rapl_msr snd_timer cfg80211 asus_wmi intel_rapl_perf processor_thermal_device rtsx_pci_ms snd sparse_keymap joydev mei_me memstick i2c_i801 soundcore idma64 rfkill intel_rapl_common mei Oct 03 04:39:51 laptop kernel: intel_pch_thermal elan_i2c intel_lpss_pci intel_lpss intel_soc_dts_iosf int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad dm_crypt i915 nouveau ttm crct10dif_pclmul i2c_algo_bit crc32_pclmul rtsx_pci_sdmmc drm_kms_helper crc32c_intel mmc_core nvme mxm_wmi drm nvme_core ghash_clmulni_intel serio_raw rtsx_pci r8169 hid_microsoft ff_memless i2c_hid wmi video [last unloaded: vmnet] Oct 03 04:39:51 laptop kernel: CPU: 5 PID: 198 Comm: kworker/u16:4 Tainted: G O 5.3.2-300.fc30.x86_64 #1 Oct 03 04:39:51 laptop kernel: Hardware name: ASUSTeK COMPUTER INC. N552VX/N552VX, BIOS N552VX.300 08/31/2016 Oct 03 04:39:51 laptop kernel: Workqueue: i915 __i915_gem_free_work [i915] Oct 03 04:39:51 laptop kernel: RIP: 0010:ext4_set_page_dirty+0x3e/0x50 Oct 03 04:39:51 laptop kernel: Code: 48 8b 00 a8 01 75 16 48 8b 57 08 48 8d 42 ff 83 e2 01 48 0f 44 c7 48 8b 00 a8 08 74 0d 48 8b 07 f6 c4 20 74 0f e9 a2 ef f7 ff <0f> 0b 48 8b 07 f6 c4 20 75 f1 0f 0b e9 91 ef f7 ff 90 0f 1f 44 00 Oct 03 04:39:51 laptop kernel: RSP: 0018:ffffa617002d3d90 EFLAGS: 00010246 Oct 03 04:39:51 laptop kernel: RAX: 0017fffe00002016 RBX: ffff99931606f800 RCX: 0000000000000000 Oct 03 04:39:51 laptop kernel: RDX: 0000000000000000 RSI: 00000003d4800000 RDI: ffffd2e80fe496c0 Oct 03 04:39:51 laptop kernel: RBP: ffffd2e80fe496c0 R08: 00000003d4800000 R09: ffff999429548188 Oct 03 04:39:51 laptop kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 00000000003f925b Oct 03 04:39:51 laptop kernel: R13: ffff99931630a400 R14: ffff99942d4917a0 R15: 0000000000000000 Oct 03 04:39:51 laptop kernel: FS: 0000000000000000(0000) GS:ffff999433b40000(0000) knlGS:0000000000000000 Oct 03 04:39:51 laptop kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 03 04:39:51 laptop kernel: CR2: 00007f16bfc25688 CR3: 000000037a40a002 CR4: 00000000003606e0 Oct 03 04:39:51 laptop kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 03 04:39:51 laptop kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Oct 03 04:39:51 laptop kernel: Call Trace: Oct 03 04:39:51 laptop kernel: i915_gem_userptr_put_pages+0x14a/0x1e0 [i915] Oct 03 04:39:51 laptop kernel: __i915_gem_object_put_pages+0x5e/0xa0 [i915] Oct 03 04:39:51 laptop kernel: __i915_gem_free_objects+0x123/0x220 [i915] Oct 03 04:39:51 laptop kernel: __i915_gem_free_work+0x64/0x90 [i915] Oct 03 04:39:51 laptop kernel: process_one_work+0x19d/0x340 Oct 03 04:39:51 laptop kernel: worker_thread+0x50/0x3b0 Oct 03 04:39:51 laptop kernel: kthread+0xfb/0x130 Oct 03 04:39:51 laptop kernel: ? process_one_work+0x340/0x340 Oct 03 04:39:51 laptop kernel: ? kthread_park+0x80/0x80 Oct 03 04:39:51 laptop kernel: ret_from_fork+0x35/0x40 Oct 03 04:39:51 laptop kernel: ---[ end trace 8ee114643cf24b2e ]--- Followed, possibly a few days later, by a WARNING and a BUG: Oct 07 01:32:44 laptop kernel: WARNING: CPU: 6 PID: 14209 at fs/ext4/inode.c:3942 ext4_set_page_dirty+0x48/0x50 Oct 07 01:32:44 laptop kernel: Modules linked in: xfs btrfs xor zstd_compress raid6_pq uas usb_storage squashfs zstd_decompress loop ccm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ppdev parport fuse sunrpc vfat fat snd_hda_codec_hdmi snd_hda_codec_realtek iwlmvm snd_hda_codec_generic ledtrig_audio x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_intel mac80211 snd_hda_codec kvm_intel snd_hda_core snd_hwdep libarc4 snd_seq iwlwifi kvm snd_seq_device irqbypass intel_cstate snd_pcm iTCO_wdt intel_uncore asus_nb_wmi mei_hdcp iTCO_vendor_support intel_rapl_msr snd_timer cfg80211 asus_wmi intel_rapl_perf processor_thermal_device rtsx_pci_ms snd sparse_keymap joydev mei_me memstick i2c_i801 Oct 07 01:32:44 laptop kernel: soundcore idma64 rfkill intel_rapl_common mei intel_pch_thermal elan_i2c intel_lpss_pci intel_lpss intel_soc_dts_iosf int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad dm_crypt i915 nouveau ttm crct10dif_pclmul i2c_algo_bit crc32_pclmul rtsx_pci_sdmmc drm_kms_helper crc32c_intel mmc_core nvme mxm_wmi drm nvme_core ghash_clmulni_intel serio_raw rtsx_pci r8169 hid_microsoft ff_memless i2c_hid wmi video [last unloaded: vmnet] Oct 07 01:32:44 laptop kernel: CPU: 6 PID: 14209 Comm: kworker/u16:1 Tainted: G W O 5.3.2-300.fc30.x86_64 #1 Oct 07 01:32:44 laptop kernel: Hardware name: ASUSTeK COMPUTER INC. N552VX/N552VX, BIOS N552VX.300 08/31/2016 Oct 07 01:32:44 laptop kernel: Workqueue: i915 __i915_gem_free_work [i915] Oct 07 01:32:44 laptop kernel: RIP: 0010:ext4_set_page_dirty+0x48/0x50 Oct 07 01:32:44 laptop kernel: Code: 08 48 8d 42 ff 83 e2 01 48 0f 44 c7 48 8b 00 a8 08 74 0d 48 8b 07 f6 c4 20 74 0f e9 a2 ef f7 ff 0f 0b 48 8b 07 f6 c4 20 75 f1 <0f> 0b e9 91 ef f7 ff 90 0f 1f 44 00 00 41 54 49 89 fc 55 89 d5 53 Oct 07 01:32:44 laptop kernel: RSP: 0018:ffffa617060dbd90 EFLAGS: 00010246 Oct 07 01:32:44 laptop kernel: RAX: 0017fffe00020016 RBX: ffff9993a4991000 RCX: 0000000000000000 Oct 07 01:32:44 laptop kernel: RDX: 0000000000000000 RSI: 000000037614f000 RDI: ffffd2e808eaac40 Oct 07 01:32:44 laptop kernel: RBP: ffffd2e808eaac40 R08: 000000037614f000 R09: 0000000000000001 Oct 07 01:32:44 laptop kernel: R10: ffff99940ceb6201 R11: 0000000000000000 R12: 000000000023aab1 Oct 07 01:32:44 laptop kernel: R13: ffff9990316eb600 R14: ffff9992c0bc8240 R15: 0000000000000000 Oct 07 01:32:44 laptop kernel: FS: 0000000000000000(0000) GS:ffff999433b80000(0000) knlGS:0000000000000000 Oct 07 01:32:44 laptop kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 07 01:32:44 laptop kernel: CR2: 00003ae54cb46000 CR3: 000000037a40a001 CR4: 00000000003606e0 Oct 07 01:32:44 laptop kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 07 01:32:44 laptop kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Oct 07 01:32:44 laptop kernel: Call Trace: Oct 07 01:32:44 laptop kernel: i915_gem_userptr_put_pages+0x14a/0x1e0 [i915] Oct 07 01:32:44 laptop kernel: __i915_gem_object_put_pages+0x5e/0xa0 [i915] Oct 07 01:32:44 laptop kernel: __i915_gem_free_objects+0x123/0x220 [i915] Oct 07 01:32:44 laptop kernel: __i915_gem_free_work+0x64/0x90 [i915] Oct 07 01:32:44 laptop kernel: process_one_work+0x19d/0x340 Oct 07 01:32:44 laptop kernel: worker_thread+0x50/0x3b0 Oct 07 01:32:44 laptop kernel: kthread+0xfb/0x130 Oct 07 01:32:44 laptop kernel: ? process_one_work+0x340/0x340 Oct 07 01:32:44 laptop kernel: ? kthread_park+0x80/0x80 Oct 07 01:32:44 laptop kernel: ret_from_fork+0x35/0x40 Oct 07 01:32:44 laptop kernel: ---[ end trace 8ee114643cf24b2f ]--- Oct 07 01:33:15 laptop kernel: ------------[ cut here ]------------ Oct 07 01:33:15 laptop kernel: kernel BUG at fs/ext4/inode.c:2721! Oct 07 01:33:15 laptop kernel: invalid opcode: 0000 [#1] SMP PTI Oct 07 01:33:15 laptop kernel: CPU: 2 PID: 14209 Comm: kworker/u16:1 Tainted: G W O 5.3.2-300.fc30.x86_64 #1 Oct 07 01:33:15 laptop kernel: Hardware name: ASUSTeK COMPUTER INC. N552VX/N552VX, BIOS N552VX.300 08/31/2016 Oct 07 01:33:15 laptop kernel: Workqueue: writeback wb_workfn (flush-253:1) Oct 07 01:33:15 laptop kernel: RIP: 0010:mpage_prepare_extent_to_map+0x25d/0x290 Oct 07 01:33:15 laptop kernel: Code: 00 75 3d e8 d5 3a 62 00 48 8b 04 24 48 39 44 24 10 0f 86 49 fe ff ff 31 c0 eb af 4c 89 ff e8 3a 2d e8 ff e9 b8 fe ff ff 0f 0b <0f> 0b 48 8d 7c 24 18 89 44 24 08 e8 43 3a e9 ff 8b 44 24 08 eb 8a Oct 07 01:33:15 laptop kernel: RSP: 0018:ffffa617060db988 EFLAGS: 00010246 Oct 07 01:33:15 laptop kernel: RAX: 0017fffe0002003f RBX: ffffa617060db9b0 RCX: 000000000000074c Oct 07 01:33:15 laptop kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffd2e80c2d41c0 Oct 07 01:33:15 laptop kernel: RBP: 0000000000002800 R08: 0000000000000000 R09: 0000000000000000 Oct 07 01:33:15 laptop kernel: R10: 0000000000000228 R11: ffffffffffffffff R12: ffffa617060dba20 Oct 07 01:33:15 laptop kernel: R13: ffff99940fc26798 R14: ffffa617060dbae0 R15: ffffd2e80c2d41c0 Oct 07 01:33:15 laptop kernel: FS: 0000000000000000(0000) GS:ffff999433a80000(0000) knlGS:0000000000000000 Oct 07 01:33:15 laptop kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 07 01:33:15 laptop kernel: CR2: 00002cf01cf1a000 CR3: 000000037a40a003 CR4: 00000000003606e0 Oct 07 01:33:15 laptop kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 07 01:33:15 laptop kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Oct 07 01:33:15 laptop kernel: Call Trace: Oct 07 01:33:15 laptop kernel: ext4_writepages+0x3da/0xe20 Oct 07 01:33:15 laptop kernel: ? recalibrate_cpu_khz+0x10/0x10 Oct 07 01:33:15 laptop kernel: ? ktime_get+0x3c/0x90 Oct 07 01:33:15 laptop kernel: ? __switch_to_asm+0x40/0x70 Oct 07 01:33:15 laptop kernel: ? __switch_to_asm+0x40/0x70 Oct 07 01:33:15 laptop kernel: ? __switch_to_asm+0x40/0x70 Oct 07 01:33:15 laptop kernel: ? __switch_to_asm+0x34/0x70 Oct 07 01:33:15 laptop kernel: ? do_writepages+0x43/0xd0 Oct 07 01:33:15 laptop kernel: ? ext4_mark_inode_dirty+0x1d0/0x1d0 Oct 07 01:33:15 laptop kernel: do_writepages+0x43/0xd0 Oct 07 01:33:15 laptop kernel: ? __switch_to_asm+0x34/0x70 Oct 07 01:33:15 laptop kernel: __writeback_single_inode+0x3d/0x330 Oct 07 01:33:15 laptop kernel: writeback_sb_inodes+0x1fd/0x480 Oct 07 01:33:15 laptop kernel: __writeback_inodes_wb+0x4c/0xc0 Oct 07 01:33:15 laptop kernel: wb_writeback+0x255/0x2f0 Oct 07 01:33:15 laptop kernel: ? get_nr_inodes+0x32/0x50 Oct 07 01:33:15 laptop kernel: wb_workfn+0x38f/0x450 Oct 07 01:33:15 laptop kernel: ? __switch_to_asm+0x34/0x70 Oct 07 01:33:15 laptop kernel: process_one_work+0x19d/0x340 Oct 07 01:33:15 laptop kernel: worker_thread+0x50/0x3b0 Oct 07 01:33:15 laptop kernel: kthread+0xfb/0x130 Oct 07 01:33:15 laptop kernel: ? process_one_work+0x340/0x340 Oct 07 01:33:15 laptop kernel: ? kthread_park+0x80/0x80 Oct 07 01:33:15 laptop kernel: ret_from_fork+0x35/0x40 Oct 07 01:33:15 laptop kernel: Modules linked in: xfs btrfs xor zstd_compress raid6_pq uas usb_storage squashfs zstd_decompress loop ccm ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter ip_tables ppdev parport fuse sunrpc vfat fat snd_hda_codec_hdmi snd_hda_codec_realtek iwlmvm snd_hda_codec_generic ledtrig_audio x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_intel mac80211 snd_hda_codec kvm_intel snd_hda_core snd_hwdep libarc4 snd_seq iwlwifi kvm snd_seq_device irqbypass intel_cstate snd_pcm iTCO_wdt intel_uncore asus_nb_wmi mei_hdcp iTCO_vendor_support intel_rapl_msr snd_timer cfg80211 asus_wmi intel_rapl_perf processor_thermal_device rtsx_pci_ms snd sparse_keymap joydev mei_me memstick i2c_i801 Oct 07 01:33:15 laptop kernel: soundcore idma64 rfkill intel_rapl_common mei intel_pch_thermal elan_i2c intel_lpss_pci intel_lpss intel_soc_dts_iosf int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad dm_crypt i915 nouveau ttm crct10dif_pclmul i2c_algo_bit crc32_pclmul rtsx_pci_sdmmc drm_kms_helper crc32c_intel mmc_core nvme mxm_wmi drm nvme_core ghash_clmulni_intel serio_raw rtsx_pci r8169 hid_microsoft ff_memless i2c_hid wmi video [last unloaded: vmnet] Oct 07 01:33:15 laptop kernel: ---[ end trace 8ee114643cf24b30 ]--- Oct 07 01:33:15 laptop kernel: RIP: 0010:mpage_prepare_extent_to_map+0x25d/0x290 Oct 07 01:33:15 laptop kernel: Code: 00 75 3d e8 d5 3a 62 00 48 8b 04 24 48 39 44 24 10 0f 86 49 fe ff ff 31 c0 eb af 4c 89 ff e8 3a 2d e8 ff e9 b8 fe ff ff 0f 0b <0f> 0b 48 8d 7c 24 18 89 44 24 08 e8 43 3a e9 ff 8b 44 24 08 eb 8a Oct 07 01:33:15 laptop kernel: RSP: 0018:ffffa617060db988 EFLAGS: 00010246 Oct 07 01:33:15 laptop kernel: RAX: 0017fffe0002003f RBX: ffffa617060db9b0 RCX: 000000000000074c Oct 07 01:33:15 laptop kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffd2e80c2d41c0 Oct 07 01:33:15 laptop kernel: RBP: 0000000000002800 R08: 0000000000000000 R09: 0000000000000000 Oct 07 01:33:15 laptop kernel: R10: 0000000000000228 R11: ffffffffffffffff R12: ffffa617060dba20 Oct 07 01:33:15 laptop kernel: R13: ffff99940fc26798 R14: ffffa617060dbae0 R15: ffffd2e80c2d41c0 Oct 07 01:33:15 laptop kernel: FS: 0000000000000000(0000) GS:ffff999433a80000(0000) knlGS:0000000000000000 Oct 07 01:33:15 laptop kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 07 01:33:15 laptop kernel: CR2: 00002cf01cf1a000 CR3: 000000037a40a003 CR4: 00000000003606e0 Oct 07 01:33:15 laptop kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Oct 07 01:33:15 laptop kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 This happened both on 5.3.1 and 5.3.2, and never on any previous version that I have record of---certainly not 5.2.x. The call stacks in all crash instances I've recorded, 4, are all the same: warnings coming from somewhere in the i915 stack, and finally the bug on ext4.
Possibly related to https://bugzilla.kernel.org/show_bug.cgi?id=203317
I've bisected the issue down to the following commit: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.3.y&id=0bd6cb6b58f7332c61cef2e4ae48db1ca9910b6b To make sure that I got the right commit, I've also applied "Revert 0bd6cb6b58f7332c61cef2e4ae48db1ca9910b6b" to the latest 5.3.6 stable kernel, and the WARNING above no longer reproduces. Whether the later bugs still happen anyway is a mystery, I do not understand this subsystem sufficiently to understand why/how this happens.
Thanks for the bisect. This is best reported at the freedesktop.org bugzilla since that's where all the graphics people are.
Reported at https://bugs.freedesktop.org/show_bug.cgi?id=112012.
Attempt to `killall -9 telegram-desktop` triggers this kernel panic on my F31 desktop in 99% of cases. It crashes spontaneously multiple times per day without special actions from my side. Lenovo Thinkpad Carbon X1 Gen 5 Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz options i915 enable_guc=3 enable_fbc=1 All available 5.3.x kernels in Fedora 31 are affected: - 5.3.4-300.fc31 - 5.3.6-300.fc31 - 5.3.7-301.fc31 ``` [27414.743340] mei_hdcp mei::b638ab7e-94e2-4ea2-a552-d1c54b627f04:01: bound 0000:00:02.0 (ops i915_hdcp_component_ops [i915]) [27420.392550] [drm] HuC: Loaded firmware i915/kbl_huc_ver02_00_1810.bin (version 2.0) [27420.403045] [drm] GuC: Loaded firmware i915/kbl_guc_32.0.3.bin (version 32.0) [27420.414134] i915 0000:00:02.0: GuC firmware version 32.0 [27420.414135] i915 0000:00:02.0: GuC submission disabled [27420.414136] i915 0000:00:02.0: HuC enabled ``` i915 is the most popular GPU on the desktop. The most popular driver just crashes dozens times per day. All available kernels in F31 are affected. That means that F31 release has been completely screwed up. I propose to raise priority to Urgent, because the bug makes whole system unusable. The priority should be raisen to
Kernel 5.3.14, just released, fixes this with 2d691aeca4aecbb8d0414a777a46981a8e142b05. As well as 5.4.x.
Great! I'm testing 5.4.0-2.fc32.x86_64 for couple hours. No crashes yet. Thanks, Robert!
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There are a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 30 kernel bugs. Fedora 30 has now been rebased to 5.5.7-100.fc30. Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel. If you have moved on to Fedora 31, and are still experiencing this issue, please change the version to Fedora 31. If you experience different issues, please open a new bug report for those.
I keep getting "outstanding request" emails from bugzilla. To be clear, this bug is fixed.