Bug 1768092 - (Suspected problem with BFQ) Frequent kernel freeze after upgrading to Fedora 31
Summary: (Suspected problem with BFQ) Frequent kernel freeze after upgrading to Fedora 31
Keywords:
Status: CLOSED DUPLICATE of bug 1767539
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 31
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-02 13:31 UTC by Patrick Dung
Modified: 2020-03-03 19:51 UTC (History)
21 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-03 19:51:14 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
kernel stack trace at console (447.32 KB, image/jpeg)
2019-11-02 13:31 UTC, Patrick Dung
no flags Details
kernel stack trace at console (452.59 KB, image/jpeg)
2019-11-02 13:31 UTC, Patrick Dung
no flags Details
kernel stack trace at console (429.26 KB, image/jpeg)
2019-11-02 13:32 UTC, Patrick Dung
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Linux Kernel 205447 0 None None None 2019-11-09 03:59:21 UTC
Red Hat Bugzilla 1767539 0 high CLOSED BUG: kernel NULL pointer dereference RIP: 0010:rb_erase+0x1b1/0x370 2023-09-14 05:45:17 UTC

Description Patrick Dung 2019-11-02 13:31:09 UTC
Created attachment 1631839 [details]
kernel stack trace at console

1. Please describe the problem:
Previously I was using Fedora 30 without problem.
In Oct, around Oct-17. I had installed kernel 5.3.5-fc30 and used about one week without problem.
Then I had upgraded to Fedora 31 with default (current) kernel 5.3.7-fc31.
I had experienced at about one or two system freeze/crash since upgraded to FC31.
I had also tried to use kernel 5.3.5-fc30 but still got a crash just now.

When it freezes, there is no network connection.
For the console, it is not accepting the magic sequence for the keyboard.
I had enabled kdump but there is no core files generated.

2. What is the Version-Release number of the kernel:
5.3.5-200.fc30 and 5.3.7-200.fc31

3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :
5.3.5-200.fc30 and 5.3.7-200.fc31

4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:
Just use the desktop, it crashes randomly.

5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:
Haven't tried this because I am not sure if my GPU and vmware driver support the rawhide kernel.

6. Are you running any modules that not shipped with directly Fedora's kernel?:
Nvidia and Vmware workstation.

7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

Comment 1 Patrick Dung 2019-11-02 13:31:39 UTC
Created attachment 1631840 [details]
kernel stack trace at console

Comment 2 Patrick Dung 2019-11-02 13:32:09 UTC
Created attachment 1631841 [details]
kernel stack trace at console

Comment 3 Patrick Dung 2019-11-02 13:37:24 UTC
When it freezes, there is no useful information logged in journalctl or /var/log/messages.
In one occurrence of kernel freeze, it logged a few lines into system log but there no further useful information being logged:

Oct 31 02:03:50 home kernel: BUG: kernel NULL pointer dereference, address: 0000000000000000
Oct 31 02:03:50 home kernel: BUG: kernel NULL pointer dereference, address: 0000000000000000
Oct 31 02:03:50 home kernel: #PF: supervisor read access in kernel mode
Oct 31 02:03:50 home kernel: #PF: error_code(0x0000) - not-present page
Oct 31 02:31:04 home kernel: microcode: microcode updated early to revision 0x43, date = 2019-03-01

The last line is from the manual system reboot after the system freezes

Comment 4 Patrick Dung 2019-11-04 21:58:13 UTC
Just now I had updated to kernel 5.3.8 from update-testing. I tried to perform a CPU intensive task and the system hanged.
Seems related to BFQ scheduler. Previously (FC30) I was using the deadline scheduler without problem.

Nov 05 05:26:24 kernel: Oops: 0000 [#1] SMP PTI
Nov 05 05:26:24 kernel: CPU: 8 PID: 23976 Comm: ora_ckpt_homedb Kdump: loaded Tainted: P           OE     5.3.8-300.fc31.x86_64 #1
Nov 05 05:26:24 kernel: Hardware name: [REDACTED]
Nov 05 05:26:24 kernel: RIP: 0010:bfq_insert+0x21/0x70
Nov 05 05:26:24 kernel: Code: 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 fa 48 89 fd 53 48 8b 07 48 89 f3 48 85 c0 74 27 48 8b 7e 28 eb 03 48 89 c8 <48> 8b 48 28 48 8d 70 10 48 8d 50 08 48 29 f9 48 85 c9 48 0f 4f d6
Nov 05 05:26:24 kernel: RSP: 0018:ffffa2e346f57768 EFLAGS: 00010006
Nov 05 05:26:24 kernel: RAX: 0000000000001000 RBX: ffff936208036488 RCX: 0000000000001000
Nov 05 05:26:24 kernel: RDX: ffff93587a1d30a0 RSI: ffff93587a1d30a8 RDI: 000000aa4e352b55
Nov 05 05:26:24 kernel: RBP: ffff93626803bd58 R08: ffffa2e346f57739 R09: 0000000000000004
Nov 05 05:26:24 kernel: R10: 0000000000000002 R11: ffffa2e346f57739 R12: ffff93626803bd50
Nov 05 05:26:24 kernel: R13: 0000000000000001 R14: ffff936208036400 R15: ffff936208036488
Nov 05 05:26:24 kernel: FS:  00007f64fe2823c0(0000) GS:ffff93627fa00000(0000) knlGS:0000000000000000
Nov 05 05:26:24 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 05 05:26:24 kernel: CR2: 0000000000001028 CR3: 0000002ca90a6004 CR4: 00000000001606e0
Nov 05 05:26:24 kernel: Call Trace:
Nov 05 05:26:24 kernel:  __bfq_deactivate_entity+0x125/0x1a0
Nov 05 05:26:24 kernel:  bfq_deactivate_entity+0x4f/0xc0
Nov 05 05:26:24 kernel:  bfq_del_bfqq_busy+0xac/0x160
Nov 05 05:26:24 kernel:  __bfq_bfqq_expire+0x52/0xc0
Nov 05 05:26:24 kernel:  bfq_bfqq_expire+0x363/0x8e0
Nov 05 05:26:24 kernel:  ? bfq_may_expire_for_budg_timeout+0x4b/0x190
Nov 05 05:26:24 kernel:  bfq_dispatch_request+0x1c5/0xea0
Nov 05 05:26:24 kernel:  blk_mq_do_dispatch_sched+0xc6/0x120
Nov 05 05:26:24 kernel:  blk_mq_sched_dispatch_requests+0x111/0x160
Nov 05 05:26:24 kernel:  __blk_mq_run_hw_queue+0x55/0x110
Nov 05 05:26:24 kernel:  __blk_mq_delay_run_hw_queue+0x164/0x170
Nov 05 05:26:24 kernel:  blk_mq_run_hw_queue+0x87/0x110
Nov 05 05:26:24 kernel:  blk_mq_sched_insert_requests+0x70/0xf0
Nov 05 05:26:24 kernel:  blk_mq_flush_plug_list+0x214/0x2b0
Nov 05 05:26:24 kernel:  blk_flush_plug_list+0xec/0x110
Nov 05 05:26:24 kernel:  blk_finish_plug+0x21/0x2e
Nov 05 05:26:24 kernel:  ext4_writepages+0xa39/0xe20
Nov 05 05:26:24 kernel:  ? finish_task_switch+0x108/0x2a0
Nov 05 05:26:24 kernel:  ? do_writepages+0x43/0xd0
Nov 05 05:26:24 kernel:  ? ext4_mark_inode_dirty+0x1d0/0x1d0
Nov 05 05:26:24 kernel:  do_writepages+0x43/0xd0
Nov 05 05:26:24 kernel:  ? ext4_write_end+0x14e/0x430
Nov 05 05:26:24 kernel:  __filemap_fdatawrite_range+0xbf/0x100
Nov 05 05:26:24 kernel:  file_write_and_wait_range+0x4e/0xa0
Nov 05 05:26:24 kernel:  ext4_sync_file+0x86/0x3d0
Nov 05 05:26:24 kernel:  ext4_file_write_iter+0xff/0x3b0
Nov 05 05:26:24 kernel:  ? _cond_resched+0x15/0x30
Nov 05 05:26:24 kernel:  new_sync_write+0x12d/0x1d0
Nov 05 05:26:24 kernel:  vfs_write+0xb6/0x1a0
Nov 05 05:26:24 kernel:  ksys_pwrite64+0x65/0xa0
Nov 05 05:26:24 kernel:  do_syscall_64+0x5f/0x1a0
Nov 05 05:26:24 kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Nov 05 05:26:24 kernel: RIP: 0033:0x7f64fe49821a
Nov 05 05:26:24 kernel: Code: 48 c7 c0 ff ff ff ff eb be 0f 1f 80 00 00 00 00 f3 0f 1e fa 49 89 ca 64 8b 04 25 18 00 00 00 85 c0 75 15 b8 12 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 5e c3 0f 1f 44 00 00 48 83 ec 28 48 89 54 24
Nov 05 05:26:24 kernel: RSP: 002b:00007ffd8d174c58 EFLAGS: 00000246 ORIG_RAX: 0000000000000012
Nov 05 05:26:24 kernel: RAX: ffffffffffffffda RBX: 00007f64fd14d020 RCX: 00007f64fe49821a
Nov 05 05:26:24 kernel: RDX: 0000000000004000 RSI: 00007f64fd148000 RDI: 0000000000000100
Nov 05 05:26:24 kernel: RBP: 00007ffd8d179600 R08: 00007ffd8d1e7090 R09: 000000009dbcf778
Nov 05 05:26:24 kernel: R10: 000000000000c000 R11: 0000000000000246 R12: 0000000000004000
Nov 05 05:26:24 kernel: R13: 00000001f49df2d9 R14: 0000000000000000 R15: 00007f64fd14d228
Nov 05 05:26:24 kernel: Modules linked in: nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) drm_kms_helper drm ipmi_devintf ipmi_msghandler vfio_iommu_type1 vfio mpt3sas raid_class mptctl mptbase vmnet(OE) vmmon(OE) ebtable_filter ebtables openvswitch nsh nf_conncount bridge stp llc parport_pc parport nf_log_ipv6 nf_log_ipv4 nf_log_common nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_masq nft_chain_nat nf_nat nft_counter nft_ct nf_conntrack vmw_vsock_vmci_transport nf_defrag_ipv6 vsock nf_defrag_ipv4 nf_tables vmw_vmci nfnetlink nct6775 hwmon_vid lm92 xfs vfat fat btrfs zstd_compress zstd_decompress intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_hdmi kvm_intel snd_hda_codec_realtek kvm dm_raid snd_hda_codec_generic irqbypass ledtrig_audio snd_hda_intel iTCO_wdt crct10dif_pclmul iTCO_vendor_support snd_hda_codec crc32_pclmul snd_hda_core joydev ghash_clmulni_intel snd_hwdep intel_cstate snd_seq intel_uncore snd_seq_device
Nov 05 05:26:24 kernel:  intel_rapl_perf snd_pcm pcspkr i2c_i801 snd_timer snd lpc_ich ses enclosure scsi_transport_sas soundcore nfsd nfs_acl lockd auth_rpcgss grace sunrpc binfmt_misc ip_tables raid456 libcrc32c async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 crc32c_intel igb megaraid_sas dca i2c_algo_bit wmi target_core_mod vhost_net tun tap vhost fuse ecryptfs [last unloaded: ipmi_msghandler]
Nov 05 05:26:24 kernel: CR2: 0000000000001028
Nov 05 05:26:24 kernel: ---[ end trace ea552b29d0f8a0af ]---
Nov 05 05:26:24 kernel: RIP: 0010:bfq_insert+0x21/0x70
Nov 05 05:26:24 kernel: Code: 66 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 48 89 fa 48 89 fd 53 48 8b 07 48 89 f3 48 85 c0 74 27 48 8b 7e 28 eb 03 48 89 c8 <48> 8b 48 28 48 8d 70 10 48 8d 50 08 48 29 f9 48 85 c9 48 0f 4f d6
Nov 05 05:26:24 kernel: RSP: 0018:ffffa2e346f57768 EFLAGS: 00010006
Nov 05 05:26:24 kernel: RAX: 0000000000001000 RBX: ffff936208036488 RCX: 0000000000001000
Nov 05 05:26:24 kernel: RDX: ffff93587a1d30a0 RSI: ffff93587a1d30a8 RDI: 000000aa4e352b55
Nov 05 05:26:24 kernel: RBP: ffff93626803bd58 R08: ffffa2e346f57739 R09: 0000000000000004
Nov 05 05:26:24 kernel: R10: 0000000000000002 R11: ffffa2e346f57739 R12: ffff93626803bd50
Nov 05 05:26:24 kernel: R13: 0000000000000001 R14: ffff936208036400 R15: ffff936208036488
Nov 05 05:26:24 kernel: FS:  00007f64fe2823c0(0000) GS:ffff93627fa00000(0000) knlGS:0000000000000000
Nov 05 05:26:24 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 05 05:26:24 kernel: CR2: 0000000000001028 CR3: 0000002ca90a6004 CR4: 00000000001606e0

Nov 05 05:27:32 kernel: watchdog: BUG: soft lockup - CPU#10 stuck for 23s! [pmdalinux:3983]
Nov 05 05:27:32 kernel: Modules linked in: nvidia_drm(POE) nvidia_modeset(POE) nvidia(POE) drm_kms_helper drm ipmi_devintf ipmi_msghandler vfio_iommu_type1 vfio mpt3sas raid_class mptctl mptbase vmnet(OE) vmmon(OE) ebtable_filter ebtables openvswitch nsh nf_conncount bridge stp llc parport_pc parport nf_log_ipv6 nf_log_ipv4 nf_log_common nft_log nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_masq nft_chain_nat nf_nat nft_counter nft_ct nf_conntrack vmw_vsock_vmci_transport nf_defrag_ipv6 vsock nf_defrag_ipv4 nf_tables vmw_vmci nfnetlink nct6775 hwmon_vid lm92 xfs vfat fat btrfs zstd_compress zstd_decompress intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_hdmi kvm_intel snd_hda_codec_realtek kvm dm_raid snd_hda_codec_generic irqbypass ledtrig_audio snd_hda_intel iTCO_wdt crct10dif_pclmul iTCO_vendor_support snd_hda_codec crc32_pclmul snd_hda_core joydev ghash_clmulni_intel snd_hwdep intel_cstate snd_seq intel_uncore snd_seq_device
Nov 05 05:27:32 kernel:  intel_rapl_perf snd_pcm pcspkr i2c_i801 snd_timer snd lpc_ich ses enclosure scsi_transport_sas soundcore nfsd nfs_acl lockd auth_rpcgss grace sunrpc binfmt_misc ip_tables raid456 libcrc32c async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 crc32c_intel igb megaraid_sas dca i2c_algo_bit wmi target_core_mod vhost_net tun tap vhost fuse ecryptfs [last unloaded: ipmi_msghandler]
Nov 05 05:27:32 kernel: CPU: 10 PID: 3983 Comm: pmdalinux Kdump: loaded Tainted: P      D    OE     5.3.8-300.fc31.x86_64 #1
Nov 05 05:27:32 kernel: Hardware name: [REDACTED]
Nov 05 05:27:32 kernel: RIP: 0010:smp_call_function_single+0x99/0x110
Nov 05 05:27:32 kernel: Code: 74 76 65 8b 05 90 a6 e9 48 a9 00 01 1f 00 75 79 85 c9 75 40 48 c7 c6 00 98 02 00 65 48 03 35 46 3e e9 48 8b 46 18 a8 01 74 09 <f3> 90 8b 46 18 a8 01 75 f7 83 4e 18 01 4c 89 c9 4c 89 c2 e8 6f fe
Nov 05 05:27:32 kernel: RSP: 0018:ffffa2e343b6fbe0 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
Nov 05 05:27:32 kernel: RAX: 0000000000000001 RBX: 0000000b111cf8cf RCX: 0000000000000000
Nov 05 05:27:32 kernel: RDX: 0000000000000000 RSI: ffff93627faa9800 RDI: 0000000000000003
Nov 05 05:27:32 kernel: RBP: ffffa2e343b6fc38 R08: ffffffffb703ddf0 R09: 0000000000000000
Nov 05 05:27:32 kernel: R10: ffff93626f7370c0 R11: 0000000000000000 R12: 000007aeb10c542b
Nov 05 05:27:32 kernel: R13: 0000000000000001 R14: ffff935d95042500 R15: ffff935d95042500
Nov 05 05:27:32 kernel: FS:  00007fe92cded900(0000) GS:ffff93627fa80000(0000) knlGS:0000000000000000
Nov 05 05:27:32 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 05 05:27:32 kernel: CR2: 00007f56be1b6c30 CR3: 0000002f4fdec004 CR4: 00000000001606e0
Nov 05 05:27:32 kernel: Call Trace:
Nov 05 05:27:32 kernel:  ? xattr_resolve_name+0xa0/0xc0
Nov 05 05:27:32 kernel:  ? recalibrate_cpu_khz+0x10/0x10
Nov 05 05:27:32 kernel:  ? ktime_get+0x3c/0x90
Nov 05 05:27:32 kernel:  aperfmperf_snapshot_cpu+0x40/0x50
Nov 05 05:27:32 kernel:  arch_freq_prepare_all+0x5e/0xa0
Nov 05 05:27:32 kernel:  cpuinfo_open+0xe/0x20
Nov 05 05:27:32 kernel:  proc_reg_open+0x6f/0x130
Nov 05 05:27:32 kernel:  ? proc_put_link+0x10/0x10
Nov 05 05:27:32 kernel:  do_dentry_open+0x13a/0x380
Nov 05 05:27:32 kernel:  path_openat+0x591/0x1470
Nov 05 05:27:32 kernel:  do_filp_open+0x91/0x100
Nov 05 05:27:32 kernel:  ? __check_object_size+0x136/0x147
Nov 05 05:27:32 kernel:  ? audit_alloc_name+0x8c/0xe0
Nov 05 05:27:32 kernel:  ? __alloc_fd+0x3d/0x140
Nov 05 05:27:32 kernel:  do_sys_open+0x184/0x220
Nov 05 05:27:32 kernel:  do_syscall_64+0x5f/0x1a0
Nov 05 05:27:32 kernel:  entry_SYSCALL_64_after_hwframe+0x44/0xa9
Nov 05 05:27:32 kernel: RIP: 0033:0x7fe92daf712b
Nov 05 05:27:32 kernel: Code: 25 00 00 41 00 3d 00 00 41 00 74 4b 64 8b 04 25 18 00 00 00 85 c0 75 67 44 89 e2 48 89 ee bf 9c ff ff ff b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 91 00 00 00 48 8b 4c 24 28 64 48 33 0c 25
Nov 05 05:27:32 kernel: RSP: 002b:00007ffe47d867f0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
Nov 05 05:27:32 kernel: RAX: ffffffffffffffda RBX: 0000560b18975be0 RCX: 00007fe92daf712b
Nov 05 05:27:32 kernel: RDX: 0000000000000000 RSI: 00007ffe47d86990 RDI: 00000000ffffff9c
Nov 05 05:27:32 kernel: RBP: 00007ffe47d86990 R08: 0000000000000008 R09: 0000000000000001
Nov 05 05:27:32 kernel: R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
Nov 05 05:27:32 kernel: R13: 0000560b18975be0 R14: 0000000000000001 R15: 00007ffe47d87a80

Comment 5 Patrick Dung 2019-11-09 03:59:22 UTC
Included bug report in upstream

Comment 6 Dmitrij S. Kryzhevich 2019-11-19 04:37:02 UTC
Have the same (or similar?) issue. System freez randomly after upgrade to f31. Usage of f30 kernel does not help. And i didn't swith the scelduer. Journal conatains the following:
nov 19 10:53:55 bear.pdc.lkkm kernel: BUG: kernel NULL pointer dereference, address: 0000000000000014
nov 19 10:53:55 bear.pdc.lkkm kernel: #PF: supervisor read access in kernel mode
nov 19 10:53:55 bear.pdc.lkkm kernel: #PF: error_code(0x0000) - not-present page
nov 19 10:53:55 bear.pdc.lkkm kernel: PGD 0 P4D 0
nov 19 10:53:56 bear.pdc.lkkm kernel: Oops: 0000 [#1] SMP PTI
nov 19 10:53:56 bear.pdc.lkkm kernel: CPU: 14 PID: 216 Comm: kworker/14:1 Tainted: P           OE     5.3.11-300.fc31.x86_64 #1
nov 19 10:53:56 bear.pdc.lkkm kernel: Hardware name: ASUS All Series/X99-A, BIOS 3801 08/10/2017
nov 19 10:53:56 bear.pdc.lkkm kernel: Workqueue: events key_garbage_collector
nov 19 10:53:56 bear.pdc.lkkm kernel: RIP: 0010:keyring_gc_check_iterator+0x2c/0x40
nov 19 10:53:56 bear.pdc.lkkm kernel: Code: 44 00 00 48 83 e7 fc b8 01 00 00 00 f6 87 80 00 00 00 21 75 19 48 8b 57 58 48 39 16 7c 05 48 85 d2 7f 0b 48 8b 87 a0 00 00 00 <0f> b6 40 14 c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f
nov 19 10:53:56 bear.pdc.lkkm kernel: RSP: 0018:ffffbbc3016e7dd8 EFLAGS: 00010246
nov 19 10:53:56 bear.pdc.lkkm kernel: RAX: 0000000000000000 RBX: ffff9636b239c1c8 RCX: ffffbbc3016e7e20
nov 19 10:53:56 bear.pdc.lkkm kernel: RDX: 0000000000000000 RSI: ffffbbc3016e7e20 RDI: ffff963895da2800
nov 19 10:53:56 bear.pdc.lkkm kernel: RBP: ffffbbc3016e7e20 R08: ffff96387dd04a08 R09: 000000000000000f
nov 19 10:53:56 bear.pdc.lkkm kernel: R10: ffff96389ba3472c R11: 0000000000000018 R12: ffffffff91427f10
nov 19 10:53:56 bear.pdc.lkkm kernel: R13: ffff9636b239c210 R14: ffff963effffff41 R15: ffff9636b239c180
nov 19 10:53:56 bear.pdc.lkkm kernel: FS:  0000000000000000(0000) GS:ffff96389fb80000(0000) knlGS:0000000000000000
nov 19 10:53:56 bear.pdc.lkkm kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
nov 19 10:53:56 bear.pdc.lkkm kernel: CR2: 0000000000000014 CR3: 00000007ee026006 CR4: 00000000001626e0
nov 19 10:53:56 bear.pdc.lkkm kernel: Call Trace:
nov 19 10:53:56 bear.pdc.lkkm kernel:  assoc_array_subtree_iterate+0x57/0xd0
nov 19 10:53:56 bear.pdc.lkkm kernel:  keyring_gc+0x3d/0x80
nov 19 10:53:56 bear.pdc.lkkm kernel:  key_garbage_collector+0x363/0x3d0
nov 19 10:53:56 bear.pdc.lkkm kernel:  ? __schedule+0x2a7/0x680
nov 19 10:53:56 bear.pdc.lkkm kernel:  process_one_work+0x19d/0x340
nov 19 10:53:56 bear.pdc.lkkm kernel:  worker_thread+0x50/0x3b0
nov 19 10:53:56 bear.pdc.lkkm kernel:  kthread+0xfb/0x130
nov 19 10:53:56 bear.pdc.lkkm kernel:  ? process_one_work+0x340/0x340
nov 19 10:53:56 bear.pdc.lkkm kernel:  ? kthread_park+0x80/0x80
nov 19 10:53:56 bear.pdc.lkkm kernel:  ret_from_fork+0x35/0x40
nov 19 10:53:56 bear.pdc.lkkm kernel: Modules linked in: 8021q garp mrp stp llc nfsv4 dns_resolver nfs fscache vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) rp csec_gss_krb5 vfat fat nvidia_drm(POE) nvidia_modeset(POE) nvidia_uvm(OE) nvidia(POE) snd_hda_codec_hdmi snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio snd_hda_intel intel_rapl_msr intel_rapl_common snd_hda_codec joydev x86_pkg_temp_thermal intel_powerclamp snd_hda_core coretemp kvm_intel snd_hwdep snd_seq snd_seq_device snd_pcm kvm snd_timer drm_kms_helper drm ipmi_devintf ipmi_msghandler irqbypass crct10dif_pclmul crc32_pclmul snd raid1 ghash_clmulni_intel intel_cstate mei_me mei eee pc_wmi asus_wmi sparse_keymap rfkill video intel_wmi_thunderbolt wmi_bmof intel_uncore iTCO_wdt iTCO_vendor_support intel_rapl_perf i2c_i801 soundcore e1000e lpc_ich nfsd auth_rpcgss nfs_acl lockd grace sunrpc ip_tables uas usb_storage mxm_wmi crc32c_intel wmi fuse [last unloaded: vboxdrv]
nov 19 10:53:56 bear.pdc.lkkm kernel: CR2: 0000000000000014
nov 19 10:53:56 bear.pdc.lkkm kernel: ---[ end trace 86d8799f1807a8d4 ]---
nov 19 10:53:56 bear.pdc.lkkm kernel: RIP: 0010:keyring_gc_check_iterator+0x2c/0x40
nov 19 10:53:56 bear.pdc.lkkm kernel: Code: 44 00 00 48 83 e7 fc b8 01 00 00 00 f6 87 80 00 00 00 21 75 19 48 8b 57 58 48 39 16 7c 05 48 85 d2 7f 0b 48 8b 87 a0 00 00 00 <0f> b6 40 14 c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 0f 1f
nov 19 10:53:56 bear.pdc.lkkm kernel: RSP: 0018:ffffbbc3016e7dd8 EFLAGS: 00010246
nov 19 10:53:56 bear.pdc.lkkm kernel: RAX: 0000000000000000 RBX: ffff9636b239c1c8 RCX: ffffbbc3016e7e20
nov 19 10:53:56 bear.pdc.lkkm kernel: RDX: 0000000000000000 RSI: ffffbbc3016e7e20 RDI: ffff963895da2800
nov 19 10:53:56 bear.pdc.lkkm kernel: RBP: ffffbbc3016e7e20 R08: ffff96387dd04a08 R09: 000000000000000f
nov 19 10:53:56 bear.pdc.lkkm kernel: R10: ffff96389ba3472c R11: 0000000000000018 R12: ffffffff91427f10
nov 19 10:53:56 bear.pdc.lkkm kernel: R13: ffff9636b239c210 R14: ffff963effffff41 R15: ffff9636b239c180
nov 19 10:53:56 bear.pdc.lkkm kernel: FS:  0000000000000000(0000) GS:ffff96389fb80000(0000) knlGS:0000000000000000
nov 19 10:53:56 bear.pdc.lkkm kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
nov 19 10:53:56 bear.pdc.lkkm kernel: CR2: 0000000000000014 CR3: 00000007ee026006 CR4: 00000000001626e0

Aditional info.
# inxi -Fx
System:    Host: bear.pdc.lkkm Kernel: 5.3.11-300.fc31.x86_64 x86_64 bits: 64 compiler: gcc v: 9.2.1 Console: tty 2 
           Distro: Fedora release 31 (Thirty One) 
Machine:   Type: Desktop System: ASUS product: All Series v: N/A serial: N/A 
           Mobo: ASUSTeK model: X99-A v: Rev 1.xx serial: 140932111700268 UEFI: American Megatrends v: 3801 date: 08/10/2017 
CPU:       Topology: 8-Core model: Intel Core i7-5960X bits: 64 type: MT MCP arch: Haswell rev: 2 L2 cache: 20.0 MiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 112008 
           Speed: 1500 MHz min/max: 1200/2800 MHz Core speeds (MHz): 1: 1500 2: 1502 3: 1501 4: 1508 5: 1504 6: 1505 7: 1502 
           8: 1503 9: 1501 10: 1500 11: 1501 12: 1501 13: 1501 14: 1500 15: 1507 16: 1500 
Graphics:  Device-1: NVIDIA GK107 [GeForce GTX 650] vendor: ASUSTeK GTX650-DC-1GD5 driver: nvidia v: 440.31 bus ID: 02:00.0 
           Display: server: Fedora Project X.org 1.20.5 driver: nvidia unloaded: fbdev,modesetting,nouveau,vesa 
           resolution: 1920x1200~60Hz, 1920x1200~60Hz 
           OpenGL: renderer: GeForce GTX 650/PCIe/SSE2 v: 4.6.0 NVIDIA 440.31 direct render: Yes 
Audio:     Device-1: Intel C610/X99 series HD Audio vendor: ASUSTeK driver: snd_hda_intel v: kernel bus ID: 00:1b.0 
           Device-2: NVIDIA GK107 HDMI Audio vendor: ASUSTeK GTX650-DC-1GD5 driver: snd_hda_intel v: kernel bus ID: 02:00.1 
           Sound Server: ALSA v: k5.3.11-300.fc31.x86_64 
Network:   Device-1: Intel Ethernet I218-V vendor: ASUSTeK driver: e1000e v: 3.2.6-k port: f000 bus ID: 00:19.0 
           IF: eno1 state: up speed: 1000 Mbps duplex: full mac: 38:2c:4a:6d:e5:84 
Drives:    Local Storage: total: 10.01 TiB used: 5.55 TiB (55.5%) 
           ID-1: /dev/sda vendor: Western Digital model: WD10EFRX-68PJCN0 size: 931.51 GiB temp: 34 C 
           ID-2: /dev/sdb vendor: HGST (Hitachi) model: HDN724030ALE640 size: 2.73 TiB temp: 44 C 
           ID-3: /dev/sdc vendor: Seagate model: ST4000DM004-2CV104 size: 3.64 TiB 
           ID-4: /dev/sdd vendor: HGST (Hitachi) model: HDN724030ALE640 size: 2.73 TiB temp: 43 C 
RAID:      Device-1: md127 type: mdraid status: active Components: online: sdd1~c1 sdb1~c0 
           Info: raid: mirror blocks: 2930134016 report: 2/2 UU chunk size: N/A 
Partition: ID-1: / size: 48.97 GiB used: 13.69 GiB (28.0%) fs: ext4 dev: /dev/sda4 
           ID-2: /boot size: 476.2 MiB used: 187.0 MiB (39.3%) fs: ext4 dev: /dev/sda2 
           ID-3: swap-1 size: 15.69 GiB used: 0 KiB (0.0%) fs: swap dev: /dev/sda5 
Sensors:   System Temperatures: cpu: 39.0 C mobo: N/A gpu: nvidia temp: 35 C 
           Fan Speeds (RPM): cpu: 0 gpu: nvidia fan: 10% 
Info:      Processes: 367 Uptime: 27m Memory: 31.28 GiB used: 1.80 GiB (5.7%) Init: systemd runlevel: 5 Compilers: gcc: 9.2.1 
           Shell: bash v: 5.0.7 inxi: 3.0.36

Additioanl info #2. I did try to move disk with system to another machine with the same CPU model (and moved GPU as that machine do not has it at all). Freezes continue. Memory was tested, all good.
I see reports on system freez from users with AMD graphics but I have Nvidia one.

It's very, very anoying to restart machine every 2-3 hours.

Comment 7 Dmitrij S. Kryzhevich 2019-11-19 04:44:27 UTC
The main difference from #1: I have only one kind of workqueue entry for all freezes if present: "Workqueue: events key_garbage_collector". It can absent still (rare).

May be I should make new BR?

Comment 8 Greg Oster 2019-11-22 19:40:46 UTC
Did you file a new BR?  If so, I'll add my "me too" there.

Nov 21 19:00:55 tux5 kernel: BUG: kernel NULL pointer dereference, address: 0000000000000014
Nov 21 19:00:55 tux5 kernel: #PF: supervisor read access in kernel mode
Nov 21 19:00:55 tux5 kernel: #PF: error_code(0x0000) - not-present page
Nov 21 19:00:55 tux5 kernel: PGD 0 P4D 0 
Nov 21 19:00:55 tux5 kernel: Oops: 0000 [#1] SMP PTI
Nov 21 19:00:55 tux5 kernel: CPU: 19 PID: 42945 Comm: kworker/19:1 Tainted: G           OE     5.3.8-200.fc30.x86_64 #1
Nov 21 19:00:55 tux5 kernel: Hardware name: Dell Inc. PowerEdge R430/0HFG24, BIOS 1.6.2 01/08/2016
Nov 21 19:00:55 tux5 kernel: Workqueue: events key_garbage_collector
Nov 21 19:00:55 tux5 kernel: RIP: 0010:keyring_gc_select_iterator+0x2f/0x60
Nov 21 19:00:55 tux5 kernel: Code: 48 83 e7 fc 31 c0 f6 87 80 00 00 00 21 75 3a 48 8b 57 58 48 39 16 0f 9d c0 48 85 d2 0f 9f c2 20 d0 75 27 48 8b 97 a0 00 00 00 <80> 7a 14 00 75 19 48 85 ff 74 0f f0 ff 07 0f 88 8e d2 5a 00 b8 01
Nov 21 19:00:55 tux5 kernel: RSP: 0018:ffffbdb10964fda8 EFLAGS: 00010246
Nov 21 19:00:55 tux5 kernel: RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000
Nov 21 19:00:55 tux5 kernel: RDX: 0000000000000000 RSI: ffffbdb10964fe20 RDI: ffff9d8d23b65b00
Nov 21 19:00:55 tux5 kernel: RBP: ffff9d8d3970b980 R08: ffff9d8d7f86d100 R09: ffff9d8d3970b980
Nov 21 19:00:55 tux5 kernel: R10: ffff9d8d3970b0ec R11: 0000000000000018 R12: ffff9d8d3970b981
Nov 21 19:00:55 tux5 kernel: R13: ffff9d8d23b65b00 R14: ffff9d8d22ba89c0 R15: 0000000000000001
Nov 21 19:00:55 tux5 kernel: FS:  0000000000000000(0000) GS:ffff9d8d7f840000(0000) knlGS:0000000000000000
Nov 21 19:00:55 tux5 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 21 19:00:55 tux5 kernel: CR2: 0000000000000014 CR3: 0000001fa208c001 CR4: 00000000001606e0
Nov 21 19:00:55 tux5 kernel: Call Trace:
Nov 21 19:00:55 tux5 kernel:  assoc_array_gc+0x16b/0x4b0
Nov 21 19:00:55 tux5 kernel:  ? key_unlink+0xa0/0xa0
Nov 21 19:00:55 tux5 kernel:  ? keyring_detect_cycle_iterator+0x20/0x20
Nov 21 19:00:55 tux5 kernel:  keyring_gc+0x67/0x80
Nov 21 19:00:55 tux5 kernel:  key_garbage_collector+0x363/0x3d0
Nov 21 19:00:55 tux5 kernel:  ? __schedule+0x2a7/0x680
Nov 21 19:00:55 tux5 kernel:  process_one_work+0x19d/0x340
Nov 21 19:00:55 tux5 kernel:  worker_thread+0x50/0x3b0
Nov 21 19:00:55 tux5 kernel:  kthread+0xfb/0x130
Nov 21 19:00:55 tux5 kernel:  ? process_one_work+0x340/0x340
Nov 21 19:00:55 tux5 kernel:  ? kthread_park+0x80/0x80
Nov 21 19:00:55 tux5 kernel:  ret_from_fork+0x35/0x40
Nov 21 19:00:55 tux5 kernel: Modules linked in: rpcsec_gss_krb5 nfsv4 dns_resolver xt_CHECKSUM xt_MASQUERADE tun bridge stp llc xt_multiport nfsv3 nfs_acl nfs lockd grace fscache cfg80211 rfkill nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_rpfilter ip6t_REJECT nf_reject_ipv6 ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter vboxpci(OE) vboxnetadp(OE) vboxnetflt(OE) vboxdrv(OE) rpcrdma ib_isert iscsi_target_mod ib_iser ib_srpt target_core_mod ib_srp scsi_transport_srp ib_ipoib rdma_ucm ib_umad vfat fat iw_cxgb4 rdma_cm iw_cm ib_cm iw_cxgb3 ib_uverbs ib_core intel_rapl_msr intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm ipmi_ssif irqbypass pktcdvd intel_cstate ipmi_si intel_uncore mei_me
Nov 21 19:00:55 tux5 kernel:  ipmi_devintf intel_rapl_perf iTCO_wdt ipmi_msghandler iTCO_vendor_support dcdbas mei lpc_ich acpi_power_meter auth_rpcgss ip_tables mgag200 drm_vram_helper i2c_algo_bit drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ttm drm ghash_clmulni_intel tg3 megaraid_sas wmi dm_multipath sunrpc be2iscsi bnx2i cnic uio cxgb4i cxgb4 cxgb3i cxgb3 mdio libcxgbi libcxgb qla4xxx iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi fuse
Nov 21 19:00:55 tux5 kernel: CR2: 0000000000000014
Nov 21 19:00:55 tux5 kernel: ---[ end trace 92e8aff0517b4b77 ]---
Nov 21 19:00:55 tux5 kernel: RIP: 0010:keyring_gc_select_iterator+0x2f/0x60
Nov 21 19:00:55 tux5 kernel: Code: 48 83 e7 fc 31 c0 f6 87 80 00 00 00 21 75 3a 48 8b 57 58 48 39 16 0f 9d c0 48 85 d2 0f 9f c2 20 d0 75 27 48 8b 97 a0 00 00 00 <80> 7a 14 00 75 19 48 85 ff 74 0f f0 ff 07 0f 88 8e d2 5a 00 b8 01
Nov 21 19:00:55 tux5 kernel: RSP: 0018:ffffbdb10964fda8 EFLAGS: 00010246
Nov 21 19:00:55 tux5 kernel: RAX: 0000000000000000 RBX: 0000000000000003 RCX: 0000000000000000
Nov 21 19:00:55 tux5 kernel: RDX: 0000000000000000 RSI: ffffbdb10964fe20 RDI: ffff9d8d23b65b00
Nov 21 19:00:55 tux5 kernel: RBP: ffff9d8d3970b980 R08: ffff9d8d7f86d100 R09: ffff9d8d3970b980
Nov 21 19:00:55 tux5 kernel: R10: ffff9d8d3970b0ec R11: 0000000000000018 R12: ffff9d8d3970b981
Nov 21 19:00:55 tux5 kernel: R13: ffff9d8d23b65b00 R14: ffff9d8d22ba89c0 R15: 0000000000000001
Nov 21 19:00:55 tux5 kernel: FS:  0000000000000000(0000) GS:ffff9d8d7f840000(0000) knlGS:0000000000000000
Nov 21 19:00:55 tux5 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Nov 21 19:00:55 tux5 kernel: CR2: 0000000000000014 CR3: 0000001fa208c001 CR4: 00000000001606e0

The hardware is completely different (Dell R430).

Comment 9 Dmitrij S. Kryzhevich 2019-11-26 07:44:08 UTC
(In reply to Greg Oster from comment #8)
> Did you file a new BR?  If so, I'll add my "me too" there.
> 

See https://bugzilla.redhat.com/show_bug.cgi?id=1776572

Comment 10 Justin M. Forbes 2020-03-03 16:21:50 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 31 kernel bugs.

Fedora 31 has now been rebased to 5.5.7-200.fc31.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 32, and are still experiencing this issue, please change the version to Fedora 32.

If you experience different issues, please open a new bug report for those.

Comment 11 Patrick Dung 2020-03-03 17:56:45 UTC
This bug report is believed to be a duplicate of 1767539.
https://bugzilla.redhat.com/show_bug.cgi?id=1767539

Comment 12 Justin M. Forbes 2020-03-03 19:51:14 UTC

*** This bug has been marked as a duplicate of bug 1767539 ***


Note You need to log in before you can comment on or make changes to this bug.