Bug 1758218 - cockpit/remotectl OOM out of memory errors, Bad RIP value; hardware tests passed
Summary: cockpit/remotectl OOM out of memory errors, Bad RIP value; hardware tests passed
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: cockpit
Version: 33
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Kernel Maintainer List
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-10-03 15:05 UTC by RobbieTheK
Modified: 2021-05-06 02:27 UTC (History)
20 users (show)

Fixed In Version: cockpit-242-1.fc34
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-04-24 20:11:52 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github cockpit-project cockpit issues 12949 0 'None' open remotectl OOM out of memory errors 2020-11-03 21:30:32 UTC

Description RobbieTheK 2019-10-03 15:05:53 UTC
I’ve run all firmware updates, dnf updates and still OOM’s on various processes a couple times an hour. Could this be a bug? The 5 hour hardware test within the Dell BIOS passed I can provide a see screen shot, no errors. No errors in the Dell debug logs within the Lifecyle Controller. Kernel is 5.2.17-200.fc30.x86_6 but it’s been happening for a few kernel versions. 

Oct  3 10:29:01 mandelbrot kernel: x2gocleansessio: page allocation failure: order:0, mode:0x800(GFP_NOWAIT), nodemask=(null),cpuset=/,mems_allowed=0-1
Oct  3 10:29:01 mandelbrot kernel: CPU: 4 PID: 1436 Comm: x2gocleansessio Not tainted 5.2.17-200.fc30.x86_64 #1
Oct  3 10:29:01 mandelbrot kernel: Hardware name: Dell Inc. PowerEdge R740/00WGD1, BIOS 2.3.10 08/15/2019
Oct  3 10:29:01 mandelbrot kernel: Call Trace:
Oct  3 10:29:01 mandelbrot kernel: dump_stack+0x5c/0x80
Oct  3 10:29:01 mandelbrot kernel: warn_alloc.cold+0x7b/0xfb
Oct  3 10:29:01 mandelbrot kernel: __alloc_pages_slowpath+0xdb7/0xde0
Oct  3 10:29:01 mandelbrot kernel: ? __switch_to_asm+0x34/0x70
Oct  3 10:29:01 mandelbrot kernel: ? sched_clock+0x5/0x10
Oct  3 10:29:01 mandelbrot kernel: ? sched_clock_cpu+0xc/0xa0
Oct  3 10:29:01 mandelbrot kernel: __alloc_pages_nodemask+0x2ee/0x340
Oct  3 10:29:01 mandelbrot kernel: new_slab+0x3e4/0x6d0
Oct  3 10:29:01 mandelbrot kernel: ___slab_alloc+0x507/0x6a0
Oct  3 10:29:01 mandelbrot kernel: ? alloc_vmap_area+0x31f/0x830
Oct  3 10:29:01 mandelbrot kernel: ? security_inode_alloc+0x24/0x80
Oct  3 10:29:01 mandelbrot kernel: ? __follow_mount_rcu.isra.0+0x5c/0xe0
Oct  3 10:29:01 mandelbrot kernel: ? alloc_vmap_area+0x31f/0x830
Oct  3 10:29:01 mandelbrot kernel: __slab_alloc+0x1c/0x30
Oct  3 10:29:01 mandelbrot kernel: kmem_cache_alloc+0x18b/0x1c0
Oct  3 10:29:01 mandelbrot kernel: alloc_vmap_area+0x31f/0x830
Oct  3 10:29:01 mandelbrot kernel: ? _cond_resched+0x15/0x30
Oct  3 10:29:02 mandelbrot kernel: ? kmem_cache_alloc_node_trace+0x1c4/0x200
Oct  3 10:29:02 mandelbrot kernel: __get_vm_area_node+0xb2/0x160
Oct  3 10:29:02 mandelbrot kernel: ? page_trans_huge_mapcount+0x12/0xf0
Oct  3 10:29:02 mandelbrot kernel: __vmalloc_node_range+0x6d/0x280
Oct  3 10:29:02 mandelbrot kernel: ? _do_fork+0xdf/0x3a0
Oct  3 10:29:02 mandelbrot kernel: ? memcg_kmem_get_cache+0x65/0x1b0
Oct  3 10:29:02 mandelbrot kernel: copy_process.part.0+0x91c/0x1a30
Oct  3 10:29:02 mandelbrot kernel: ? _do_fork+0xdf/0x3a0
Oct  3 10:29:02 mandelbrot kernel: ? security_file_alloc+0x24/0x90
Oct  3 10:29:02 mandelbrot kernel: _do_fork+0xdf/0x3a0
Oct  3 10:29:02 mandelbrot kernel: ? sigprocmask+0x72/0xa0
Oct  3 10:29:02 mandelbrot kernel: do_syscall_64+0x5f/0x1a0
Oct  3 10:29:02 mandelbrot kernel: ? page_fault+0x8/0x30
Oct  3 10:29:02 mandelbrot kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Oct  3 10:29:02 mandelbrot kernel: RIP: 0033:0x7fadee8a4c8f
Oct  3 10:29:02 mandelbrot kernel: Code: ed 0f 85 f4 00 00 00 64 4c 8b 0c 25 10 00 00 00 45 31 c0 4d 8d 91 d0 02 00 00 31 d2 31 f6 bf 11 00 20 01 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 8d 00 00 00 41 89 c5 85 c0 0f 85 9a 00 00
Oct  3 10:29:02 mandelbrot kernel: RSP: 002b:00007ffe12cf3280 EFLAGS: 00000246 ORIG_RAX: 0000000000000038
Oct  3 10:29:02 mandelbrot kernel: RAX: ffffffffffffffda RBX: 0000560aece33a70 RCX: 00007fadee8a4c8f
Oct  3 10:29:02 mandelbrot kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011
Oct  3 10:29:02 mandelbrot kernel: RBP: 0000000000000000 R08: 0000000000000000 R09: 00007fadee62f740
Oct  3 10:29:02 mandelbrot kernel: R10: 00007fadee62fa10 R11: 0000000000000246 R12: 0000000000000000
Oct  3 10:29:02 mandelbrot kernel: R13: 00007ffe12cf32e0 R14: 00007ffe12cf3360 R15: 0000560aece33a80
Oct  3 10:29:02 mandelbrot kernel: Mem-Info:
Oct  3 10:29:02 mandelbrot kernel: active_anon:15223312 inactive_anon:896743 isolated_anon:96#012 active_file:5499 inactive_file:6235 isolated_file:0#012 unevictable:0 dirty:0 writeback:5325 unstable:0#012 slab_reclaimable:11869 slab_unreclaimable:32871#012 mapped:6831 shmem:4 pagetables:40763 bounce:0#012 free:56492 free_pcp:384 free_cma:0
Oct  3 10:29:02 mandelbrot kernel: Node 0 active_anon:30049500kB inactive_anon:1770108kB active_file:2968kB inactive_file:5144kB unevictable:0kB isolated(anon):256kB isolated(file):0kB mapped:4100kB dirty:0kB writeback:6988kB shmem:12kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Oct  3 10:29:02 mandelbrot kernel: Node 1 active_anon:30843748kB inactive_anon:1816864kB active_file:19028kB inactive_file:19796kB unevictable:0kB isolated(anon):128kB isolated(file):0kB mapped:23224kB dirty:0kB writeback:14312kB shmem:4kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Oct  3 10:29:02 mandelbrot kernel: Node 0 DMA free:15872kB min:20kB low:32kB high:44kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15972kB managed:15888kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:02 mandelbrot kernel: lowmem_reserve[]: 0 1396 31608 31608 31608
Oct  3 10:29:02 mandelbrot kernel: Node 0 DMA32 free:122816kB min:1968kB low:3396kB high:4824kB active_anon:1324200kB inactive_anon:4kB active_file:8kB inactive_file:0kB unevictable:0kB writepending:0kB present:1521664kB managed:1456124kB mlocked:0kB kernel_stack:0kB pagetables:2592kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:02 mandelbrot kernel: lowmem_reserve[]: 0 0 30212 30212 30212
Oct  3 10:29:02 mandelbrot kernel: Node 0 Normal free:42180kB min:42640kB low:73576kB high:104512kB active_anon:28725056kB inactive_anon:1770300kB active_file:2960kB inactive_file:5144kB unevictable:0kB writepending:6784kB present:31457280kB managed:30937480kB mlocked:0kB kernel_stack:2936kB pagetables:77624kB bounce:0kB free_pcp:1616kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:02 mandelbrot kernel: lowmem_reserve[]: 0 0 0 0 0
Oct  3 10:29:02 mandelbrot kernel: Node 1 Normal free:45100kB min:45476kB low:78468kB high:111460kB active_anon:30843916kB inactive_anon:1816728kB active_file:19028kB inactive_file:19796kB unevictable:0kB writepending:14292kB present:33554432kB managed:33003216kB mlocked:0kB kernel_stack:3224kB pagetables:82836kB bounce:0kB free_pcp:48kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:02 mandelbrot kernel: lowmem_reserve[]: 0 0 0 0 0
Oct  3 10:29:02 mandelbrot kernel: Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15872kB
Oct  3 10:29:02 mandelbrot kernel: Node 0 DMA32: 2*4kB (M) 3*8kB (U) 8*16kB (UM) 5*32kB (UM) 4*64kB (U) 3*128kB (U) 4*256kB (UM) 6*512kB (UM) 5*1024kB (U) 1*2048kB (U) 27*4096kB (M) = 122816kB
Oct  3 10:29:02 mandelbrot kernel: Node 0 Normal: 87*4kB (UE) 61*8kB (UE) 106*16kB (UE) 84*32kB (UME) 28*64kB (UE) 10*128kB (UME) 6*256kB (UME) 0*512kB 2*1024kB (UE) 1*2048kB (M) 7*4096kB (M) = 42596kB
Oct  3 10:29:02 mandelbrot kernel: Node 1 Normal: 121*4kB (UME) 96*8kB (UME) 81*16kB (UME) 63*32kB (UME) 44*64kB (UME) 17*128kB (UME) 5*256kB (UME) 6*512kB (UME) 4*1024kB (UE) 2*2048kB (ME) 6*4096kB (M) = 46676kB
Oct  3 10:29:02 mandelbrot kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct  3 10:29:02 mandelbrot kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  3 10:29:02 mandelbrot kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct  3 10:29:02 mandelbrot kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  3 10:29:02 mandelbrot kernel: 21742 total pagecache pages
Oct  3 10:29:02 mandelbrot kernel: 10098 pages in swap cache
Oct  3 10:29:02 mandelbrot kernel: Swap cache stats: add 4086896, delete 4076446, find 1826/6486
Oct  3 10:29:02 mandelbrot kernel: Free swap  = 7134972kB
Oct  3 10:29:02 mandelbrot kernel: Total swap = 23441404kB
Oct  3 10:29:02 mandelbrot kernel: 16637337 pages RAM
Oct  3 10:29:02 mandelbrot kernel: 0 pages HighMem/MovableOnly
Oct  3 10:29:02 mandelbrot kernel: 284160 pages reserved
Oct  3 10:29:02 mandelbrot kernel: 0 pages cma reserved
Oct  3 10:29:02 mandelbrot kernel: 0 pages hwpoisoned
Oct  3 10:29:02 mandelbrot kernel: SLUB: Unable to allocate memory on node -1, gfp=0x800(GFP_NOWAIT)
Oct  3 10:29:02 mandelbrot kernel:  cache: vmap_area, object size: 88, buffer size: 88, default order: 0, min order: 0
Oct  3 10:29:02 mandelbrot kernel:  node 0: slabs: 70, objs: 3220, free: 0
Oct  3 10:29:02 mandelbrot kernel:  node 1: slabs: 30, objs: 1380, free: 0
Oct  3 10:29:33 mandelbrot systemd[1]: Starting Crash recovery kernel arming...
Oct  3 10:29:34 mandelbrot kernel: systemd invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Oct  3 10:29:34 mandelbrot kernel: CPU: 5 PID: 2323 Comm: systemd Not tainted 5.2.17-200.fc30.x86_64 #1
Oct  3 10:29:34 mandelbrot kernel: Hardware name: Dell Inc. PowerEdge R740/00WGD1, BIOS 2.3.10 08/15/2019
Oct  3 10:29:34 mandelbrot kernel: Call Trace:
Oct  3 10:29:34 mandelbrot kernel: dump_stack+0x5c/0x80
Oct  3 10:29:34 mandelbrot kernel: dump_header+0x51/0x2ff
Oct  3 10:29:34 mandelbrot kernel: ? sched_clock+0x5/0x10
Oct  3 10:29:34 mandelbrot kernel: ? oom_unkillable_task+0x9a/0xd0
Oct  3 10:29:34 mandelbrot kernel: oom_kill_process.cold+0xb/0x10
Oct  3 10:29:34 mandelbrot kernel: out_of_memory+0x1d4/0x480
Oct  3 10:29:34 mandelbrot kernel: __alloc_pages_slowpath+0xcc4/0xde0
Oct  3 10:29:34 mandelbrot kernel: __alloc_pages_nodemask+0x2ee/0x340
Oct  3 10:29:34 mandelbrot kernel: pagecache_get_page+0xb4/0x370
Oct  3 10:29:34 mandelbrot kernel: filemap_fault+0x683/0x9f0
Oct  3 10:29:34 mandelbrot kernel: ? __mod_lruvec_state+0x3f/0xe0
Oct  3 10:29:34 mandelbrot kernel: ? page_add_file_rmap+0xf4/0x230
Oct  3 10:29:34 mandelbrot kernel: ? alloc_set_pte+0x2d9/0x6b0
Oct  3 10:29:34 mandelbrot kernel: ? file_fdatawait_range+0x20/0x20
Oct  3 10:29:34 mandelbrot kernel: ? _cond_resched+0x15/0x30
Oct  3 10:29:34 mandelbrot kernel: __xfs_filemap_fault+0x6d/0x200 [xfs]
Oct  3 10:29:34 mandelbrot kernel: __do_fault+0x36/0x180
Oct  3 10:29:34 mandelbrot kernel: __handle_mm_fault+0x1091/0x1ac0
Oct  3 10:29:34 mandelbrot kernel: handle_mm_fault+0xc4/0x1e0
Oct  3 10:29:34 mandelbrot kernel: do_user_addr_fault+0x1f6/0x450
Oct  3 10:29:34 mandelbrot kernel: do_page_fault+0x33/0x120
Oct  3 10:29:34 mandelbrot kernel: ? page_fault+0x8/0x30
Oct  3 10:29:34 mandelbrot kernel: page_fault+0x1e/0x30
Oct  3 10:29:34 mandelbrot kernel: RIP: 0033:0x7f890dca8626
Oct  3 10:29:34 mandelbrot kernel: Code: Bad RIP value.
Oct  3 10:29:34 mandelbrot kernel: RSP: 002b:00007ffdcd26b1b0 EFLAGS: 00010206
Oct  3 10:29:34 mandelbrot kernel: RAX: 00007f890cefb830 RBX: 0000561d3f9ff320 RCX: 6c7463706d75646b
Oct  3 10:29:34 mandelbrot kernel: RDX: 00007f890cefc2e0 RSI: 6c7463706d75646b RDI: 00007f890de59de8
Oct  3 10:29:34 mandelbrot kernel: RBP: 00007ffdcd26b400 R08: 000000000003ffff R09: 0000561d3f9ffc28
Oct  3 10:29:34 mandelbrot kernel: R10: 00007f890cefb980 R11: 0000000000000206 R12: 00000000ffffffea
Oct  3 10:29:34 mandelbrot kernel: R13: 0000000000000009 R14: 00007ffdcd26b330 R15: 00007ffdcd26b330
Oct  3 10:29:34 mandelbrot kernel: Mem-Info:
Oct  3 10:29:34 mandelbrot kernel: active_anon:15231097 inactive_anon:897593 isolated_anon:0#012 active_file:146 inactive_file:43 isolated_file:0#012 unevictable:0 dirty:0 writeback:26 unstable:0#012 slab_reclaimable:11892 slab_unreclaimable:32443#012 mapped:111 shmem:2 pagetables:44275 bounce:0#012 free:56861 free_pcp:0 free_cma:0
Oct  3 10:29:34 mandelbrot kernel: Node 0 active_anon:30050612kB inactive_anon:1771372kB active_file:120kB inactive_file:132kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:20kB dirty:0kB writeback:104kB shmem:4kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
Oct  3 10:29:34 mandelbrot kernel: Node 1 active_anon:30873776kB inactive_anon:1819000kB active_file:464kB inactive_file:40kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:424kB dirty:0kB writeback:0kB shmem:4kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
Oct  3 10:29:34 mandelbrot kernel: Node 0 DMA free:15872kB min:20kB low:32kB high:44kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15972kB managed:15888kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:34 mandelbrot kernel: lowmem_reserve[]: 0 1396 31608 31608 31608
Oct  3 10:29:34 mandelbrot kernel: Node 0 DMA32 free:122808kB min:1968kB low:3396kB high:4824kB active_anon:1324208kB inactive_anon:0kB active_file:0kB inactive_file:8kB unevictable:0kB writepending:0kB present:1521664kB managed:1456124kB mlocked:0kB kernel_stack:0kB pagetables:2592kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:34 mandelbrot kernel: lowmem_reserve[]: 0 0 30212 30212 30212
Oct  3 10:29:34 mandelbrot kernel: Node 0 Normal free:42832kB min:42640kB low:73576kB high:104512kB active_anon:28726276kB inactive_anon:1771336kB active_file:124kB inactive_file:44kB unevictable:0kB writepending:396kB present:31457280kB managed:30937480kB mlocked:0kB kernel_stack:2952kB pagetables:83868kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:34 mandelbrot kernel: lowmem_reserve[]: 0 0 0 0 0
Oct  3 10:29:34 mandelbrot kernel: Node 1 Normal free:45932kB min:45476kB low:78468kB high:111460kB active_anon:30874480kB inactive_anon:1818516kB active_file:740kB inactive_file:0kB unevictable:0kB writepending:0kB present:33554432kB managed:33003216kB mlocked:0kB kernel_stack:3240kB pagetables:90640kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  3 10:29:34 mandelbrot kernel: lowmem_reserve[]: 0 0 0 0 0
Oct  3 10:29:34 mandelbrot kernel: Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15872kB
Oct  3 10:29:34 mandelbrot kernel: Node 0 DMA32: 5*4kB (UM) 4*8kB (UM) 9*16kB (UM) 6*32kB (UM) 5*64kB (UM) 4*128kB (UM) 5*256kB (UM) 5*512kB (U) 5*1024kB (U) 1*2048kB (U) 27*4096kB (M) = 122820kB
Oct  3 10:29:34 mandelbrot kernel: Node 0 Normal: 363*4kB (UME) 187*8kB (UME) 134*16kB (UME) 100*32kB (UME) 32*64kB (UME) 8*128kB (UME) 2*256kB (UE) 2*512kB (UM) 2*1024kB (UE) 2*2048kB (M) 6*4096kB (M) = 43620kB
Oct  3 10:29:34 mandelbrot kernel: Node 1 Normal: 539*4kB (UME) 140*8kB (UME) 109*16kB (UME) 57*32kB (UME) 33*64kB (UE) 15*128kB (UE) 3*256kB (ME) 2*512kB (ME) 3*1024kB (E) 1*2048kB (E) 7*4096kB (UM) = 46460kB
Oct  3 10:29:34 mandelbrot kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct  3 10:29:34 mandelbrot kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  3 10:29:34 mandelbrot kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct  3 10:29:34 mandelbrot kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  3 10:29:34 mandelbrot kernel: 3492 total pagecache pages
Oct  3 10:29:34 mandelbrot kernel: 3409 pages in swap cache
Oct  3 10:29:34 mandelbrot kernel: Swap cache stats: add 5873187, delete 5869247, find 2432/7525
Oct  3 10:29:34 mandelbrot kernel: Free swap  = 0kB
Oct  3 10:29:34 mandelbrot kernel: Total swap = 23441404kB
Oct  3 10:29:34 mandelbrot kernel: 16637337 pages RAM
Oct  3 10:29:34 mandelbrot kernel: 0 pages HighMem/MovableOnly
Oct  3 10:29:34 mandelbrot kernel: 284160 pages reserved
Oct  3 10:29:34 mandelbrot kernel: 0 pages cma reserved
Oct  3 10:29:34 mandelbrot kernel: 0 pages hwpoisoned
Oct  3 10:29:34 mandelbrot kernel: Tasks state (memory values in pages):
Oct  3 10:29:34 mandelbrot kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Oct  3 10:29:34 mandelbrot kernel: [    857]     0   857    16545       64   163840      248             0 systemd-journal
Oct  3 10:29:34 mandelbrot kernel: [    887]     0   887    38364       11    65536       85             0 lvmetad
Oct  3 10:29:34 mandelbrot kernel: [    895]     0   895     7983        1    90112      657         -1000 systemd-udevd
Oct  3 10:29:34 mandelbrot kernel: [   1308]    32  1308     2674        0    57344      182             0 rpcbind
Oct  3 10:29:34 mandelbrot kernel: [   1316]     0  1316     1560        0    49152      127             0 rpc.idmapd
Oct  3 10:29:34 mandelbrot kernel: [   1321]     0  1321   116170       17   139264      751             0 NetworkManager
Oct  3 10:29:34 mandelbrot kernel: [   1322]    70  1322     2989        0    65536      203             0 avahi-daemon
Oct  3 10:29:34 mandelbrot kernel: [   1328]     0  1328      624        0    45056       28             0 mcelog
Oct  3 10:29:34 mandelbrot kernel: [   1329]     0  1329    78329        0   114688      397             0 ModemManager
Oct  3 10:29:34 mandelbrot kernel: [   1330]     0  1330   179193        0   925696      238             0 rsyslogd
Oct  3 10:29:34 mandelbrot kernel: [   1331]     0  1331    78035        0   114688      264             0 rngd
Oct  3 10:29:34 mandelbrot kernel: [   1332]     0  1332    19842       43    53248       44             0 irqbalance
Oct  3 10:29:34 mandelbrot kernel: [   1333]     0  1333     2693        4    57344      352             0 smartd
Oct  3 10:29:34 mandelbrot kernel: [   1336]   994  1336    21125        0    65536      205             0 chronyd
Oct  3 10:29:34 mandelbrot kernel: [   1338]    70  1338     2956        0    61440      172             0 avahi-daemon
Oct  3 10:29:34 mandelbrot kernel: [   1345]    81  1345    67202        0    61440      218          -900 dbus-broker-lau
Oct  3 10:29:34 mandelbrot kernel: [   1348]  1001  1348     1438       10    49152      136             0 lmgrd
Oct  3 10:29:34 mandelbrot kernel: [   1349]  1001  1349    39782       29    77824      191             0 MLM
Oct  3 10:29:34 mandelbrot kernel: [   1354]     0  1354   114629        0   118784      485             0 abrtd
Oct  3 10:29:34 mandelbrot kernel: [   1358]     0  1358    65916        1    81920      177             0 gssproxy
Oct  3 10:29:34 mandelbrot kernel: [   1419]    81  1419     1528       16    49152      293          -900 dbus-broker
Oct  3 10:29:34 mandelbrot kernel: [   1424]     0  1424   198604       21   397312      437             0 abrt-dump-journ
Oct  3 10:29:34 mandelbrot kernel: [   1425]     0  1425   200008       25   749568      456             0 abrt-dump-journ
Oct  3 10:29:34 mandelbrot kernel: [   1426]     0  1426   202699       25   712704      440             0 abrt-dump-journ
Oct  3 10:29:34 mandelbrot kernel: [   1436]     0  1436    58908      680    86016     3128             0 x2gocleansessio
Oct  3 10:29:34 mandelbrot kernel: [   1449]   997  1449   483385        0   245760     1749             0 polkitd
Oct  3 10:29:34 mandelbrot kernel: [   1781]     0  1781     2993        0    65536      210         -1000 sshd
Oct  3 10:29:34 mandelbrot kernel: [   1782]     0  1782     9838        0   114688     6315             0 rpc.ypxfrd
Oct  3 10:29:34 mandelbrot kernel: [   1783]     0  1783     9824        0   114688     6312             0 ypserv
Oct  3 10:29:34 mandelbrot kernel: [   1793]     0  1793     9882        0   122880     6307             0 rpc.yppasswdd
Oct  3 10:29:34 mandelbrot kernel: [   1814]    29  1814    15483        0   159744    12480             0 rpc.statd
Oct  3 10:29:34 mandelbrot kernel: [   1820]     0  1820     9633        2   114688     6313             0 rpc.mountd
Oct  3 10:29:34 mandelbrot kernel: [   1845]     0  1845    89667        0   184320    12469             0 ypbind
Oct  3 10:29:34 mandelbrot kernel: [   1885]     0  1885     3470        0    69632      497             0 sendmail
Oct  3 10:29:34 mandelbrot kernel: [   1892]    51  1892     3326        1    69632      391             0 sendmail
Oct  3 10:29:34 mandelbrot kernel: [   1918]     0  1918    59631        0    86016      336             0 cupsd
Oct  3 10:29:34 mandelbrot kernel: [   1919]     0  1919    11714        0   131072     6450             0 systemd-logind
Oct  3 10:29:34 mandelbrot kernel: [   1920]     0  1920   213044        0   258048    12811             0 automount
Oct  3 10:29:34 mandelbrot kernel: [   1924]     0  1924      910        0    53248       47             0 atd
Oct  3 10:29:34 mandelbrot kernel: [   1925]     0  1925    53864        0    57344       33             0 agetty
Oct  3 10:29:34 mandelbrot kernel: [   1939]     0  1939     5015        3    77824      430             0 systemd
Oct  3 10:29:34 mandelbrot kernel: [   1948]     0  1948     6131       19    81920      828             0 (sd-pam)
Oct  3 10:29:34 mandelbrot kernel: [   1954]     0  1954    54299        0    57344      214             0 crond
Oct  3 10:29:34 mandelbrot kernel: [   1974]     0  1974    16706        0   172032    12610             0 sshd
Oct  3 10:29:34 mandelbrot kernel: [   2012]     0  2012 27268901 16121383 175370240  5744455             0 (systemd)
Oct  3 10:29:34 mandelbrot kernel: [   2070]  6105  2070    16706       19   172032    12592             0 sshd
Oct  3 10:29:34 mandelbrot kernel: [   2077]  6105  2077    62591        1   122880     6649             0 bash
Oct  3 10:29:34 mandelbrot kernel: [   2125]  6105  2125    69007        0   176128    12519             0 su
Oct  3 10:29:34 mandelbrot kernel: [   2134]     0  2134    54447      102    65536      318             0 bash
Oct  3 10:29:34 mandelbrot kernel: [   2320]     0  2320    57650      213    90112        0             0 systemctl
Oct  3 10:29:34 mandelbrot kernel: [   2321]     0  2321    56913     1876    77824        0             0 x2golistsession
Oct  3 10:29:34 mandelbrot kernel: [   2322]     0  2322     1838       32    53248        0             0 systemd-tty-ask
Oct  3 10:29:34 mandelbrot kernel: [   2323]     0  2323    42262      490    94208      312             0 systemd
Oct  3 10:29:34 mandelbrot kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0-1,global_oom,task_memcg=/init.scope,task=(systemd),pid=2012,uid=0
Oct  3 10:29:34 mandelbrot kernel: Out of memory: Killed process 2012 ((systemd)) total-vm:109075604kB, anon-rss:64485532kB, file-rss:0kB, shmem-rss:0kB
Oct  3 10:29:35 mandelbrot kdumpctl[2323]: No kdump initial ramdisk found.
Oct  3 10:29:35 mandelbrot kdumpctl[2323]: Rebuilding /boot/initramfs-5.2.17-200.fc30.x86_64kdump.img
Oct  3 10:29:37 mandelbrot kernel: oom_reaper: reaped process 2012 ((systemd)), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB


I'm trying to set up kdump to perhaps provide more debug but I keep getting this error:
kdumpctl[15501]: Cannot load /boot/vmlinuz-5.2.17-200.fc30.x86_64
kdumpctl[15501]: kexec: failed to load kdump kernel

Comment 1 RobbieTheK 2019-10-09 16:41:11 UTC
I'm changing this to Cockpit, which I've uninstalled. I had disabled it, i.e., systemctl disable cockpit, but it appears to restart itself. This emotectl process must have a memory leak. Let me know what else I can provide.

Oct  9 10:37:10 ourserver kernel: (emotectl) invoked oom-killer: gfp_mask=0x100cca(GFP_HIGHUSER_MOVABLE), order=0, oom_score_adj=0
Oct  9 10:37:11 ourserver kernel: CPU: 2 PID: 27603 Comm: (emotectl) Not tainted 5.2.17-200.fc30.x86_64 #1
Oct  9 10:37:11 ourserver kernel: Hardware name: Dell Inc. PowerEdge R740/00WGD1, BIOS 2.3.10 08/15/2019
Oct  9 10:37:11 ourserver kernel: Call Trace:
Oct  9 10:37:11 ourserver kernel: dump_stack+0x5c/0x80
Oct  9 10:37:11 ourserver kernel: dump_header+0x51/0x2ff
Oct  9 10:37:11 ourserver kernel: ? sched_clock+0x5/0x10
Oct  9 10:37:11 ourserver kernel: ? oom_unkillable_task+0x9a/0xd0
Oct  9 10:37:11 ourserver kernel: oom_kill_process.cold+0xb/0x10
Oct  9 10:37:11 ourserver kernel: out_of_memory+0x1d4/0x480
Oct  9 10:37:11 ourserver kernel: __alloc_pages_slowpath+0xcc4/0xde0
Oct  9 10:37:11 ourserver kernel: __alloc_pages_nodemask+0x2ee/0x340
Oct  9 10:37:11 ourserver kernel: pagecache_get_page+0xb4/0x370
Oct  9 10:37:11 ourserver kernel: filemap_fault+0x683/0x9f0
Oct  9 10:37:11 ourserver kernel: ? __mod_lruvec_state+0x3f/0xe0
Oct  9 10:37:11 ourserver kernel: ? page_add_file_rmap+0xf4/0x230
Oct  9 10:37:11 ourserver kernel: ? alloc_set_pte+0x498/0x6b0
Oct  9 10:37:11 ourserver kernel: ? file_fdatawait_range+0x20/0x20
Oct  9 10:37:11 ourserver kernel: ? _cond_resched+0x15/0x30
Oct  9 10:37:11 ourserver kernel: __xfs_filemap_fault+0x6d/0x200 [xfs]
Oct  9 10:37:11 ourserver kernel: __do_fault+0x36/0x180
Oct  9 10:37:11 ourserver kernel: __handle_mm_fault+0x1091/0x1ac0
Oct  9 10:37:11 ourserver kernel: handle_mm_fault+0xc4/0x1e0
Oct  9 10:37:11 ourserver kernel: do_user_addr_fault+0x1f6/0x450
Oct  9 10:37:11 ourserver kernel: do_page_fault+0x33/0x120
Oct  9 10:37:11 ourserver kernel: ? page_fault+0x8/0x30
Oct  9 10:37:11 ourserver kernel: page_fault+0x1e/0x30
Oct  9 10:37:11 ourserver kernel: RIP: 0033:0x7f995c93ad50
Oct  9 10:37:11 ourserver kernel: Code: Bad RIP value.
Oct  9 10:37:11 ourserver kernel: RSP: 002b:00007ffc33ddc5f8 EFLAGS: 00010202
Oct  9 10:37:11 ourserver kernel: RAX: 0000000000000000 RBX: 00007f88076fd040 RCX: 000000003ffffff8
Oct  9 10:37:11 ourserver kernel: RDX: 00007f7f50000010 RSI: 0000000000000000 RDI: 00007f88076fd010
Oct  9 10:37:11 ourserver kernel: RBP: 000055d4b1f9e8c0 R08: 00007f7f50000010 R09: 0000000000000000
Oct  9 10:37:11 ourserver kernel: R10: 0000000000000022 R11: 0000000000000246 R12: 000055d4b1ef2860
Oct  9 10:37:11 ourserver kernel: R13: 0000000000000004 R14: 00007ffc33ddc6b0 R15: 00007f8b4ffffe90
Oct  9 10:37:11 ourserver kernel: Mem-Info:
Oct  9 10:37:11 ourserver kernel: active_anon:15200764 inactive_anon:895460 isolated_anon:0#012 active_file:0 inactive_file:20 isolated_file:0#012 unevictable:0 dirty:0 writeback:57 unstable:0#012 slab_reclaimable:23356 slab_unreclaimable:50962#012 mapped:0 shmem:2 pagetables:44484 bounce:0#012 free:56603 free_pcp:0 free_cma:0
Oct  9 10:37:11 ourserver kernel: Node 0 active_anon:30068504kB inactive_anon:1771616kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
Oct  9 10:37:11 ourserver kernel: Node 1 active_anon:30734552kB inactive_anon:1810224kB active_file:92kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:228kB shmem:8kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
Oct  9 10:37:11 ourserver kernel: Node 0 DMA free:15880kB min:20kB low:32kB high:44kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15980kB managed:15896kB mlocked:0kB kernel_stack:0kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  9 10:37:11 ourserver kernel: lowmem_reserve[]: 0 1396 31576 31576 31576
Oct  9 10:37:11 ourserver kernel: Node 0 DMA32 free:122688kB min:1968kB low:3396kB high:4824kB active_anon:1322012kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:1521664kB managed:1456124kB mlocked:0kB kernel_stack:16kB pagetables:2584kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  9 10:37:11 ourserver kernel: lowmem_reserve[]: 0 0 30180 30180 30180
Oct  9 10:37:11 ourserver kernel: Node 0 Normal free:42556kB min:42596kB low:73500kB high:104404kB active_anon:28746492kB inactive_anon:1771616kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:31457280kB managed:30912644kB mlocked:0kB kernel_stack:3096kB pagetables:86508kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  9 10:37:11 ourserver kernel: lowmem_reserve[]: 0 0 0 0 0
Oct  9 10:37:11 ourserver kernel: Node 1 Normal free:45288kB min:45520kB low:78548kB high:111576kB active_anon:30734552kB inactive_anon:1810224kB active_file:0kB inactive_file:520kB unevictable:0kB writepending:228kB present:33554432kB managed:33028052kB mlocked:0kB kernel_stack:2648kB pagetables:88844kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
Oct  9 10:37:11 ourserver kernel: lowmem_reserve[]: 0 0 0 0 0
Oct  9 10:37:11 ourserver kernel: Node 0 DMA: 0*4kB 1*8kB (U) 0*16kB 0*32kB 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) = 15880kB
Oct  9 10:37:11 ourserver kernel: Node 0 DMA32: 102*4kB (UE) 87*8kB (UE) 79*16kB (UME) 46*32kB (UME) 37*64kB (UME) 36*128kB (UE) 31*256kB (UE) 27*512kB (UME) 22*1024kB (UME) 7*2048kB (UME) 13*4096kB (UE) = 122688kB
Oct  9 10:37:11 ourserver kernel: Node 0 Normal: 198*4kB (UME) 70*8kB (UME) 32*16kB (UME) 81*32kB (UME) 77*64kB (UME) 55*128kB (UME) 17*256kB (UM) 0*512kB 20*1024kB (M) 1*2048kB (M) 0*4096kB = 43304kB
Oct  9 10:37:11 ourserver kernel: Node 1 Normal: 229*4kB (UME) 212*8kB (UME) 56*16kB (UME) 38*32kB (UME) 143*64kB (UME) 75*128kB (UME) 1*256kB (E) 0*512kB 18*1024kB (M) 2*2048kB (M) 0*4096kB = 46260kB
Oct  9 10:37:11 ourserver kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct  9 10:37:11 ourserver kernel: Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  9 10:37:11 ourserver kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
Oct  9 10:37:11 ourserver kernel: Node 1 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Oct  9 10:37:11 ourserver kernel: 4872 total pagecache pages
Oct  9 10:37:11 ourserver kernel: 4792 pages in swap cache
Oct  9 10:37:11 ourserver kernel: Swap cache stats: add 4553406660, delete 4552431434, find 4529567/9839697
Oct  9 10:37:11 ourserver kernel: Free swap  = 0kB
Oct  9 10:37:11 ourserver kernel: Total swap = 23441404kB
Oct  9 10:37:11 ourserver kernel: 16637339 pages RAM
Oct  9 10:37:11 ourserver kernel: 0 pages HighMem/MovableOnly
Oct  9 10:37:11 ourserver kernel: 284160 pages reserved
Oct  9 10:37:11 ourserver kernel: 0 pages cma reserved
Oct  9 10:37:11 ourserver kernel: 0 pages hwpoisoned
Oct  9 10:37:11 ourserver kernel: Tasks state (memory values in pages):
Oct  9 10:37:11 ourserver kernel: [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
Oct  9 10:37:11 ourserver kernel: [    852]     0   852    29694       16   200704      425             0 systemd-journal
Oct  9 10:37:11 ourserver kernel: [    884]     0   884     7957        0    81920      645         -1000 systemd-udevd
Oct  9 10:37:11 ourserver kernel: [   1280]    32  1280     3078        0    61440      585             0 rpcbind
Oct  9 10:37:11 ourserver kernel: [   1289]     0  1289     1560        0    49152      131             0 rpc.idmapd
Oct  9 10:37:11 ourserver kernel: [   1294]     0  1294    19842       33    53248       53             0 irqbalance
Oct  9 10:37:11 ourserver kernel: [   1296]     0  1296    78035        0   106496      264             0 rngd
Oct  9 10:37:11 ourserver kernel: [   1297]     0  1297    78329        0   106496      428             0 ModemManager
Oct  9 10:37:11 ourserver kernel: [   1299]    70  1299     2989        2    57344      208             0 avahi-daemon
Oct  9 10:37:11 ourserver kernel: [   1302]     0  1302      624        0    45056       30             0 mcelog
Oct  9 10:37:11 ourserver kernel: [   1306]     0  1306     2613        0    57344      328             0 smartd
Oct  9 10:37:11 ourserver kernel: [   1308]     0  1308   134606        0   147456      772             0 NetworkManager
Oct  9 10:37:11 ourserver kernel: [   1309]     0  1309   186043        0  1073152      450             0 rsyslogd
Oct  9 10:37:11 ourserver kernel: [   1312]   994  1312    21125        0    65536      228             0 chronyd
Oct  9 10:37:11 ourserver kernel: [   1314]     0  1314   114629        0   122880      486             0 abrtd
Oct  9 10:37:11 ourserver kernel: [   1321]    70  1321     2956        0    53248      172             0 avahi-daemon
Oct  9 10:37:11 ourserver kernel: [   1322]    81  1322    67186        0    61440      201          -900 dbus-broker-lau
Oct  9 10:37:11 ourserver kernel: [   1323]     0  1323    65916        0    90112      181             0 gssproxy
Oct  9 10:37:11 ourserver kernel: [   1330]  1001  1330     1440       10    53248      147             0 lmgrd
Oct  9 10:37:11 ourserver kernel: [   1331]  1001  1331    39782       22    69632      258             0 MLM
Oct  9 10:37:11 ourserver kernel: [   1392]    81  1392     1495        8    49152      285          -900 dbus-broker
Oct  9 10:37:11 ourserver kernel: [   1397]     0  1397   204763       11   774144      469             0 abrt-dump-journ
Oct  9 10:37:11 ourserver kernel: [   1398]     0  1398   202700        3   684032      452             0 abrt-dump-journ
Oct  9 10:37:11 ourserver kernel: [   1399]     0  1399   206779       15   733184      450             0 abrt-dump-journ
Oct  9 10:37:11 ourserver kernel: [   1410]     0  1410    58909      658    90112     3155             0 x2gocleansessio
Oct  9 10:37:11 ourserver kernel: [   1432]   997  1432   482919        0   225280     1302             0 polkitd
Oct  9 10:37:11 ourserver kernel: [   1724]     0  1724     9850        1   118784     6346             0 rpc.ypxfrd
Oct  9 10:37:11 ourserver kernel: [   1726]     0  1726    10851        0   126976     6809             0 ypserv
Oct  9 10:37:11 ourserver kernel: [   1728]     0  1728     2993        0    61440      238         -1000 sshd
Oct  9 10:37:11 ourserver kernel: [   1729]    29  1729    15483        0   159744    12482             0 rpc.statd
Oct  9 10:37:11 ourserver kernel: [   1731]     0  1731     9896        1   118784     6348             0 rpc.yppasswdd
Oct  9 10:37:11 ourserver kernel: [   1742]     0  1742    16616        0   167936    12785             0 rpc.mountd
Oct  9 10:37:11 ourserver kernel: [   1822]     0  1822    89938        0   192512    12732             0 ypbind
Oct  9 10:37:11 ourserver kernel: [   1885]     0  1885     3471       15    65536      513             0 sendmail
Oct  9 10:37:11 ourserver kernel: [   1892]    51  1892     3327        0    69632      407             0 sendmail
Oct  9 10:37:11 ourserver kernel: [   2121]     0  2121    11714        1   135168     6451             0 systemd-logind
Oct  9 10:37:11 ourserver kernel: [   2123]     0  2123   297864        0   303104    13000             0 automount
Oct  9 10:37:11 ourserver kernel: [   2124]     0  2124    59631        0    98304      340             0 cupsd
Oct  9 10:37:11 ourserver kernel: [   2127]     0  2127      910        0    49152       54             0 atd
Oct  9 10:37:11 ourserver kernel: [   2151]     0  2151    54298        8    61440      205             0 crond
Oct  9 10:37:11 ourserver kernel: [   2152]     0  2152     5014        0    81920      444             0 systemd
Oct  9 10:37:11 ourserver kernel: [   2155]     0  2155     6141        0    81920      842             0 (sd-pam)
Oct  9 10:37:11 ourserver kernel: [   2423]     0  2423    53864        0    45056       34             0 agetty
Oct  9 10:37:11 ourserver kernel: [  21827]     0 21827    68897        0   167936    12630             0 crond
Oct  9 10:37:11 ourserver kernel: [  21831]     0 21831    54095        1    57344       59             0 freshclam-sleep
Oct  9 10:37:11 ourserver kernel: [  21833]     0 21833    53742        0    57344       22             0 sleep
Oct  9 10:37:11 ourserver kernel: [  25206]     0 25206    69185        0   172032    12685             0 crond
Oct  9 10:37:11 ourserver kernel: [  25229]     0 25229    54095        0    57344       56             0 sh
Oct  9 10:37:11 ourserver kernel: [  25230]     0 25230    54095        1    61440       58             0 cronic
Oct  9 10:37:11 ourserver kernel: [  25232]     0 25232    56910        0    73728     1976             0 rsnapshot
Oct  9 10:37:11 ourserver kernel: [  25413]     0 25413    54095        1    61440       57             0 rsync-no-vanish
Oct  9 10:37:11 ourserver kernel: [  25414]     0 25414   108933      146   495616    40680             0 rsync
Oct  9 10:37:11 ourserver kernel: [  25415]     0 25415    54095        0    57344       59             0 rsync-no-vanish
Oct  9 10:37:11 ourserver kernel: [  25416]     0 25416    53935        0    53248       41             0 grep
Oct  9 10:37:11 ourserver kernel: [  25417]     0 25417    56089       31    73728      812             0 ssh
Oct  9 10:37:11 ourserver kernel: [  25418]     0 25418   108725      824   491520    32552             0 rsync
Oct  9 10:37:11 ourserver kernel: [  26670]     0 26670    16711        0   172032    12615             0 sshd
Oct  9 10:37:11 ourserver kernel: [  26724]  6105 26724    16711       21   172032    12590             0 sshd
Oct  9 10:37:11 ourserver kernel: [  26731]  6105 26731    62595        0   118784     6649             0 bash
Oct  9 10:37:11 ourserver kernel: [  26842]  6105 26842    69011        0   172032    12524             0 su
Oct  9 10:37:11 ourserver kernel: [  26862]     0 26862    54439       13    65536      410             0 bash
Oct  9 10:37:11 ourserver kernel: [  27603]     0 27603 27306114 16088978 174247936  5635073             0 (emotectl)
Oct  9 10:37:11 ourserver kernel: [  27754]     0 27754    53831        9    53248       59             0 less
Oct  9 10:37:11 ourserver kernel: [  27774]     0 27774      110        1    32768        0             0 x2golistsession
Oct  9 10:37:11 ourserver kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0-1,global_oom,task_memcg=/init.scope,task=(emotectl),pid=27603,uid=0
Oct  9 10:37:11 ourserver kernel: Out of memory: Killed process 27603 ((emotectl)) total-vm:109224456kB, anon-rss:64355912kB, file-rss:0kB, shmem-rss:0kB
Oct  9 10:37:13 ourserver kernel: oom_reaper: reaped process 27603 ((emotectl)), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Oct  9 10:37:14 ourserver systemd[1]: cockpit.service: Control process exited, code=killed, status=9/KILL
Oct  9 10:37:14 ourserver systemd[1]: cockpit.service: Failed with result 'timeout'.
Oct  9 10:37:14 ourserver systemd[1]: Failed to start Cockpit Web Service.
Oct  9 10:37:14 ourserver systemd[1]: Starting Cockpit Web Service..

Comment 2 Ben Cotton 2020-04-30 20:58:57 UTC
This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 3 Ben Cotton 2020-11-03 17:03:57 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 4 Martin Pitt 2021-04-22 06:59:29 UTC
There is a really good chance that release 242 fixed that with dropping remotectl from the unit (https://github.com/cockpit-project/cockpit/pull/15608). Can you still reproduce this with the current Fedora 33/34 version?

Comment 5 Fedora Update System 2021-04-22 18:43:18 UTC
FEDORA-2021-bcf96c4434 has been submitted as an update to Fedora 34. https://bodhi.fedoraproject.org/updates/FEDORA-2021-bcf96c4434

Comment 6 Martin Pitt 2021-04-22 18:44:23 UTC
Verified in upstream issue https://github.com/cockpit-project/cockpit/issues/12949 , so this is fixed in Cockpit 242.

Comment 7 Fedora Update System 2021-04-24 20:11:52 UTC
FEDORA-2021-bcf96c4434 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 8 RobbieTheK 2021-05-06 02:27:59 UTC
(In reply to Martin Pitt from comment #4)
> There is a really good chance that release 242 fixed that with dropping
> remotectl from the unit
> (https://github.com/cockpit-project/cockpit/pull/15608). Can you still
> reproduce this with the current Fedora 33/34 version?

No more OOMs.


Note You need to log in before you can comment on or make changes to this bug.