Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 1484372 Details for
Bug 1630370
unpredictable out of memory killing multiple processes
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
dmesg output after cascade oom killer
why-2018-09-18-08-17-01 (text/plain), 635.35 KB, created by
R Bruce Hoffman
on 2018-09-18 13:20:53 UTC
(
hide
)
Description:
dmesg output after cascade oom killer
Filename:
MIME Type:
Creator:
R Bruce Hoffman
Created:
2018-09-18 13:20:53 UTC
Size:
635.35 KB
patch
obsolete
>[ 0.000000] hash-mmu: Page sizes from device-tree: >[ 0.000000] hash-mmu: base_shift=12: shift=12, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=0 >[ 0.000000] hash-mmu: base_shift=12: shift=16, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=7 >[ 0.000000] hash-mmu: base_shift=12: shift=24, sllp=0x0000, avpnm=0x00000000, tlbiel=1, penc=56 >[ 0.000000] hash-mmu: base_shift=16: shift=16, sllp=0x0110, avpnm=0x00000000, tlbiel=1, penc=1 >[ 0.000000] hash-mmu: base_shift=16: shift=24, sllp=0x0110, avpnm=0x00000000, tlbiel=1, penc=8 >[ 0.000000] hash-mmu: base_shift=24: shift=24, sllp=0x0100, avpnm=0x00000001, tlbiel=0, penc=0 >[ 0.000000] hash-mmu: base_shift=34: shift=34, sllp=0x0120, avpnm=0x000007ff, tlbiel=0, penc=3 >[ 0.000000] Page orders: linear mapping = 24, virtual = 16, io = 16, vmemmap = 24 >[ 0.000000] Using 1TB segments >[ 0.000000] hash-mmu: Initializing hash mmu with SLB >[ 0.000000] Linux version 4.18.7-200.fc28.ppc64 (mockbuild@buildvm-ppc64-03.ppc.fedoraproject.org) (gcc version 8.1.1 20180712 (Red Hat 8.1.1-5) (GCC)) #1 SMP Mon Sep 10 15:21:31 UTC 2018 >[ 0.000000] Found initrd at 0xc00000000c400000:0xc00000000dd796bb >[ 0.000000] Using pSeries machine description >[ 0.000000] bootconsole [udbg0] enabled >[ 0.000000] Partition configured for 320 cpus. >[ 0.000000] CPU maps initialized for 8 threads per core >[ 0.000000] (thread shift is 3) >[ 0.000000] Allocated 7168 bytes for 320 pacas >[ 0.000000] ----------------------------------------------------- >[ 0.000000] ppc64_pft_size = 0x1d >[ 0.000000] phys_mem_size = 0x80000000 >[ 0.000000] dcache_bsize = 0x80 >[ 0.000000] icache_bsize = 0x80 >[ 0.000000] cpu_features = 0x000000ff8f5d91a7 >[ 0.000000] possible = 0x0000ffffffffb1af >[ 0.000000] always = 0x00000000000001a0 >[ 0.000000] cpu_user_features = 0xdc0065c2 0xef000000 >[ 0.000000] mmu_features = 0x7c006001 >[ 0.000000] firmware_features = 0x00000007c45bfc57 >[ 0.000000] htab_hash_mask = 0x3fffff >[ 0.000000] ----------------------------------------------------- >[ 0.000000] numa: NODE_DATA [mem 0x7f5d2300-0x7f5dbfff] >[ 0.000000] rfi-flush: fallback displacement flush available >[ 0.000000] rfi-flush: ori type flush available >[ 0.000000] rfi-flush: patched 10 locations (ori type flush) >[ 0.000000] barrier-nospec: using ORI speculation barrier >[ 0.000000] barrier-nospec: patched 545 locations >[ 0.000000] stf-barrier: hwsync barrier available >[ 0.000000] stf-barrier: patched 61 entry locations (hwsync barrier) >[ 0.000000] stf-barrier: patched 10 exit locations (hwsync barrier) >[ 0.000000] PPC64 nvram contains 15360 bytes >[ 0.000000] Top of RAM: 0x80000000, Total RAM: 0x80000000 >[ 0.000000] Memory hole size: 0MB >[ 0.000000] Zone ranges: >[ 0.000000] DMA [mem 0x0000000000000000-0x000000007fffffff] >[ 0.000000] DMA32 empty >[ 0.000000] Normal empty >[ 0.000000] Movable zone start for each node >[ 0.000000] Early memory node ranges >[ 0.000000] node 0: [mem 0x0000000000000000-0x000000007fffffff] >[ 0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000007fffffff] >[ 0.000000] On node 0 totalpages: 32768 >[ 0.000000] DMA zone: 32 pages used for memmap >[ 0.000000] DMA zone: 0 pages reserved >[ 0.000000] DMA zone: 32768 pages, LIFO batch:1 >[ 0.000000] percpu: Embedded 4 pages/cpu @(____ptrval____) s160792 r0 d101352 u262144 >[ 0.000000] pcpu-alloc: s160792 r0 d101352 u262144 alloc=1*1048576 >[ 0.000000] pcpu-alloc: [0] 000 001 002 003 [0] 004 005 006 007 >[ 0.000000] pcpu-alloc: [0] 008 009 010 011 [0] 012 013 014 015 >[ 0.000000] pcpu-alloc: [0] 016 017 018 019 [0] 020 021 022 023 >[ 0.000000] pcpu-alloc: [0] 024 025 026 027 [0] 028 029 030 031 >[ 0.000000] pcpu-alloc: [0] 032 033 034 035 [0] 036 037 038 039 >[ 0.000000] pcpu-alloc: [0] 040 041 042 043 [0] 044 045 046 047 >[ 0.000000] pcpu-alloc: [0] 048 049 050 051 [0] 052 053 054 055 >[ 0.000000] pcpu-alloc: [0] 056 057 058 059 [0] 060 061 062 063 >[ 0.000000] pcpu-alloc: [0] 064 065 066 067 [0] 068 069 070 071 >[ 0.000000] pcpu-alloc: [0] 072 073 074 075 [0] 076 077 078 079 >[ 0.000000] pcpu-alloc: [0] 080 081 082 083 [0] 084 085 086 087 >[ 0.000000] pcpu-alloc: [0] 088 089 090 091 [0] 092 093 094 095 >[ 0.000000] pcpu-alloc: [0] 096 097 098 099 [0] 100 101 102 103 >[ 0.000000] pcpu-alloc: [0] 104 105 106 107 [0] 108 109 110 111 >[ 0.000000] pcpu-alloc: [0] 112 113 114 115 [0] 116 117 118 119 >[ 0.000000] pcpu-alloc: [0] 120 121 122 123 [0] 124 125 126 127 >[ 0.000000] pcpu-alloc: [0] 128 129 130 131 [0] 132 133 134 135 >[ 0.000000] pcpu-alloc: [0] 136 137 138 139 [0] 140 141 142 143 >[ 0.000000] pcpu-alloc: [0] 144 145 146 147 [0] 148 149 150 151 >[ 0.000000] pcpu-alloc: [0] 152 153 154 155 [0] 156 157 158 159 >[ 0.000000] pcpu-alloc: [0] 160 161 162 163 [0] 164 165 166 167 >[ 0.000000] pcpu-alloc: [0] 168 169 170 171 [0] 172 173 174 175 >[ 0.000000] pcpu-alloc: [0] 176 177 178 179 [0] 180 181 182 183 >[ 0.000000] pcpu-alloc: [0] 184 185 186 187 [0] 188 189 190 191 >[ 0.000000] pcpu-alloc: [0] 192 193 194 195 [0] 196 197 198 199 >[ 0.000000] pcpu-alloc: [0] 200 201 202 203 [0] 204 205 206 207 >[ 0.000000] pcpu-alloc: [0] 208 209 210 211 [0] 212 213 214 215 >[ 0.000000] pcpu-alloc: [0] 216 217 218 219 [0] 220 221 222 223 >[ 0.000000] pcpu-alloc: [0] 224 225 226 227 [0] 228 229 230 231 >[ 0.000000] pcpu-alloc: [0] 232 233 234 235 [0] 236 237 238 239 >[ 0.000000] pcpu-alloc: [0] 240 241 242 243 [0] 244 245 246 247 >[ 0.000000] pcpu-alloc: [0] 248 249 250 251 [0] 252 253 254 255 >[ 0.000000] pcpu-alloc: [0] 256 257 258 259 [0] 260 261 262 263 >[ 0.000000] pcpu-alloc: [0] 264 265 266 267 [0] 268 269 270 271 >[ 0.000000] pcpu-alloc: [0] 272 273 274 275 [0] 276 277 278 279 >[ 0.000000] pcpu-alloc: [0] 280 281 282 283 [0] 284 285 286 287 >[ 0.000000] pcpu-alloc: [0] 288 289 290 291 [0] 292 293 294 295 >[ 0.000000] pcpu-alloc: [0] 296 297 298 299 [0] 300 301 302 303 >[ 0.000000] pcpu-alloc: [0] 304 305 306 307 [0] 308 309 310 311 >[ 0.000000] pcpu-alloc: [0] 312 313 314 315 [0] 316 317 318 319 >[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 32736 >[ 0.000000] Policy zone: DMA >[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.18.7-200.fc28.ppc64 root=/dev/mapper/fedora_jkohan-root ro rd.lvm.lv=fedora_jkohan/root rd.lvm.lv=fedora_jkohan/swap LANG=en_US.UTF-8 >[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes >[ 0.000000] log_buf_len total cpu_extra contributions: 1306624 bytes >[ 0.000000] log_buf_len min size: 262144 bytes >[ 0.000000] log_buf_len: 2097152 bytes >[ 0.000000] early log buf free: 255264(97%) >[ 0.000000] Memory: 1891968K/2097152K available (12544K kernel code, 2496K rwdata, 4032K rodata, 6400K init, 2437K bss, 205184K reserved, 0K cma-reserved) >[ 0.000000] random: get_random_u64 called from .__kmem_cache_create+0x54/0x7e0 with crng_init=0 >[ 0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=320, Nodes=32 >[ 0.000000] ftrace: allocating 32094 entries in 12 pages >[ 0.000000] Hierarchical RCU implementation. >[ 0.000000] RCU restricting CPUs from NR_CPUS=1024 to nr_cpu_ids=320. >[ 0.000000] Tasks RCU enabled. >[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=320 >[ 0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16 >[ 0.000000] pic: no ISA interrupt controller >[ 0.000000] time_init: decrementer frequency = 512.000000 MHz >[ 0.000000] time_init: processor frequency = 3026.000000 MHz >[ 0.000002] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x761537d007, max_idle_ns: 440795202126 ns >[ 0.000048] clocksource: timebase mult[1f40000] shift[24] registered >[ 0.000071] clockevent: decrementer mult[83126e98] shift[32] cpu[0] >[ 0.000514] Console: colour dummy device 80x25 >[ 0.000533] console [hvc0] enabled >[ 0.000548] bootconsole [udbg0] disabled >[ 0.000665] pid_max: default: 327680 minimum: 2560 >[ 0.001024] Security Framework initialized >[ 0.001027] Yama: becoming mindful. >[ 0.001034] SELinux: Initializing. >[ 0.001315] SELinux: Starting in permissive mode >[ 0.001536] Dentry cache hash table entries: 262144 (order: 5, 2097152 bytes) >[ 0.001607] Inode-cache hash table entries: 131072 (order: 4, 1048576 bytes) >[ 0.001674] Mount-cache hash table entries: 8192 (order: 0, 65536 bytes) >[ 0.001682] Mountpoint-cache hash table entries: 8192 (order: 0, 65536 bytes) >[ 0.011707] EEH: pSeries platform initialized >[ 0.011715] POWER8 performance monitor hardware support registered >[ 0.011794] Hierarchical SRCU implementation. >[ 0.021406] smp: Bringing up secondary CPUs ... >[ 0.051164] smp: Brought up 1 node, 16 CPUs >[ 0.051176] numa: Node 0 CPUs: 0-15 >[ 0.051182] Using standard scheduler topology >[ 0.061283] devtmpfs: initialized >[ 0.070241] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns >[ 0.070300] futex hash table entries: 131072 (order: 8, 16777216 bytes) >[ 0.080720] NET: Registered protocol family 16 >[ 0.080897] EEH: No capable adapters found >[ 0.080969] IBM eBus Device Driver >[ 0.081062] audit: initializing netlink subsys (disabled) >[ 0.081339] audit: type=2000 audit(1537243673.080:1): state=initialized audit_enabled=0 res=1 >[ 0.081482] cpuidle: using governor menu >[ 0.081533] RTAS daemon started >[ 0.081635] pstore: Registered nvram as persistent store backend >[ 0.131237] PCI: Probing PCI hardware >[ 0.131249] PCI: Probing PCI hardware done >[ 0.131413] pseries-rng: Registering arch random hook. >[ 0.141002] HugeTLB registered 16.0 MiB page size, pre-allocated 0 pages >[ 0.141007] HugeTLB registered 16.0 GiB page size, pre-allocated 0 pages >[ 0.150406] vgaarb: loaded >[ 0.150561] SCSI subsystem initialized >[ 0.150734] libata version 3.00 loaded. >[ 0.150761] usbcore: registered new interface driver usbfs >[ 0.150772] usbcore: registered new interface driver hub >[ 0.150933] usbcore: registered new device driver usb >[ 0.150951] pps_core: LinuxPPS API ver. 1 registered >[ 0.150954] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> >[ 0.150960] PTP clock support registered >[ 0.151238] EDAC MC: Ver: 3.0.0 >[ 0.151647] NetLabel: Initializing >[ 0.151651] NetLabel: domain hash size = 128 >[ 0.151653] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO >[ 0.151671] NetLabel: unlabeled traffic allowed by default >[ 0.160031] clocksource: Switched to clocksource timebase >[ 0.240415] VFS: Disk quotas dquot_6.6.0 >[ 0.240542] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes) >[ 0.261164] NET: Registered protocol family 2 >[ 0.261700] tcp_listen_portaddr_hash hash table entries: 4096 (order: 0, 65536 bytes) >[ 0.261727] TCP established hash table entries: 16384 (order: 1, 131072 bytes) >[ 0.261782] TCP bind hash table entries: 16384 (order: 2, 262144 bytes) >[ 0.261821] TCP: Hash tables configured (established 16384 bind 16384) >[ 0.269954] UDP hash table entries: 2048 (order: 0, 65536 bytes) >[ 0.269986] UDP-Lite hash table entries: 2048 (order: 0, 65536 bytes) >[ 0.270567] NET: Registered protocol family 1 >[ 0.270577] PCI: CLS 0 bytes, default 128 >[ 0.270616] Unpacking initramfs... >[ 2.022046] Freeing initrd memory: 26048K >[ 2.031692] IOMMU table initialized, virtual merging enabled >[ 2.060800] hv-24x7: read 1330 catalog entries, created 337 event attrs (0 failures), 169 descs >[ 2.081995] Initialise system trusted keyrings >[ 2.082011] Key type blacklist registered >[ 2.082210] workingset: timestamp_bits=38 max_order=15 bucket_order=0 >[ 2.091441] zbud: loaded >[ 2.100225] pstore: using deflate compression >[ 2.100332] SELinux: Registering netfilter hooks >[ 2.880123] alg: No test for 842 (842-generic) >[ 2.880272] alg: No test for 842 (842-scomp) >[ 2.911767] NET: Registered protocol family 38 >[ 2.911779] Key type asymmetric registered >[ 2.911783] Asymmetric key parser 'x509' registered >[ 2.911871] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248) >[ 2.912075] io scheduler noop registered >[ 2.912078] io scheduler deadline registered >[ 2.920003] io scheduler cfq registered (default) >[ 2.920011] io scheduler mq-deadline registered >[ 2.920455] atomic64_test: passed >[ 2.920506] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 >[ 2.921208] random: fast init done >[ 2.921293] Linux agpgart interface v0.103 >[ 2.921405] Uniform Multi-Platform E-IDE driver >[ 2.921426] ide-gd driver 1.18 >[ 2.921584] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver >[ 2.921595] ehci-pci: EHCI PCI platform driver >[ 2.921604] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver >[ 2.921614] ohci-pci: OHCI PCI platform driver >[ 2.921622] uhci_hcd: USB Universal Host Controller Interface driver >[ 2.921657] usbcore: registered new interface driver usbserial_generic >[ 2.921665] usbserial: USB Serial support registered for generic >[ 2.921778] mousedev: PS/2 mouse device common for all mice >[ 2.929924] rtc-generic rtc-generic: rtc core: registered rtc-generic as rtc0 >[ 2.930018] device-mapper: uevent: version 1.0.3 >[ 2.930135] device-mapper: ioctl: 4.39.0-ioctl (2018-04-03) initialised: dm-devel@redhat.com >[ 2.950006] pseries_idle_driver registered >[ 2.950020] hidraw: raw HID events driver (C) Jiri Kosina >[ 2.950060] usbcore: registered new interface driver usbhid >[ 2.950063] usbhid: USB HID core driver >[ 2.950260] drop_monitor: Initializing network drop monitor service >[ 2.950594] Initializing XFRM netlink socket >[ 2.950838] NET: Registered protocol family 10 >[ 2.971288] Segment Routing with IPv6 >[ 2.971317] mip6: Mobile IPv6 >[ 2.971321] NET: Registered protocol family 17 >[ 2.971487] Running MSI bitmap self-tests ... >[ 2.980004] registered taskstats version 1 >[ 2.980021] Loading compiled-in X.509 certificates >[ 3.171713] Loaded X.509 cert 'Fedora kernel signing key: 47e4bb39b9ea43d4086cb865df2eb6b4c7b819a5' >[ 3.171769] zswap: loaded using pool lzo/zbud >[ 3.182092] Key type big_key registered >[ 3.192141] Key type encrypted registered >[ 3.199965] rtc-generic rtc-generic: setting system clock to 2018-09-18 04:07:56 UTC (1537243676) >[ 3.201738] Freeing unused kernel memory: 6400K >[ 3.201742] This architecture does not have kernel memory protection. >[ 3.281455] systemd[1]: systemd 238 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid) >[ 3.281600] systemd[1]: Detected architecture ppc64. >[ 3.281606] systemd[1]: Running in initial RAM disk. >[ 3.320494] systemd[1]: Set hostname to <mailsvrc.ctcodeworks.com>. >[ 3.640353] random: systemd: uninitialized urandom read (16 bytes read) >[ 3.640421] systemd[1]: Listening on Journal Audit Socket. >[ 3.640667] random: systemd: uninitialized urandom read (16 bytes read) >[ 3.640751] systemd[1]: Listening on udev Control Socket. >[ 3.640906] random: systemd: uninitialized urandom read (16 bytes read) >[ 3.640918] systemd[1]: Reached target Slices. >[ 3.641067] systemd[1]: Reached target Timers. >[ 3.641237] systemd[1]: Reached target Swap. >[ 3.641449] systemd[1]: Listening on Journal Socket (/dev/log). >[ 3.930243] audit: type=1130 audit(1537243677.230:2): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 3.940687] audit: type=1130 audit(1537243677.240:3): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 3.960056] audit: type=1130 audit(1537243677.260:4): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-journald comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 3.980334] audit: type=1130 audit(1537243677.280:5): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup-dev comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 4.719961] audit: type=1130 audit(1537243678.010:6): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-vconsole-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 4.719994] audit: type=1131 audit(1537243678.010:7): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-vconsole-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 4.780265] audit: type=1130 audit(1537243678.080:8): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-cmdline comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 4.939960] audit: type=1130 audit(1537243678.230:9): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-udev comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 5.029961] audit: type=1130 audit(1537243678.320:10): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udevd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 5.790389] synth uevent: /devices/vio: failed to send uevent >[ 5.790394] vio vio: uevent: failed to send synthetic uevent >[ 5.790464] synth uevent: /devices/vio/4000: failed to send uevent >[ 5.790465] vio 4000: uevent: failed to send synthetic uevent >[ 5.790482] synth uevent: /devices/vio/4001: failed to send uevent >[ 5.790484] vio 4001: uevent: failed to send synthetic uevent >[ 5.790501] synth uevent: /devices/vio/4002: failed to send uevent >[ 5.790502] vio 4002: uevent: failed to send synthetic uevent >[ 5.790522] synth uevent: /devices/vio/4004: failed to send uevent >[ 5.790523] vio 4004: uevent: failed to send synthetic uevent >[ 5.820244] audit: type=1130 audit(1537243679.120:11): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-udev-trigger comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 5.991014] audit: type=1130 audit(1537243679.290:12): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=plymouth-start comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 6.510433] ibmveth: IBM Power Virtual Ethernet Driver 1.06 >[ 6.550908] ibmvscsi 30000007: SRP_VERSION: 16.a >[ 6.551118] ibmvscsi 30000007: Maximum ID: 64 Maximum LUN: 32 Maximum Channel: 3 >[ 6.551125] scsi host0: IBM POWER Virtual SCSI Adapter 1.5.9 >[ 6.560227] ibmvscsi 30000007: partner initialization complete >[ 6.560320] ibmvscsi 30000007: host srp version: 16.a, host partition ctcp8ai1 (1), OS 1, max io 14680064 >[ 6.560340] ibmvscsi 30000007: sent SRP login >[ 6.560372] ibmvscsi 30000007: SRP_LOGIN succeeded >[ 6.600292] scsi 0:0:0:0: Direct-Access IBM VDASD CTCP8AL401 0001 PQ: 0 ANSI: 4 >[ 6.610204] ibmveth 30000002 env2: renamed from eth0 >[ 6.610355] scsi 0:0:1:0: Direct-Access IBM VDASD CTCP8AL402 0001 PQ: 0 ANSI: 4 >[ 6.611363] scsi 0:0:2:0: Direct-Access IBM VDASD CTCP8AL403 0001 PQ: 0 ANSI: 4 >[ 6.620655] scsi 0:0:3:0: Direct-Access IBM VDASD CTCP8AL404 0001 PQ: 0 ANSI: 4 >[ 6.621600] scsi 0:0:4:0: Direct-Access IBM VDASD CTCP8AL405 0001 PQ: 0 ANSI: 4 >[ 6.630242] scsi host0: scsi scan: INQUIRY result too short (5), using 36 >[ 6.920075] scsi 0:1:0:0: CD-ROM IBM VOPT OPT01 0001 PQ: 0 ANSI: 4 >[ 6.920837] scsi 0:1:1:0: CD-ROM IBM VOPT OPTVRT01 0001 PQ: 0 ANSI: 4 >[ 7.200721] scsi 0:2:0:0: Sequential-Access IBM ULT3580-HH5 G9N1 PQ: 0 ANSI: 6 >[ 7.760569] sd 0:0:0:0: Attached scsi generic sg0 type 0 >[ 7.760888] sd 0:0:0:0: [sda] 8193150 512-byte logical blocks: (4.19 GB/3.91 GiB) >[ 7.760892] sd 0:0:1:0: Attached scsi generic sg1 type 0 >[ 7.760954] sd 0:0:0:0: [sda] Write Protect is off >[ 7.760959] sd 0:0:0:0: [sda] Mode Sense: 23 00 00 00 >[ 7.761059] sd 0:0:0:0: [sda] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA >[ 7.769959] sd 0:0:1:0: [sdb] 8193150 512-byte logical blocks: (4.19 GB/3.91 GiB) >[ 7.769964] sd 0:0:2:0: Attached scsi generic sg2 type 0 >[ 7.770082] sd 0:0:1:0: [sdb] Write Protect is off >[ 7.770090] sd 0:0:1:0: [sdb] Mode Sense: 23 00 00 00 >[ 7.770212] sd 0:0:1:0: [sdb] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA >[ 7.770257] sd 0:0:2:0: [sdc] 8193150 512-byte logical blocks: (4.19 GB/3.91 GiB) >[ 7.770405] sd 0:0:2:0: [sdc] Write Protect is off >[ 7.770415] sd 0:0:2:0: [sdc] Mode Sense: 23 00 00 00 >[ 7.770550] sd 0:0:2:0: [sdc] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA >[ 7.770551] sd 0:0:3:0: Attached scsi generic sg3 type 0 >[ 7.780154] sd 0:0:3:0: [sdd] 8193150 512-byte logical blocks: (4.19 GB/3.91 GiB) >[ 7.780306] sd 0:0:3:0: [sdd] Write Protect is off >[ 7.780321] sd 0:0:3:0: [sdd] Mode Sense: 23 00 00 00 >[ 7.780392] sda: sda1 sda2 sda3 >[ 7.780461] sd 0:0:3:0: [sdd] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA >[ 7.780647] sd 0:0:4:0: [sde] 8193150 512-byte logical blocks: (4.19 GB/3.91 GiB) >[ 7.780968] sd 0:0:4:0: [sde] Write Protect is off >[ 7.780974] sd 0:0:4:0: [sde] Mode Sense: 23 00 00 00 >[ 7.781112] sd 0:0:4:0: [sde] Write cache: disabled, read cache: disabled, doesn't support DPO or FUA >[ 7.789996] sd 0:0:4:0: Attached scsi generic sg4 type 0 >[ 7.790375] sr 0:1:0:0: [sr0] scsi3-mmc drive: 0x/0x dvd-ram cd/rw tray >[ 7.790381] cdrom: Uniform CD-ROM driver Revision: 3.20 >[ 7.790544] sdb: sdb1 >[ 7.790595] sr 0:1:0:0: Attached scsi CD-ROM sr0 >[ 7.790713] random: crng init done >[ 7.790719] random: 7 urandom warning(s) missed due to ratelimiting >[ 7.790773] sr 0:1:0:0: Attached scsi generic sg5 type 5 >[ 7.791157] sr 0:1:1:0: [sr1] scsi3-mmc drive: 0x/0x dvd-ram cd/rw tray >[ 7.800188] sr 0:1:1:0: Attached scsi CD-ROM sr1 >[ 7.800334] sr 0:1:1:0: Attached scsi generic sg6 type 5 >[ 7.800559] scsi 0:2:0:0: Attached scsi generic sg7 type 1 >[ 7.810221] sde: sde1 >[ 7.810471] sdd: sdd1 >[ 7.810682] sdc: sdc1 >[ 7.810909] sd 0:0:0:0: [sda] Attached SCSI disk >[ 7.810913] sd 0:0:1:0: [sdb] Attached SCSI disk >[ 7.819958] sd 0:0:4:0: [sde] Attached SCSI disk >[ 7.820224] sd 0:0:3:0: [sdd] Attached SCSI disk >[ 7.820322] sd 0:0:2:0: [sdc] Attached SCSI disk >[ 9.891066] audit: type=1130 audit(1537243683.190:13): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-initqueue comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 9.981152] audit: type=1130 audit(1537243683.280:14): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-fsck-root comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 10.569943] SGI XFS with ACLs, security attributes, scrub, no debug enabled >[ 10.601018] XFS (dm-0): Mounting V5 Filesystem >[ 10.685200] XFS (dm-0): Ending clean mount >[ 11.369964] audit: type=1130 audit(1537243684.660:15): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=initrd-parse-etc comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.370001] audit: type=1131 audit(1537243684.660:16): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=initrd-parse-etc comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.739849] audit: type=1130 audit(1537243685.030:17): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-pivot comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.829976] audit: type=1131 audit(1537243685.120:18): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-pre-pivot comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.839995] audit: type=1130 audit(1537243685.130:19): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-initqueue comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.840018] audit: type=1131 audit(1537243685.130:20): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=dracut-initqueue comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.840914] audit: type=1130 audit(1537243685.140:21): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.840930] audit: type=1131 audit(1537243685.140:22): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.859962] audit: type=1130 audit(1537243685.150:23): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 11.860008] audit: type=1131 audit(1537243685.160:24): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=kernel msg='unit=systemd-tmpfiles-setup comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 12.651367] systemd-journald[248]: Received SIGTERM from PID 1 (systemd). >[ 12.750179] systemd: 15 output lines suppressed due to ratelimiting >[ 13.033271] SELinux: Disabled at runtime. >[ 13.033316] SELinux: Unregistering netfilter hooks >[ 13.310146] systemd[1]: systemd 238 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid) >[ 13.310281] systemd[1]: Detected architecture ppc64. >[ 13.323251] systemd[1]: Set hostname to <mailsvrc.ctcodeworks.com>. >[ 14.761524] systemd[1]: Stopped Switch Root. >[ 14.761789] systemd[1]: systemd-journald.service: Service has no hold-off time, scheduling restart. >[ 14.761897] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. >[ 14.761920] systemd[1]: Stopped Journal Service. >[ 14.772187] systemd[1]: Starting Journal Service... >[ 14.780640] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point. >[ 14.780784] systemd[1]: Listening on LVM2 metadata daemon socket. >[ 15.640104] systemd-journald[716]: Received request to flush runtime journal from PID 1 >[ 16.560502] synth uevent: /devices/vio: failed to send uevent >[ 16.560508] vio vio: uevent: failed to send synthetic uevent >[ 16.561364] synth uevent: /devices/vio/4000: failed to send uevent >[ 16.561365] vio 4000: uevent: failed to send synthetic uevent >[ 16.561382] synth uevent: /devices/vio/4001: failed to send uevent >[ 16.561383] vio 4001: uevent: failed to send synthetic uevent >[ 16.561399] synth uevent: /devices/vio/4002: failed to send uevent >[ 16.561400] vio 4002: uevent: failed to send synthetic uevent >[ 16.561416] synth uevent: /devices/vio/4004: failed to send uevent >[ 16.561418] vio 4004: uevent: failed to send synthetic uevent >[ 16.581381] kauditd_printk_skb: 40 callbacks suppressed >[ 16.581383] audit: type=1130 audit(1537243689.880:65): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-udev-trigger comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 16.810858] audit: type=1130 audit(1537243690.110:66): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-udevd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 18.840243] crypto_register_alg 'aes' = 0 >[ 18.859746] crypto_register_alg 'cbc(aes)' = 0 >[ 18.870383] crypto_register_alg 'ctr(aes)' = 0 >[ 18.900491] crypto_register_alg 'xts(aes)' = 0 >[ 29.610827] pseries_rng: Registering IBM pSeries RNG driver >[ 30.130877] st: Version 20160209, fixed bufsize 32768, s/g segs 256 >[ 30.140836] st 0:2:0:0: Attached scsi tape st0 >[ 30.140860] st 0:2:0:0: st0: try direct i/o: yes (alignment 4 B) >[ 30.280756] osst :I: Tape driver with OnStream support version 0.99.4 > osst :I: $Id: osst.c,v 1.73 2005/01/01 21:13:34 wriede Exp $ >[ 31.030676] audit: type=1130 audit(1537243704.330:67): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-udev-settle comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.050811] audit: type=1130 audit(1537243704.350:68): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=lvm2-lvmetad comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.070424] Adding 2052032k swap on /dev/mapper/fedora_jkohan-swap. Priority:-2 extents:1 across:2052032k FS >[ 31.460967] audit: type=1130 audit(1537243704.760:69): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=lvm2-monitor comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.700259] audit: type=1130 audit(1537243705.000:70): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-fsck@dev-disk-by\x2duuid-08c7324c\x2de4b1\x2d468e\x2d8ac9\x2da8da953dd63b comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.770965] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null) >[ 31.950225] audit: type=1130 audit(1537243705.250:71): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=lvm2-pvscan@8:33 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.960338] audit: type=1130 audit(1537243705.260:72): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=plymouth-read-write comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.960362] audit: type=1131 audit(1537243705.260:73): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=plymouth-read-write comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.969932] audit: type=1130 audit(1537243705.260:74): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=dracut-shutdown comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.970842] audit: type=1130 audit(1537243705.270:75): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=nfs-config comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 31.970862] audit: type=1131 audit(1537243705.270:76): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=nfs-config comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' >[ 32.481221] RPC: Registered named UNIX socket transport module. >[ 32.481243] RPC: Registered udp transport module. >[ 32.481247] RPC: Registered tcp transport module. >[ 32.481251] RPC: Registered tcp NFSv4.1 backchannel transport module. >[ 37.030964] IPv6: ADDRCONF(NETDEV_UP): env2: link is not ready >[ 4044.879908] env2: Budget exhausted after napi rescheduled >[29190.070215] irqbalance invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29190.070249] irqbalance cpuset=/ mems_allowed=0 >[29190.070276] CPU: 10 PID: 2089 Comm: irqbalance Not tainted 4.18.7-200.fc28.ppc64 #1 >[29190.070290] Call Trace: >[29190.070307] [c0000000607db380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29190.070329] [c0000000607db410] [c0000000002fb670] .dump_header+0x70/0x300 >[29190.070347] [c0000000607db4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29190.070366] [c0000000607db5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29190.070385] [c0000000607db660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29190.070406] [c0000000607db850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29190.070425] [c0000000607db8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29190.070445] [c0000000607db980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29190.070750] [c0000000607dba50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29190.070771] [c0000000607dbb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29190.070789] [c0000000607dbb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29190.070809] [c0000000607dbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29190.070830] [c0000000607dbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29190.070849] [c0000000607dbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29190.070864] Mem-Info: >[29190.070884] active_anon:40 inactive_anon:53 isolated_anon:0 > active_file:74 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:52 unstable:0 > slab_reclaimable:2795 slab_unreclaimable:15715 > mapped:59 shmem:1 pagetables:87 bounce:0 > free:1464 free_pcp:1 free_cma:0 >[29190.070940] Node 0 active_anon:2560kB inactive_anon:3392kB active_file:4736kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3776kB dirty:0kB writeback:3328kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29190.070971] Node 0 DMA free:93696kB min:94144kB low:117632kB high:141120kB active_anon:5568kB inactive_anon:640kB active_file:4416kB inactive_file:0kB unevictable:0kB writepending:1152kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:14816kB pagetables:5568kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29190.071017] lowmem_reserve[]: 0 0 0 0 >[29190.071035] Node 0 DMA: 381*64kB (UM) 362*128kB (UM) 107*256kB (UM) 5*512kB (M) 3*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103744kB >[29190.071092] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29190.071109] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29190.071123] 182 total pagecache pages >[29190.071133] 0 pages in swap cache >[29190.071144] Swap cache stats: add 2921379, delete 2921485, find 2478596/4198862 >[29190.071157] Free swap = 1463168kB >[29190.071165] Total swap = 2052032kB >[29190.079952] 32768 pages RAM >[29190.079965] 0 pages HighMem/MovableOnly >[29190.079976] 2684 pages reserved >[29190.079986] 0 pages cma reserved >[29190.079996] 0 pages hwpoisoned >[29190.080006] Unreclaimable slab info: >[29190.080015] Name Used Total >[29190.080241] xfs_efd_item 1017KB 1017KB >[29190.080256] xfs_buf_item 1020KB 1020KB >[29190.080269] xfs_trans 1022KB 1022KB >[29190.080283] xfs_da_state 1020KB 1020KB >[29190.080296] xfs_log_ticket 1023KB 1023KB >[29190.080316] scsi_sense_cache 1024KB 1024KB >[29190.080351] RAWv6 504KB 504KB >[29190.080363] UDPv6 1012KB 1012KB >[29190.080376] tw_sock_TCPv6 127KB 127KB >[29190.080390] TCPv6 1386KB 1386KB >[29190.080405] cfq_io_cq 1023KB 1023KB >[29190.080419] cfq_queue 1023KB 1023KB >[29190.080433] mqueue_inode_cache 64KB 64KB >[29190.080447] kioctx 63KB 63KB >[29190.080463] posix_timers_cache 1022KB 1022KB >[29190.080475] ip4-frags 1023KB 1023KB >[29190.080488] secpath_cache 768KB 768KB >[29190.080502] RAW 896KB 896KB >[29190.080514] UDP 1008KB 1008KB >[29190.080527] tw_sock_TCP 127KB 127KB >[29190.080540] request_sock_TCP 191KB 191KB >[29190.080553] TCP 2040KB 2040KB >[29190.080566] hugetlbfs_inode_cache 127KB 127KB >[29190.080578] eventpoll_pwq 1023KB 1023KB >[29190.080590] inotify_inode_mark 1023KB 1023KB >[29190.080602] request_queue 755KB 755KB >[29190.080615] blkdev_ioc 1023KB 1023KB >[29190.080627] biovec-128 256KB 256KB >[29190.080640] biovec-64 128KB 128KB >[29190.080654] skbuff_fclone_cache 1024KB 1024KB >[29190.080666] skbuff_head_cache 1600KB 1600KB >[29190.080679] file_lock_cache 1021KB 1021KB >[29190.080693] net_namespace 510KB 510KB >[29190.080706] shmem_inode_cache 1087KB 1087KB >[29190.080721] dtl 10040KB 11520KB >[29190.080734] taskstats 701KB 701KB >[29190.080747] proc_dir_entry 640KB 640KB >[29190.080760] pde_opener 1023KB 1023KB >[29190.080773] seq_file 1024KB 1024KB >[29190.080785] sigqueue 1022KB 1022KB >[29190.080798] kernfs_node_cache 5302KB 5302KB >[29190.080811] mnt_cache 2996KB 2996KB >[29190.080824] names_cache 4096KB 4096KB >[29190.080838] ebitmap_node 1024KB 1024KB >[29190.080851] avtab_node 1023KB 1023KB >[29190.080864] avc_xperms_data 1024KB 1024KB >[29190.080877] selinux_file_security 1024KB 1024KB >[29190.080889] selinux_inode_security 1855KB 1855KB >[29190.080903] nsproxy 1023KB 1023KB >[29190.080915] vm_area_struct 1660KB 1660KB >[29190.080928] mm_struct 8056KB 8056KB >[29190.080940] files_cache 1211KB 1211KB >[29190.080953] signal_cache 1472KB 1472KB >[29190.080967] sighand_cache 2932KB 2932KB >[29190.080980] task_struct 12210KB 12210KB >[29190.080993] cred_jar 4800KB 4800KB >[29190.081007] thread_stack 17408KB 17408KB >[29190.081021] anon_vma 1086KB 1086KB >[29190.081035] pid 1280KB 1280KB >[29190.081048] numa_policy 383KB 383KB >[29190.081061] trace_event_file 255KB 255KB >[29190.081074] ftrace_event_field 447KB 447KB >[29190.081087] pool_workqueue 1024KB 1024KB >[29190.081100] task_group 1020KB 1020KB >[29190.081113] pgtable-2^11 9216KB 9216KB >[29190.081126] pgtable-2^8 4096KB 4096KB >[29190.081143] dma-kmalloc-512 128KB 128KB >[29190.089961] kmalloc-131072 35328KB 35328KB >[29190.089976] kmalloc-65536 9216KB 9216KB >[29190.089990] kmalloc-32768 8192KB 8192KB >[29190.090003] kmalloc-16384 18432KB 20480KB >[29190.090015] kmalloc-8192 8704KB 8704KB >[29190.090026] kmalloc-4096 8192KB 8192KB >[29190.090039] kmalloc-2048 4352KB 4352KB >[29190.090053] kmalloc-1024 19123KB 19456KB >[29190.090068] kmalloc-512 10176KB 11072KB >[29190.090082] kmalloc-256 1408KB 1408KB >[29190.090094] kmalloc-192 1790KB 1790KB >[29190.090107] kmalloc-128 1536KB 1536KB >[29190.090120] kmalloc-96 1470KB 1470KB >[29190.090133] kmalloc-64 1792KB 1792KB >[29190.090146] kmalloc-32 12992KB 12992KB >[29190.090158] kmalloc-16 1024KB 1024KB >[29190.090171] kmalloc-8 1024KB 1024KB >[29190.090184] kmem_cache_node 1024KB 1024KB >[29190.090197] kmem_cache 3315KB 3315KB >[29190.090208] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29190.090238] [ 716] 0 716 2668 0 69632 88 0 systemd-journal >[29190.090259] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29190.090279] [ 2033] 0 2033 346 0 49152 61 -1000 auditd >[29190.090298] [ 2067] 0 2067 328 0 51200 82 0 systemd-logind >[29190.090317] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29190.090335] [ 2069] 81 2069 208 0 51200 60 -900 dbus-daemon >[29190.090354] [ 2089] 0 2089 1288 0 51200 43 0 irqbalance >[29190.090372] [ 2093] 0 2093 7324 0 63488 190 0 NetworkManager >[29190.090391] [ 2094] 0 2094 10692 0 122880 94 0 rsyslogd >[29190.090409] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29190.090426] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29190.090444] [ 2102] 0 2102 4079 0 57344 70 0 gssproxy >[29190.090461] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29190.090478] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29190.090493] [ 2139] 0 2139 97 0 51200 37 0 atd >[29190.090511] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29190.090529] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29190.090547] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29190.090565] [ 2182] 25 2182 21852 0 131072 3398 0 named >[29190.090582] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29190.090600] [ 2314] 27 2314 29913 0 141312 1504 0 mysqld >[29190.090618] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29190.090635] [ 2337] 0 2337 449 0 53248 90 0 keepalived >[29190.090654] [ 2338] 0 2338 452 0 53248 89 0 keepalived >[29190.090673] [ 2339] 0 2339 474 0 55296 99 0 keepalived >[29190.090692] [ 2607] 177 2607 534 0 53248 177 0 dhcpd >[29190.090710] [ 2695] 0 2695 91 0 49152 19 0 dovecot >[29190.090727] [ 2696] 97 2696 84 0 51200 21 0 anvil >[29190.090745] [ 2697] 0 2697 87 2 49152 18 0 log >[29190.090764] [ 2699] 0 2699 103 0 51200 47 0 config >[29190.090782] [ 2711] 0 2711 398 0 51200 114 0 sendmail >[29190.090799] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29190.090817] [ 2723] 0 2723 699 1 53248 291 0 /usr/sbin/httpd >[29190.090836] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29190.090855] [ 2725] 48 2725 39835 0 198656 604 0 /usr/sbin/httpd >[29190.090874] [ 2726] 48 2726 36747 0 178176 576 0 /usr/sbin/httpd >[29190.090892] [ 2727] 48 2727 36747 0 178176 572 0 /usr/sbin/httpd >[29190.090911] [ 2971] 1000 2971 156 0 51200 40 0 imap >[29190.090929] [ 3367] 48 3367 36747 0 178176 575 0 /usr/sbin/httpd >[29190.090947] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29190.090965] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29190.090982] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29190.090999] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29190.091017] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29190.091035] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29190.091053] [ 8357] 0 8357 4883 0 63488 294 0 php-fpm >[29190.091070] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29190.091089] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29190.091108] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29190.091126] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29190.091145] [ 8900] 0 8900 410 0 51200 118 0 sendmail >[29190.091163] [ 8907] 0 8907 411 0 51200 133 0 sendmail >[29190.099962] [ 8908] 0 8908 3270 0 51200 27 0 rsync >[29190.099981] [ 8909] 0 8909 209 0 49152 63 0 ssh >[29190.099999] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29190.100016] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29190.100035] [ 8912] 0 8912 402 1 51200 112 0 sendmail >[29190.100053] [ 8913] 0 8913 166 0 40960 36 0 ps >[29190.100071] [ 8914] 0 8914 3249 0 53248 14 0 head >[29190.100089] [ 8915] 0 8915 159 0 51200 45 0 pop3-login >[29190.100104] Out of memory: Kill process 2182 (named) score 54 or sacrifice child >[29190.100158] Killed process 2182 (named) total-vm:1398528kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29190.220261] oom_reaper: reaped process 2182 (named), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29203.860385] mysqld invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29203.860414] mysqld cpuset=/ mems_allowed=0 >[29203.860438] CPU: 0 PID: 2616 Comm: mysqld Not tainted 4.18.7-200.fc28.ppc64 #1 >[29203.860450] Call Trace: >[29203.860466] [c00000003933f380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29203.860487] [c00000003933f410] [c0000000002fb670] .dump_header+0x70/0x300 >[29203.860505] [c00000003933f4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29203.860523] [c00000003933f5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29203.860541] [c00000003933f660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29203.860564] [c00000003933f850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29203.860585] [c00000003933f8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29203.860603] [c00000003933f980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29203.860881] [c00000003933fa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29203.860901] [c00000003933fb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29203.860919] [c00000003933fb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29203.860938] [c00000003933fcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29203.860958] [c00000003933fd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29203.860977] [c00000003933fe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29203.861052] Mem-Info: >[29203.861073] active_anon:34 inactive_anon:46 isolated_anon:0 > active_file:29 inactive_file:64 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2796 slab_unreclaimable:15722 > mapped:77 shmem:0 pagetables:85 bounce:0 > free:1457 free_pcp:0 free_cma:0 >[29203.861125] Node 0 active_anon:2176kB inactive_anon:2944kB active_file:1856kB inactive_file:4096kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4928kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29203.861154] Node 0 DMA free:93248kB min:94144kB low:117632kB high:141120kB active_anon:384kB inactive_anon:3072kB active_file:320kB inactive_file:1856kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:14496kB pagetables:5440kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29203.869970] lowmem_reserve[]: 0 0 0 0 >[29203.869990] Node 0 DMA: 385*64kB (UM) 363*128kB (UM) 108*256kB (UM) 4*512kB (M) 2*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102848kB >[29203.870048] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29203.870065] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29203.870080] 170 total pagecache pages >[29203.870090] 9 pages in swap cache >[29203.870101] Swap cache stats: add 2938770, delete 2938887, find 2493312/4225382 >[29203.870113] Free swap = 1638656kB >[29203.870123] Total swap = 2052032kB >[29203.870133] 32768 pages RAM >[29203.870141] 0 pages HighMem/MovableOnly >[29203.870150] 2684 pages reserved >[29203.870159] 0 pages cma reserved >[29203.870168] 0 pages hwpoisoned >[29203.870178] Unreclaimable slab info: >[29203.870186] Name Used Total >[29203.870412] xfs_efd_item 1017KB 1017KB >[29203.870423] xfs_buf_item 1020KB 1020KB >[29203.870437] xfs_trans 1022KB 1022KB >[29203.870450] xfs_da_state 1020KB 1020KB >[29203.870463] xfs_log_ticket 1023KB 1023KB >[29203.870483] scsi_sense_cache 1024KB 1024KB >[29203.870516] RAWv6 504KB 504KB >[29203.870529] UDPv6 1012KB 1012KB >[29203.870541] tw_sock_TCPv6 127KB 127KB >[29203.870555] TCPv6 1386KB 1386KB >[29203.870569] cfq_io_cq 1023KB 1023KB >[29203.870582] cfq_queue 1023KB 1023KB >[29203.870595] mqueue_inode_cache 64KB 64KB >[29203.870609] kioctx 63KB 63KB >[29203.870624] posix_timers_cache 1022KB 1022KB >[29203.870636] ip4-frags 1023KB 1023KB >[29203.870648] secpath_cache 768KB 768KB >[29203.870659] RAW 896KB 896KB >[29203.870670] UDP 1008KB 1008KB >[29203.870683] tw_sock_TCP 127KB 127KB >[29203.870696] request_sock_TCP 191KB 191KB >[29203.870710] TCP 2040KB 2040KB >[29203.870722] hugetlbfs_inode_cache 127KB 127KB >[29203.870736] eventpoll_pwq 1023KB 1023KB >[29203.870749] inotify_inode_mark 1023KB 1023KB >[29203.870762] request_queue 755KB 755KB >[29203.870776] blkdev_ioc 1023KB 1023KB >[29203.870788] biovec-128 256KB 256KB >[29203.870801] biovec-64 128KB 128KB >[29203.870814] skbuff_fclone_cache 1024KB 1024KB >[29203.870826] skbuff_head_cache 1600KB 1600KB >[29203.870838] file_lock_cache 1021KB 1021KB >[29203.870851] net_namespace 510KB 510KB >[29203.870863] shmem_inode_cache 1087KB 1087KB >[29203.870876] dtl 10040KB 11520KB >[29203.870888] taskstats 764KB 764KB >[29203.870900] proc_dir_entry 640KB 640KB >[29203.870912] pde_opener 1023KB 1023KB >[29203.870925] seq_file 1024KB 1024KB >[29203.870937] sigqueue 1022KB 1022KB >[29203.870949] kernfs_node_cache 5302KB 5302KB >[29203.870961] mnt_cache 2996KB 2996KB >[29203.870972] names_cache 4096KB 4096KB >[29203.870985] ebitmap_node 1024KB 1024KB >[29203.870998] avtab_node 1023KB 1023KB >[29203.871011] avc_xperms_data 1024KB 1024KB >[29203.871023] selinux_file_security 1024KB 1024KB >[29203.871036] selinux_inode_security 1855KB 1855KB >[29203.871050] nsproxy 1023KB 1023KB >[29203.871062] vm_area_struct 1660KB 1660KB >[29203.871075] mm_struct 8056KB 8056KB >[29203.871087] files_cache 1211KB 1211KB >[29203.871099] signal_cache 1472KB 1472KB >[29203.871111] sighand_cache 2932KB 2932KB >[29203.871123] task_struct 12210KB 12210KB >[29203.871136] cred_jar 4800KB 4800KB >[29203.871148] thread_stack 17408KB 17408KB >[29203.879967] anon_vma 1086KB 1086KB >[29203.879980] pid 1280KB 1280KB >[29203.879992] numa_policy 383KB 383KB >[29203.880005] trace_event_file 255KB 255KB >[29203.880017] ftrace_event_field 447KB 447KB >[29203.880030] pool_workqueue 1024KB 1024KB >[29203.880042] task_group 1020KB 1020KB >[29203.880054] pgtable-2^11 9216KB 9216KB >[29203.880066] pgtable-2^8 4096KB 4096KB >[29203.880083] dma-kmalloc-512 128KB 128KB >[29203.880101] kmalloc-131072 35328KB 35328KB >[29203.880114] kmalloc-65536 9216KB 9216KB >[29203.880126] kmalloc-32768 8192KB 8192KB >[29203.880139] kmalloc-16384 18432KB 20480KB >[29203.880151] kmalloc-8192 8704KB 8704KB >[29203.880163] kmalloc-4096 8192KB 8192KB >[29203.880175] kmalloc-2048 4352KB 4352KB >[29203.880189] kmalloc-1024 19123KB 19456KB >[29203.880203] kmalloc-512 10176KB 11072KB >[29203.880216] kmalloc-256 1408KB 1408KB >[29203.880228] kmalloc-192 1790KB 1790KB >[29203.880241] kmalloc-128 1536KB 1536KB >[29203.880254] kmalloc-96 1470KB 1470KB >[29203.880266] kmalloc-64 1792KB 1792KB >[29203.880278] kmalloc-32 12992KB 12992KB >[29203.880291] kmalloc-16 1024KB 1024KB >[29203.880304] kmalloc-8 1024KB 1024KB >[29203.880316] kmem_cache_node 1024KB 1024KB >[29203.880328] kmem_cache 3315KB 3315KB >[29203.880338] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29203.880373] [ 716] 0 716 2668 0 69632 99 0 systemd-journal >[29203.880391] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29203.880411] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29203.880430] [ 2067] 0 2067 328 0 51200 92 0 systemd-logind >[29203.880448] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29203.880465] [ 2069] 81 2069 208 0 51200 60 -900 dbus-daemon >[29203.880483] [ 2089] 0 2089 1288 0 51200 44 0 irqbalance >[29203.880501] [ 2093] 0 2093 7324 0 63488 195 0 NetworkManager >[29203.880519] [ 2094] 0 2094 10692 0 122880 107 0 rsyslogd >[29203.880537] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29203.880555] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29203.880571] [ 2102] 0 2102 4079 0 57344 70 0 gssproxy >[29203.880589] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29203.880606] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29203.880623] [ 2139] 0 2139 97 0 51200 37 0 atd >[29203.880640] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29203.880658] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29203.880676] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29203.880694] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29203.880711] [ 2314] 27 2314 29913 0 141312 1494 0 mysqld >[29203.880729] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29203.880746] [ 2337] 0 2337 449 0 53248 84 0 keepalived >[29203.880764] [ 2338] 0 2338 452 0 53248 87 0 keepalived >[29203.880783] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29203.880801] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29203.880818] [ 2695] 0 2695 91 2 49152 19 0 dovecot >[29203.880836] [ 2696] 97 2696 84 0 51200 18 0 anvil >[29203.880853] [ 2697] 0 2697 87 0 49152 28 0 log >[29203.880871] [ 2699] 0 2699 103 2 51200 35 0 config >[29203.880889] [ 2711] 0 2711 398 0 51200 109 0 sendmail >[29203.880907] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29203.880926] [ 2723] 0 2723 699 0 53248 300 0 /usr/sbin/httpd >[29203.880945] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29203.880964] [ 2725] 48 2725 39835 0 198656 604 0 /usr/sbin/httpd >[29203.880983] [ 2726] 48 2726 36747 0 178176 564 0 /usr/sbin/httpd >[29203.881001] [ 2727] 48 2727 36747 0 178176 579 0 /usr/sbin/httpd >[29203.881017] [ 2971] 1000 2971 156 0 51200 48 0 imap >[29203.881035] [ 3367] 48 3367 36747 0 178176 578 0 /usr/sbin/httpd >[29203.881054] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29203.881070] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29203.881088] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29203.881105] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29203.881123] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29203.881141] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29203.881158] [ 8357] 0 8357 4883 0 63488 304 0 php-fpm >[29203.889940] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29203.889974] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29203.889992] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29203.890010] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29203.890028] [ 8900] 0 8900 410 0 51200 129 0 sendmail >[29203.890047] [ 8907] 0 8907 411 0 51200 121 0 sendmail >[29203.890063] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29203.890080] [ 8909] 0 8909 209 0 49152 60 0 ssh >[29203.890096] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29203.890113] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29203.890130] [ 8912] 0 8912 407 0 51200 120 0 sendmail >[29203.890147] [ 8913] 0 8913 166 1 40960 38 0 ps >[29203.890164] [ 8914] 0 8914 3249 0 53248 14 0 head >[29203.890180] [ 8918] 0 8918 157 1 40960 24 0 pop3-login >[29203.890196] Out of memory: Kill process 2314 (mysqld) score 24 or sacrifice child >[29203.890347] Killed process 2314 (mysqld) total-vm:1914432kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29203.910773] mysqld[2616]: bad frame in setup_rt_frame: 00007fff85fbcca0 nip 00000001296e372c lr 00000001296e372c >[29203.910833] mysqld[2616]: bad frame in setup_rt_frame: 00007fff85fbcca0 nip 00000001296e372c lr 00000001296e372c >[29203.950551] oom_reaper: reaped process 2314 (mysqld), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29208.140070] /usr/sbin/httpd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29208.140102] /usr/sbin/httpd cpuset=/ mems_allowed=0 >[29208.140127] CPU: 13 PID: 2728 Comm: /usr/sbin/httpd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29208.140141] Call Trace: >[29208.140158] [c0000000053bb1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29208.140179] [c0000000053bb230] [c0000000002fb670] .dump_header+0x70/0x300 >[29208.140197] [c0000000053bb300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29208.140216] [c0000000053bb3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29208.140235] [c0000000053bb480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29208.140256] [c0000000053bb670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29208.140276] [c0000000053bb730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29208.140296] [c0000000053bb7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29208.140315] [c0000000053bb880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29208.140334] [c0000000053bb990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29208.140354] [c0000000053bbac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29208.140372] [c0000000053bbb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29208.140391] [c0000000053bbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29208.140412] [c0000000053bbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29208.140433] [c0000000053bbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29208.140448] Mem-Info: >[29208.140468] active_anon:0 inactive_anon:37 isolated_anon:0 > active_file:19 inactive_file:8 isolated_file:0 > unevictable:0 dirty:2 writeback:5 unstable:0 > slab_reclaimable:2799 slab_unreclaimable:15722 > mapped:33 shmem:0 pagetables:82 bounce:0 > free:1465 free_pcp:0 free_cma:0 >[29208.140522] Node 0 active_anon:0kB inactive_anon:2368kB active_file:1216kB inactive_file:512kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2112kB dirty:128kB writeback:320kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29208.140552] Node 0 DMA free:93760kB min:94144kB low:117632kB high:141120kB active_anon:576kB inactive_anon:0kB active_file:4160kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:14032kB pagetables:5248kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29208.140596] lowmem_reserve[]: 0 0 0 0 >[29208.140615] Node 0 DMA: 384*64kB (UM) 362*128kB (UM) 107*256kB (UM) 7*512kB (ME) 3*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 104960kB >[29208.140672] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29208.140689] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29208.140703] 194 total pagecache pages >[29208.140713] 0 pages in swap cache >[29208.140724] Swap cache stats: add 2944604, delete 2944732, find 2498507/4234625 >[29208.140736] Free swap = 1720576kB >[29208.140745] Total swap = 2052032kB >[29208.140755] 32768 pages RAM >[29208.140763] 0 pages HighMem/MovableOnly >[29208.140772] 2684 pages reserved >[29208.140781] 0 pages cma reserved >[29208.140789] 0 pages hwpoisoned >[29208.140798] Unreclaimable slab info: >[29208.140808] Name Used Total >[29208.141028] xfs_efd_item 1017KB 1017KB >[29208.141041] xfs_buf_item 1020KB 1020KB >[29208.141054] xfs_trans 1022KB 1022KB >[29208.141066] xfs_da_state 1020KB 1020KB >[29208.141079] xfs_log_ticket 1023KB 1023KB >[29208.141098] scsi_sense_cache 1024KB 1024KB >[29208.141131] RAWv6 504KB 504KB >[29208.141145] UDPv6 1012KB 1012KB >[29208.141157] tw_sock_TCPv6 127KB 127KB >[29208.141170] TCPv6 1386KB 1386KB >[29208.149967] cfq_io_cq 1023KB 1023KB >[29208.149983] cfq_queue 1023KB 1023KB >[29208.149996] mqueue_inode_cache 64KB 64KB >[29208.150010] kioctx 63KB 63KB >[29208.150025] posix_timers_cache 1022KB 1022KB >[29208.150038] ip4-frags 1023KB 1023KB >[29208.150051] secpath_cache 768KB 768KB >[29208.150064] RAW 896KB 896KB >[29208.150076] UDP 1008KB 1008KB >[29208.150089] tw_sock_TCP 127KB 127KB >[29208.150101] request_sock_TCP 191KB 191KB >[29208.150113] TCP 2040KB 2040KB >[29208.150126] hugetlbfs_inode_cache 127KB 127KB >[29208.150138] eventpoll_pwq 1023KB 1023KB >[29208.150151] inotify_inode_mark 1023KB 1023KB >[29208.150163] request_queue 755KB 755KB >[29208.150176] blkdev_ioc 1023KB 1023KB >[29208.150188] biovec-128 256KB 256KB >[29208.150200] biovec-64 128KB 128KB >[29208.150212] skbuff_fclone_cache 1024KB 1024KB >[29208.150223] skbuff_head_cache 1600KB 1600KB >[29208.150235] file_lock_cache 1021KB 1021KB >[29208.150247] net_namespace 510KB 510KB >[29208.150259] shmem_inode_cache 1087KB 1087KB >[29208.150273] dtl 10040KB 11520KB >[29208.150285] taskstats 764KB 764KB >[29208.150298] proc_dir_entry 640KB 640KB >[29208.150311] pde_opener 1023KB 1023KB >[29208.150324] seq_file 1024KB 1024KB >[29208.150337] sigqueue 1022KB 1022KB >[29208.150349] kernfs_node_cache 5302KB 5302KB >[29208.150361] mnt_cache 2996KB 2996KB >[29208.150374] names_cache 4096KB 4096KB >[29208.150388] ebitmap_node 1024KB 1024KB >[29208.150400] avtab_node 1023KB 1023KB >[29208.150412] avc_xperms_data 1024KB 1024KB >[29208.150424] selinux_file_security 1024KB 1024KB >[29208.150436] selinux_inode_security 1855KB 1855KB >[29208.150449] nsproxy 1023KB 1023KB >[29208.150461] vm_area_struct 1660KB 1660KB >[29208.150473] mm_struct 8056KB 8056KB >[29208.150486] files_cache 1211KB 1211KB >[29208.150498] signal_cache 1472KB 1472KB >[29208.150510] sighand_cache 2932KB 2932KB >[29208.150522] task_struct 12210KB 12210KB >[29208.150535] cred_jar 4800KB 4800KB >[29208.150548] thread_stack 17408KB 17408KB >[29208.150559] anon_vma 1086KB 1086KB >[29208.150571] pid 1280KB 1280KB >[29208.150583] numa_policy 383KB 383KB >[29208.150595] trace_event_file 255KB 255KB >[29208.150607] ftrace_event_field 447KB 447KB >[29208.150619] pool_workqueue 1024KB 1024KB >[29208.150632] task_group 1020KB 1020KB >[29208.150645] pgtable-2^11 9216KB 9216KB >[29208.150657] pgtable-2^8 4096KB 4096KB >[29208.150675] dma-kmalloc-512 128KB 128KB >[29208.150692] kmalloc-131072 35328KB 35328KB >[29208.150705] kmalloc-65536 9216KB 9216KB >[29208.150717] kmalloc-32768 8192KB 8192KB >[29208.150730] kmalloc-16384 18432KB 20480KB >[29208.150742] kmalloc-8192 8704KB 8704KB >[29208.150754] kmalloc-4096 8192KB 8192KB >[29208.150766] kmalloc-2048 4352KB 4352KB >[29208.150780] kmalloc-1024 19123KB 19456KB >[29208.150793] kmalloc-512 10176KB 11072KB >[29208.150805] kmalloc-256 1408KB 1408KB >[29208.150818] kmalloc-192 1790KB 1790KB >[29208.150829] kmalloc-128 1536KB 1536KB >[29208.150842] kmalloc-96 1470KB 1470KB >[29208.150853] kmalloc-64 1792KB 1792KB >[29208.150866] kmalloc-32 12992KB 12992KB >[29208.150878] kmalloc-16 1024KB 1024KB >[29208.150891] kmalloc-8 1024KB 1024KB >[29208.150903] kmem_cache_node 1024KB 1024KB >[29208.150917] kmem_cache 3315KB 3315KB >[29208.150927] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29208.150962] [ 716] 0 716 2668 0 69632 93 0 systemd-journal >[29208.150982] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29208.151002] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29208.151020] [ 2067] 0 2067 328 0 51200 81 0 systemd-logind >[29208.151037] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29208.151054] [ 2069] 81 2069 208 0 51200 61 -900 dbus-daemon >[29208.151072] [ 2089] 0 2089 1288 1 51200 33 0 irqbalance >[29208.151090] [ 2093] 0 2093 7324 0 63488 208 0 NetworkManager >[29208.151109] [ 2094] 0 2094 10692 0 122880 88 0 rsyslogd >[29208.151126] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29208.151144] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29208.159961] [ 2102] 0 2102 4079 0 57344 70 0 gssproxy >[29208.159979] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29208.159997] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29208.160014] [ 2139] 0 2139 97 0 51200 37 0 atd >[29208.160032] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29208.160049] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29208.160066] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29208.160083] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29208.160100] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29208.160117] [ 2337] 0 2337 449 0 53248 91 0 keepalived >[29208.160134] [ 2338] 0 2338 452 0 53248 82 0 keepalived >[29208.160152] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29208.160171] [ 2607] 177 2607 534 0 53248 183 0 dhcpd >[29208.160189] [ 2695] 0 2695 91 2 49152 19 0 dovecot >[29208.160206] [ 2696] 97 2696 84 0 51200 18 0 anvil >[29208.160223] [ 2697] 0 2697 87 0 49152 28 0 log >[29208.160240] [ 2699] 0 2699 103 0 51200 38 0 config >[29208.160257] [ 2711] 0 2711 398 1 51200 112 0 sendmail >[29208.160275] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29208.160292] [ 2723] 0 2723 699 0 53248 297 0 /usr/sbin/httpd >[29208.160311] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29208.160329] [ 2725] 48 2725 39835 0 198656 602 0 /usr/sbin/httpd >[29208.160347] [ 2726] 48 2726 36747 0 178176 563 0 /usr/sbin/httpd >[29208.160365] [ 2727] 48 2727 36747 0 178176 587 0 /usr/sbin/httpd >[29208.160382] [ 2971] 1000 2971 156 0 51200 43 0 imap >[29208.160400] [ 3367] 48 3367 36747 0 178176 576 0 /usr/sbin/httpd >[29208.160418] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29208.160436] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29208.160453] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29208.160469] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29208.160486] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29208.160505] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29208.160521] [ 8357] 0 8357 4883 0 63488 298 0 php-fpm >[29208.160539] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29208.160557] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29208.160575] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29208.160593] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29208.160611] [ 8900] 0 8900 410 0 51200 121 0 sendmail >[29208.160629] [ 8907] 0 8907 410 0 51200 133 0 sendmail >[29208.160646] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29208.160663] [ 8909] 0 8909 209 0 49152 62 0 ssh >[29208.160680] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29208.160697] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29208.160714] [ 8912] 0 8912 411 0 51200 131 0 sendmail >[29208.160731] [ 8913] 0 8913 166 0 40960 40 0 ps >[29208.160749] [ 8914] 0 8914 3249 0 53248 14 0 head >[29208.160767] [ 8918] 0 8918 156 1 40960 24 0 pop3-login >[29208.160783] Out of memory: Kill process 2725 (/usr/sbin/httpd) score 9 or sacrifice child >[29208.169992] Killed process 2725 (/usr/sbin/httpd) total-vm:2549440kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29208.181106] oom_reaper: reaped process 2725 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29223.990374] ssh invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29223.990402] ssh cpuset=/ mems_allowed=0 >[29223.990425] CPU: 4 PID: 8909 Comm: ssh Not tainted 4.18.7-200.fc28.ppc64 #1 >[29223.990436] Call Trace: >[29223.990452] [c000000009f37230] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29223.990472] [c000000009f372c0] [c0000000002fb670] .dump_header+0x70/0x300 >[29223.990489] [c000000009f37390] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29223.990507] [c000000009f37460] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29223.990525] [c000000009f37510] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29223.990544] [c000000009f37700] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29223.990563] [c000000009f377c0] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29223.990582] [c000000009f37880] [c00000000037a284] .swap_cluster_readahead+0x1e4/0x3e0 >[29223.990601] [c000000009f37990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29223.990618] [c000000009f37ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29223.990637] [c000000009f37b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29223.990654] [c000000009f37cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29223.990673] [c000000009f37d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29223.990692] [c000000009f37e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29223.990705] Mem-Info: >[29223.990724] active_anon:1 inactive_anon:0 isolated_anon:0 > active_file:23 inactive_file:25 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15755 > mapped:37 shmem:0 pagetables:78 bounce:0 > free:1467 free_pcp:0 free_cma:0 >[29223.990775] Node 0 active_anon:64kB inactive_anon:0kB active_file:1472kB inactive_file:1600kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2368kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29223.990802] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1792kB active_file:2816kB inactive_file:1088kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:12864kB pagetables:4992kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29223.990844] lowmem_reserve[]: 0 0 0 0 >[29223.990863] Node 0 DMA: 392*64kB (UM) 366*128kB (UM) 108*256kB (UM) 3*512kB (M) 2*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103168kB >[29223.990918] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29223.990934] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29223.990946] 72 total pagecache pages >[29223.990955] 5 pages in swap cache >[29223.990965] Swap cache stats: add 2964317, delete 2964428, find 2515890/4265557 >[29223.990977] Free swap = 1732864kB >[29223.990985] Total swap = 2052032kB >[29223.990995] 32768 pages RAM >[29223.991002] 0 pages HighMem/MovableOnly >[29223.991010] 2684 pages reserved >[29223.991018] 0 pages cma reserved >[29223.991026] 0 pages hwpoisoned >[29223.991035] Unreclaimable slab info: >[29223.991043] Name Used Total >[29224.000060] xfs_efd_item 1017KB 1017KB >[29224.000072] xfs_buf_item 1020KB 1020KB >[29224.000084] xfs_trans 1022KB 1022KB >[29224.000096] xfs_da_state 1020KB 1020KB >[29224.000107] xfs_log_ticket 1023KB 1023KB >[29224.000126] scsi_sense_cache 1024KB 1024KB >[29224.000157] RAWv6 504KB 504KB >[29224.000169] UDPv6 1012KB 1012KB >[29224.000180] tw_sock_TCPv6 127KB 127KB >[29224.000192] TCPv6 1386KB 1386KB >[29224.000206] cfq_io_cq 1023KB 1023KB >[29224.000219] cfq_queue 1023KB 1023KB >[29224.000232] mqueue_inode_cache 64KB 64KB >[29224.000245] kioctx 63KB 63KB >[29224.000260] posix_timers_cache 1022KB 1022KB >[29224.000272] ip4-frags 1023KB 1023KB >[29224.000284] secpath_cache 768KB 768KB >[29224.000296] RAW 896KB 896KB >[29224.000307] UDP 1008KB 1008KB >[29224.000319] tw_sock_TCP 127KB 127KB >[29224.000330] request_sock_TCP 191KB 191KB >[29224.000342] TCP 2040KB 2040KB >[29224.000354] hugetlbfs_inode_cache 127KB 127KB >[29224.000367] eventpoll_pwq 1023KB 1023KB >[29224.000377] inotify_inode_mark 1023KB 1023KB >[29224.000389] request_queue 755KB 755KB >[29224.000400] blkdev_ioc 1023KB 1023KB >[29224.000410] biovec-128 256KB 256KB >[29224.000418] biovec-64 128KB 128KB >[29224.000428] skbuff_fclone_cache 1024KB 1024KB >[29224.000438] skbuff_head_cache 1600KB 1600KB >[29224.000447] file_lock_cache 1021KB 1021KB >[29224.000457] net_namespace 510KB 510KB >[29224.000466] shmem_inode_cache 1087KB 1087KB >[29224.000477] dtl 10040KB 11520KB >[29224.000486] taskstats 764KB 764KB >[29224.000495] proc_dir_entry 640KB 640KB >[29224.000505] pde_opener 1023KB 1023KB >[29224.000514] seq_file 1024KB 1024KB >[29224.000524] sigqueue 1022KB 1022KB >[29224.000533] kernfs_node_cache 5302KB 5302KB >[29224.000543] mnt_cache 2996KB 2996KB >[29224.000553] names_cache 4096KB 4096KB >[29224.000563] ebitmap_node 1024KB 1024KB >[29224.000572] avtab_node 1023KB 1023KB >[29224.000582] avc_xperms_data 1024KB 1024KB >[29224.000591] selinux_file_security 1024KB 1024KB >[29224.000600] selinux_inode_security 1855KB 1855KB >[29224.000612] nsproxy 1023KB 1023KB >[29224.000621] vm_area_struct 1660KB 1660KB >[29224.000631] mm_struct 8056KB 8056KB >[29224.000640] files_cache 1211KB 1211KB >[29224.000649] signal_cache 1472KB 1472KB >[29224.000658] sighand_cache 2932KB 2932KB >[29224.000668] task_struct 12210KB 12210KB >[29224.000677] cred_jar 4800KB 4800KB >[29224.000686] thread_stack 17408KB 17408KB >[29224.000696] anon_vma 1086KB 1086KB >[29224.000705] pid 1280KB 1280KB >[29224.000715] numa_policy 383KB 383KB >[29224.000725] trace_event_file 255KB 255KB >[29224.000735] ftrace_event_field 447KB 447KB >[29224.000745] pool_workqueue 1024KB 1024KB >[29224.000755] task_group 1020KB 1020KB >[29224.000764] pgtable-2^11 9216KB 9216KB >[29224.000774] pgtable-2^8 4096KB 4096KB >[29224.000788] dma-kmalloc-512 128KB 128KB >[29224.000801] kmalloc-131072 35328KB 35328KB >[29224.000810] kmalloc-65536 9216KB 9216KB >[29224.000819] kmalloc-32768 8192KB 8192KB >[29224.000829] kmalloc-16384 18432KB 20480KB >[29224.000838] kmalloc-8192 8704KB 8704KB >[29224.000848] kmalloc-4096 8192KB 8192KB >[29224.000857] kmalloc-2048 4352KB 4352KB >[29224.000869] kmalloc-1024 19123KB 19456KB >[29224.000880] kmalloc-512 10176KB 11072KB >[29224.000889] kmalloc-256 1408KB 1408KB >[29224.000899] kmalloc-192 1790KB 1790KB >[29224.000909] kmalloc-128 1536KB 1536KB >[29224.000918] kmalloc-96 1470KB 1470KB >[29224.000927] kmalloc-64 1792KB 1792KB >[29224.000937] kmalloc-32 12992KB 12992KB >[29224.000946] kmalloc-16 1024KB 1024KB >[29224.000955] kmalloc-8 1024KB 1024KB >[29224.000965] kmem_cache_node 1024KB 1024KB >[29224.000975] kmem_cache 3315KB 3315KB >[29224.000983] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29224.001005] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29224.001020] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29224.001034] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29224.001048] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29224.001062] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29224.001076] [ 2069] 81 2069 208 0 51200 67 -900 dbus-daemon >[29224.001090] [ 2089] 0 2089 1288 1 51200 35 0 irqbalance >[29224.001104] [ 2093] 0 2093 7324 0 63488 204 0 NetworkManager >[29224.001118] [ 2094] 0 2094 10692 0 122880 88 0 rsyslogd >[29224.001132] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29224.001145] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29224.001158] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29224.001171] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29224.009967] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29224.009984] [ 2139] 0 2139 97 0 51200 37 0 atd >[29224.009999] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29224.010013] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29224.010028] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29224.010043] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29224.010059] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29224.010074] [ 2337] 0 2337 449 0 53248 78 0 keepalived >[29224.010089] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29224.010104] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29224.010120] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29224.010134] [ 2695] 0 2695 91 0 49152 23 0 dovecot >[29224.010150] [ 2696] 97 2696 84 0 51200 21 0 anvil >[29224.010165] [ 2697] 0 2697 87 1 49152 16 0 log >[29224.010179] [ 2699] 0 2699 103 0 51200 34 0 config >[29224.010194] [ 2711] 0 2711 398 1 51200 110 0 sendmail >[29224.010209] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29224.010223] [ 2723] 0 2723 699 0 53248 309 0 /usr/sbin/httpd >[29224.010240] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29224.010256] [ 2726] 48 2726 37650 0 186368 682 0 /usr/sbin/httpd >[29224.010272] [ 2727] 48 2727 36747 0 178176 573 0 /usr/sbin/httpd >[29224.010288] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29224.010302] [ 3367] 48 3367 36747 0 178176 570 0 /usr/sbin/httpd >[29224.010319] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29224.010334] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29224.010349] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29224.010364] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29224.010378] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29224.010393] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29224.010408] [ 8357] 0 8357 4883 0 63488 288 0 php-fpm >[29224.010423] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29224.010441] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29224.010456] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29224.010470] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29224.010486] [ 8900] 0 8900 410 0 51200 120 0 sendmail >[29224.010501] [ 8907] 0 8907 410 0 51200 126 0 sendmail >[29224.010516] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29224.010531] [ 8909] 0 8909 3410 0 51200 59 0 ssh >[29224.010546] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29224.010560] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29224.010575] [ 8912] 0 8912 411 0 51200 125 0 sendmail >[29224.010588] [ 8913] 0 8913 166 0 40960 46 0 ps >[29224.010601] [ 8914] 0 8914 3249 0 53248 14 0 head >[29224.010613] [ 8918] 990 8918 159 0 51200 40 0 pop3-login >[29224.010624] [ 8919] 0 8919 114 0 51200 42 0 auth >[29224.010634] [ 8920] 0 8920 159 0 51200 42 0 pop3-login >[29224.010644] Out of memory: Kill process 2726 (/usr/sbin/httpd) score 11 or sacrifice child >[29224.010998] Killed process 2726 (/usr/sbin/httpd) total-vm:2409600kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29224.039954] oom_reaper: reaped process 2726 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29224.090537] config invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29224.090554] config cpuset=/ mems_allowed=0 >[29224.090574] CPU: 12 PID: 2699 Comm: config Not tainted 4.18.7-200.fc28.ppc64 #1 >[29224.090587] Call Trace: >[29224.090602] [c000000019f471a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29224.090623] [c000000019f47230] [c0000000002fb670] .dump_header+0x70/0x300 >[29224.090641] [c000000019f47300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29224.090660] [c000000019f473d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29224.090679] [c000000019f47480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29224.090699] [c000000019f47670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29224.090719] [c000000019f47730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29224.090738] [c000000019f477f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29224.090758] [c000000019f47880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29224.090777] [c000000019f47990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29224.090796] [c000000019f47ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29224.090814] [c000000019f47b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29224.090832] [c000000019f47cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29224.090851] [c000000019f47d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29224.090871] [c000000019f47e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29224.090886] Mem-Info: >[29224.090906] active_anon:1 inactive_anon:0 isolated_anon:0 > active_file:23 inactive_file:25 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15755 > mapped:37 shmem:0 pagetables:78 bounce:0 > free:1467 free_pcp:1 free_cma:0 >[29224.090956] Node 0 active_anon:64kB inactive_anon:0kB active_file:1472kB inactive_file:1600kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2368kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29224.090986] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:0kB active_file:2816kB inactive_file:1088kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:12864kB pagetables:4992kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29224.091027] lowmem_reserve[]: 0 0 0 0 >[29224.091046] Node 0 DMA: 404*64kB (UM) 353*128kB (UM) 104*256kB (UM) 3*512kB (M) 2*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 101248kB >[29224.091103] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29224.091119] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29224.091133] 72 total pagecache pages >[29224.091142] 33 pages in swap cache >[29224.091152] Swap cache stats: add 2964394, delete 2964477, find 2515909/4265639 >[29224.091165] Free swap = 1745152kB >[29224.099938] Total swap = 2052032kB >[29224.099960] 32768 pages RAM >[29224.099969] 0 pages HighMem/MovableOnly >[29224.099978] 2684 pages reserved >[29224.099987] 0 pages cma reserved >[29224.099996] 0 pages hwpoisoned >[29224.100005] Unreclaimable slab info: >[29224.100014] Name Used Total >[29224.100147] xfs_efd_item 1017KB 1017KB >[29224.100160] xfs_buf_item 1020KB 1020KB >[29224.100172] xfs_trans 1022KB 1022KB >[29224.100185] xfs_da_state 1020KB 1020KB >[29224.100197] xfs_log_ticket 1023KB 1023KB >[29224.100213] scsi_sense_cache 1024KB 1024KB >[29224.100238] RAWv6 504KB 504KB >[29224.100250] UDPv6 1012KB 1012KB >[29224.100263] tw_sock_TCPv6 127KB 127KB >[29224.100276] TCPv6 1386KB 1386KB >[29224.100290] cfq_io_cq 1023KB 1023KB >[29224.100302] cfq_queue 1023KB 1023KB >[29224.100316] mqueue_inode_cache 64KB 64KB >[29224.100327] kioctx 63KB 63KB >[29224.100341] posix_timers_cache 1022KB 1022KB >[29224.100351] ip4-frags 1023KB 1023KB >[29224.100363] secpath_cache 768KB 768KB >[29224.100375] RAW 896KB 896KB >[29224.100387] UDP 1008KB 1008KB >[29224.100399] tw_sock_TCP 127KB 127KB >[29224.100411] request_sock_TCP 191KB 191KB >[29224.100423] TCP 2040KB 2040KB >[29224.100435] hugetlbfs_inode_cache 127KB 127KB >[29224.100448] eventpoll_pwq 1023KB 1023KB >[29224.100460] inotify_inode_mark 1023KB 1023KB >[29224.100472] request_queue 755KB 755KB >[29224.100485] blkdev_ioc 1023KB 1023KB >[29224.100498] biovec-128 256KB 256KB >[29224.100511] biovec-64 128KB 128KB >[29224.100523] skbuff_fclone_cache 1024KB 1024KB >[29224.100536] skbuff_head_cache 1600KB 1600KB >[29224.100548] file_lock_cache 1021KB 1021KB >[29224.100561] net_namespace 510KB 510KB >[29224.100573] shmem_inode_cache 1087KB 1087KB >[29224.100584] dtl 10040KB 11520KB >[29224.100595] taskstats 764KB 764KB >[29224.100607] proc_dir_entry 640KB 640KB >[29224.100619] pde_opener 1023KB 1023KB >[29224.100631] seq_file 1024KB 1024KB >[29224.100643] sigqueue 1022KB 1022KB >[29224.100655] kernfs_node_cache 5302KB 5302KB >[29224.100667] mnt_cache 2996KB 2996KB >[29224.100680] names_cache 4096KB 4096KB >[29224.100693] ebitmap_node 1024KB 1024KB >[29224.100706] avtab_node 1023KB 1023KB >[29224.100718] avc_xperms_data 1024KB 1024KB >[29224.100730] selinux_file_security 1024KB 1024KB >[29224.100742] selinux_inode_security 1855KB 1855KB >[29224.100756] nsproxy 1023KB 1023KB >[29224.100769] vm_area_struct 1660KB 1660KB >[29224.100780] mm_struct 8056KB 8056KB >[29224.100793] files_cache 1211KB 1211KB >[29224.100805] signal_cache 1472KB 1472KB >[29224.100816] sighand_cache 2932KB 2932KB >[29224.100828] task_struct 12210KB 12210KB >[29224.100840] cred_jar 4800KB 4800KB >[29224.100852] thread_stack 17408KB 17408KB >[29224.100865] anon_vma 1086KB 1086KB >[29224.100876] pid 1280KB 1280KB >[29224.100888] numa_policy 383KB 383KB >[29224.100900] trace_event_file 255KB 255KB >[29224.100912] ftrace_event_field 447KB 447KB >[29224.100924] pool_workqueue 1024KB 1024KB >[29224.100936] task_group 1020KB 1020KB >[29224.100948] pgtable-2^11 9216KB 9216KB >[29224.100960] pgtable-2^8 4096KB 4096KB >[29224.100977] dma-kmalloc-512 128KB 128KB >[29224.100994] kmalloc-131072 35328KB 35328KB >[29224.101006] kmalloc-65536 9216KB 9216KB >[29224.101017] kmalloc-32768 8192KB 8192KB >[29224.101029] kmalloc-16384 18432KB 20480KB >[29224.101042] kmalloc-8192 8704KB 8704KB >[29224.101054] kmalloc-4096 8192KB 8192KB >[29224.101066] kmalloc-2048 4352KB 4352KB >[29224.101080] kmalloc-1024 19123KB 19456KB >[29224.101094] kmalloc-512 10176KB 11072KB >[29224.101106] kmalloc-256 1408KB 1408KB >[29224.101118] kmalloc-192 1790KB 1790KB >[29224.101130] kmalloc-128 1536KB 1536KB >[29224.101142] kmalloc-96 1470KB 1470KB >[29224.109951] kmalloc-64 1792KB 1792KB >[29224.109964] kmalloc-32 12992KB 12992KB >[29224.109976] kmalloc-16 1024KB 1024KB >[29224.109989] kmalloc-8 1024KB 1024KB >[29224.110000] kmem_cache_node 1024KB 1024KB >[29224.110011] kmem_cache 3315KB 3315KB >[29224.110021] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29224.110048] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29224.110066] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29224.110084] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29224.110102] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29224.110121] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29224.110138] [ 2069] 81 2069 208 0 51200 67 -900 dbus-daemon >[29224.110156] [ 2089] 0 2089 1288 1 51200 35 0 irqbalance >[29224.110173] [ 2093] 0 2093 7324 0 63488 204 0 NetworkManager >[29224.110192] [ 2094] 0 2094 10692 0 122880 88 0 rsyslogd >[29224.110208] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29224.110225] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29224.110242] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29224.110261] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29224.110278] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29224.110295] [ 2139] 0 2139 97 0 51200 37 0 atd >[29224.110312] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29224.110330] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29224.110346] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29224.110364] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29224.110381] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29224.110398] [ 2337] 0 2337 449 0 53248 78 0 keepalived >[29224.110417] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29224.110435] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29224.110453] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29224.110470] [ 2695] 0 2695 91 0 49152 23 0 dovecot >[29224.110486] [ 2696] 97 2696 84 0 51200 21 0 anvil >[29224.110503] [ 2697] 0 2697 87 1 49152 16 0 log >[29224.110520] [ 2699] 0 2699 103 0 51200 34 0 config >[29224.110536] [ 2711] 0 2711 398 1 51200 110 0 sendmail >[29224.110552] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29224.110568] [ 2723] 0 2723 699 0 53248 309 0 /usr/sbin/httpd >[29224.110586] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29224.110605] [ 2752] 48 2726 37650 0 186368 0 0 /usr/sbin/httpd >[29224.110624] [ 2727] 48 2727 36747 0 178176 573 0 /usr/sbin/httpd >[29224.110643] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29224.110661] [ 3367] 48 3367 36747 0 178176 570 0 /usr/sbin/httpd >[29224.110681] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29224.110698] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29224.110716] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29224.110733] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29224.110750] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29224.110770] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29224.110786] [ 8357] 0 8357 4883 0 63488 288 0 php-fpm >[29224.110802] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29224.110820] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29224.110838] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29224.110855] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29224.110874] [ 8900] 0 8900 410 0 51200 120 0 sendmail >[29224.110891] [ 8907] 0 8907 410 0 51200 126 0 sendmail >[29224.110909] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29224.110925] [ 8909] 0 8909 3410 0 51200 59 0 ssh >[29224.110943] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29224.110960] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29224.110977] [ 8912] 0 8912 411 0 51200 125 0 sendmail >[29224.110995] [ 8913] 0 8913 166 0 40960 46 0 ps >[29224.111012] [ 8914] 0 8914 3249 0 53248 14 0 head >[29224.111030] [ 8918] 990 8918 159 0 51200 40 0 pop3-login >[29224.111048] [ 8919] 0 8919 114 0 51200 42 0 auth >[29224.111064] [ 8920] 0 8920 159 0 51200 42 0 pop3-login >[29224.111080] Out of memory: Kill process 2727 (/usr/sbin/httpd) score 9 or sacrifice child >[29224.120223] Killed process 2727 (/usr/sbin/httpd) total-vm:2351808kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29224.151178] oom_reaper: reaped process 2727 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29228.860670] dbus-daemon invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=-900 >[29228.860700] dbus-daemon cpuset=/ mems_allowed=0 >[29228.860724] CPU: 12 PID: 2069 Comm: dbus-daemon Not tainted 4.18.7-200.fc28.ppc64 #1 >[29228.860736] Call Trace: >[29228.860752] [c000000061fef1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29228.860773] [c000000061fef230] [c0000000002fb670] .dump_header+0x70/0x300 >[29228.860790] [c000000061fef300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29228.860808] [c000000061fef3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29228.860827] [c000000061fef480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29228.860847] [c000000061fef670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29228.860866] [c000000061fef730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29228.860884] [c000000061fef7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29228.860903] [c000000061fef880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29228.860921] [c000000061fef990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29228.860940] [c000000061fefac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29228.860957] [c000000061fefb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29228.860974] [c000000061fefcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29228.860994] [c000000061fefd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29228.861013] [c000000061fefe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29228.861028] Mem-Info: >[29228.861046] active_anon:0 inactive_anon:52 isolated_anon:0 > active_file:0 inactive_file:29 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15788 > mapped:16 shmem:0 pagetables:71 bounce:0 > free:1471 free_pcp:0 free_cma:0 >[29228.861097] Node 0 active_anon:0kB inactive_anon:3328kB active_file:0kB inactive_file:1856kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1024kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29228.861123] Node 0 DMA free:94144kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1664kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:10736kB pagetables:4544kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29228.861164] lowmem_reserve[]: 0 0 0 0 >[29228.869960] Node 0 DMA: 398*64kB (UME) 363*128kB (UME) 106*256kB (UME) 6*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102144kB >[29228.870020] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29228.870037] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29228.870051] 35 total pagecache pages >[29228.870061] 1 pages in swap cache >[29228.870072] Swap cache stats: add 2970329, delete 2970440, find 2520742/4274428 >[29228.870085] Free swap = 1757440kB >[29228.870094] Total swap = 2052032kB >[29228.870104] 32768 pages RAM >[29228.870112] 0 pages HighMem/MovableOnly >[29228.870121] 2684 pages reserved >[29228.870131] 0 pages cma reserved >[29228.870140] 0 pages hwpoisoned >[29228.870149] Unreclaimable slab info: >[29228.870157] Name Used Total >[29228.870377] xfs_efd_item 1017KB 1017KB >[29228.870391] xfs_buf_item 1020KB 1020KB >[29228.870404] xfs_trans 1022KB 1022KB >[29228.870417] xfs_da_state 1020KB 1020KB >[29228.870430] xfs_log_ticket 1023KB 1023KB >[29228.870449] scsi_sense_cache 1024KB 1024KB >[29228.870482] RAWv6 504KB 504KB >[29228.870496] UDPv6 1012KB 1012KB >[29228.870509] tw_sock_TCPv6 127KB 127KB >[29228.870522] TCPv6 1386KB 1386KB >[29228.870537] cfq_io_cq 1023KB 1023KB >[29228.870550] cfq_queue 1023KB 1023KB >[29228.870563] mqueue_inode_cache 64KB 64KB >[29228.870576] kioctx 63KB 63KB >[29228.870592] posix_timers_cache 1022KB 1022KB >[29228.870604] ip4-frags 1023KB 1023KB >[29228.870616] secpath_cache 768KB 768KB >[29228.870629] RAW 896KB 896KB >[29228.870641] UDP 1008KB 1008KB >[29228.870654] tw_sock_TCP 127KB 127KB >[29228.870667] request_sock_TCP 191KB 191KB >[29228.870679] TCP 2040KB 2040KB >[29228.870692] hugetlbfs_inode_cache 127KB 127KB >[29228.870705] eventpoll_pwq 1023KB 1023KB >[29228.870717] inotify_inode_mark 1023KB 1023KB >[29228.870731] request_queue 755KB 755KB >[29228.870744] blkdev_ioc 1023KB 1023KB >[29228.870756] biovec-128 256KB 256KB >[29228.870768] biovec-64 128KB 128KB >[29228.870781] skbuff_fclone_cache 1024KB 1024KB >[29228.870793] skbuff_head_cache 1600KB 1600KB >[29228.870806] file_lock_cache 1021KB 1021KB >[29228.870819] net_namespace 510KB 510KB >[29228.870832] shmem_inode_cache 1087KB 1087KB >[29228.870846] dtl 10040KB 11520KB >[29228.870859] taskstats 764KB 764KB >[29228.870872] proc_dir_entry 640KB 640KB >[29228.870885] pde_opener 1023KB 1023KB >[29228.870899] seq_file 1024KB 1024KB >[29228.870911] sigqueue 1022KB 1022KB >[29228.870923] kernfs_node_cache 5302KB 5302KB >[29228.870934] mnt_cache 2996KB 2996KB >[29228.870946] names_cache 4096KB 4096KB >[29228.870959] ebitmap_node 1024KB 1024KB >[29228.870970] avtab_node 1023KB 1023KB >[29228.870981] avc_xperms_data 1024KB 1024KB >[29228.870993] selinux_file_security 1024KB 1024KB >[29228.871005] selinux_inode_security 1855KB 1855KB >[29228.871019] nsproxy 1023KB 1023KB >[29228.871031] vm_area_struct 1660KB 1660KB >[29228.871042] mm_struct 8056KB 8056KB >[29228.871054] files_cache 1211KB 1211KB >[29228.871066] signal_cache 1472KB 1472KB >[29228.871078] sighand_cache 2932KB 2932KB >[29228.871090] task_struct 12210KB 12210KB >[29228.871101] cred_jar 4800KB 4800KB >[29228.871113] thread_stack 17408KB 17408KB >[29228.871124] anon_vma 1086KB 1086KB >[29228.871135] pid 1280KB 1280KB >[29228.871147] numa_policy 383KB 383KB >[29228.879956] trace_event_file 255KB 255KB >[29228.879969] ftrace_event_field 447KB 447KB >[29228.879981] pool_workqueue 1024KB 1024KB >[29228.879993] task_group 1020KB 1020KB >[29228.880006] pgtable-2^11 9216KB 9216KB >[29228.880018] pgtable-2^8 4096KB 4096KB >[29228.880034] dma-kmalloc-512 128KB 128KB >[29228.880051] kmalloc-131072 35328KB 35328KB >[29228.880064] kmalloc-65536 9216KB 9216KB >[29228.880077] kmalloc-32768 8192KB 8192KB >[29228.880090] kmalloc-16384 18432KB 20480KB >[29228.880102] kmalloc-8192 8704KB 8704KB >[29228.880115] kmalloc-4096 8192KB 8192KB >[29228.880128] kmalloc-2048 4352KB 4352KB >[29228.880142] kmalloc-1024 19123KB 19456KB >[29228.880157] kmalloc-512 10176KB 11072KB >[29228.880169] kmalloc-256 1408KB 1408KB >[29228.880181] kmalloc-192 1790KB 1790KB >[29228.880193] kmalloc-128 1536KB 1536KB >[29228.880205] kmalloc-96 1470KB 1470KB >[29228.880217] kmalloc-64 1792KB 1792KB >[29228.880229] kmalloc-32 12992KB 12992KB >[29228.880241] kmalloc-16 1024KB 1024KB >[29228.880253] kmalloc-8 1024KB 1024KB >[29228.880265] kmem_cache_node 1024KB 1024KB >[29228.880279] kmem_cache 3315KB 3315KB >[29228.880290] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29228.880317] [ 716] 0 716 2668 0 69632 87 0 systemd-journal >[29228.880336] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29228.880355] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29228.880373] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29228.880391] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29228.880409] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29228.880427] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29228.880445] [ 2093] 0 2093 7324 0 63488 194 0 NetworkManager >[29228.880463] [ 2094] 0 2094 10692 0 122880 87 0 rsyslogd >[29228.880480] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29228.880497] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29228.880515] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29228.880533] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29228.880550] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29228.880566] [ 2139] 0 2139 97 0 51200 37 0 atd >[29228.880583] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29228.880600] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29228.880617] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29228.880635] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29228.880653] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29228.880670] [ 2337] 0 2337 449 0 53248 83 0 keepalived >[29228.880688] [ 2338] 0 2338 452 0 53248 89 0 keepalived >[29228.880706] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29228.880724] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29228.880741] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29228.880758] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29228.880776] [ 2697] 0 2697 87 0 49152 17 0 log >[29228.880793] [ 2699] 0 2699 103 0 51200 32 0 config >[29228.880810] [ 2711] 0 2711 398 0 51200 112 0 sendmail >[29228.880827] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29228.880845] [ 2723] 0 2723 699 0 53248 294 0 /usr/sbin/httpd >[29228.880863] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29228.880882] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29228.880900] [ 3367] 48 3367 36747 0 178176 560 0 /usr/sbin/httpd >[29228.880919] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29228.880937] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29228.880954] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29228.880972] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29228.880989] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29228.881006] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29228.881024] [ 8357] 0 8357 4883 0 63488 300 0 php-fpm >[29228.881041] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29228.881059] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29228.881076] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29228.881094] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29228.881113] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29228.881130] [ 8907] 0 8907 410 0 51200 122 0 sendmail >[29228.881148] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29228.881165] [ 8909] 0 8909 3410 0 51200 66 0 ssh >[29228.889959] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29228.889978] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29228.889998] [ 8912] 0 8912 410 0 51200 120 0 sendmail >[29228.890015] [ 8913] 0 8913 166 0 40960 46 0 ps >[29228.890034] [ 8914] 0 8914 3249 0 53248 14 0 head >[29228.890052] [ 8918] 990 8918 159 0 51200 27 0 pop3-login >[29228.890069] [ 8919] 0 8919 142 0 51200 48 0 auth >[29228.890087] [ 8920] 990 8920 159 0 51200 30 0 pop3-login >[29228.890105] [ 8929] 0 8929 699 0 53248 300 0 /usr/sbin/httpd >[29228.890123] [ 8930] 0 8930 699 0 53248 306 0 /usr/sbin/httpd >[29228.890141] [ 8931] 0 8931 699 0 53248 306 0 /usr/sbin/httpd >[29228.890155] Out of memory: Kill process 3367 (/usr/sbin/httpd) score 9 or sacrifice child >[29228.890496] Killed process 3367 (/usr/sbin/httpd) total-vm:2351808kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29228.930184] oom_reaper: reaped process 3367 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29228.960203] systemd-journal invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29228.960218] systemd-journal cpuset=/ mems_allowed=0 >[29228.960237] CPU: 2 PID: 716 Comm: systemd-journal Not tainted 4.18.7-200.fc28.ppc64 #1 >[29228.960253] Call Trace: >[29228.960268] [c000000060273380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29228.960286] [c000000060273410] [c0000000002fb670] .dump_header+0x70/0x300 >[29228.960298] [c0000000602734e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29228.960313] [c0000000602735b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29228.960328] [c000000060273660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29228.960344] [c000000060273850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29228.960358] [c0000000602738f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29228.960374] [c000000060273980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29228.960650] [c000000060273a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29228.960667] [c000000060273b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29228.960683] [c000000060273b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29228.960700] [c000000060273cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29228.960719] [c000000060273d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29228.960735] [c000000060273e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29228.960748] Mem-Info: >[29228.960763] active_anon:0 inactive_anon:52 isolated_anon:0 > active_file:0 inactive_file:29 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15788 > mapped:16 shmem:0 pagetables:71 bounce:0 > free:1494 free_pcp:18 free_cma:0 >[29228.960803] Node 0 active_anon:0kB inactive_anon:3328kB active_file:0kB inactive_file:1856kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1024kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29228.960826] Node 0 DMA free:95616kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:10736kB pagetables:4544kB bounce:0kB free_pcp:1216kB local_pcp:64kB free_cma:0kB >[29228.960862] lowmem_reserve[]: 0 0 0 0 >[29228.960880] Node 0 DMA: 392*64kB (UME) 348*128kB (UME) 100*256kB (UME) 6*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 98304kB >[29228.960919] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29228.960931] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29228.960938] 35 total pagecache pages >[29228.960942] 27 pages in swap cache >[29228.960946] Swap cache stats: add 2970446, delete 2970531, find 2520785/4274545 >[29228.960950] Free swap = 1761472kB >[29228.960953] Total swap = 2052032kB >[29228.960957] 32768 pages RAM >[29228.960960] 0 pages HighMem/MovableOnly >[29228.960965] 2684 pages reserved >[29228.960968] 0 pages cma reserved >[29228.960971] 0 pages hwpoisoned >[29228.960975] Unreclaimable slab info: >[29228.960978] Name Used Total >[29228.961102] xfs_efd_item 1017KB 1017KB >[29228.961107] xfs_buf_item 1020KB 1020KB >[29228.961111] xfs_trans 1022KB 1022KB >[29228.961115] xfs_da_state 1020KB 1020KB >[29228.961119] xfs_log_ticket 1023KB 1023KB >[29228.961128] scsi_sense_cache 1024KB 1024KB >[29228.961143] RAWv6 504KB 504KB >[29228.961149] UDPv6 1012KB 1012KB >[29228.961154] tw_sock_TCPv6 127KB 127KB >[29228.961159] TCPv6 1386KB 1386KB >[29228.961165] cfq_io_cq 1023KB 1023KB >[29228.961172] cfq_queue 1023KB 1023KB >[29228.969961] mqueue_inode_cache 64KB 64KB >[29228.969970] kioctx 63KB 63KB >[29228.969976] posix_timers_cache 1022KB 1022KB >[29228.969980] ip4-frags 1023KB 1023KB >[29228.969985] secpath_cache 768KB 768KB >[29228.969990] RAW 896KB 896KB >[29228.969995] UDP 1008KB 1008KB >[29228.969999] tw_sock_TCP 127KB 127KB >[29228.970002] request_sock_TCP 191KB 191KB >[29228.970006] TCP 2040KB 2040KB >[29228.970010] hugetlbfs_inode_cache 127KB 127KB >[29228.970014] eventpoll_pwq 1023KB 1023KB >[29228.970018] inotify_inode_mark 1023KB 1023KB >[29228.970022] request_queue 755KB 755KB >[29228.970026] blkdev_ioc 1023KB 1023KB >[29228.970031] biovec-128 256KB 256KB >[29228.970035] biovec-64 128KB 128KB >[29228.970039] skbuff_fclone_cache 1024KB 1024KB >[29228.970044] skbuff_head_cache 1600KB 1600KB >[29228.970048] file_lock_cache 1021KB 1021KB >[29228.970053] net_namespace 510KB 510KB >[29228.970057] shmem_inode_cache 1087KB 1087KB >[29228.970062] dtl 10040KB 11520KB >[29228.970067] taskstats 764KB 764KB >[29228.970071] proc_dir_entry 640KB 640KB >[29228.970076] pde_opener 1023KB 1023KB >[29228.970083] seq_file 1024KB 1024KB >[29228.970089] sigqueue 1022KB 1022KB >[29228.970097] kernfs_node_cache 5302KB 5302KB >[29228.970104] mnt_cache 2996KB 2996KB >[29228.970111] names_cache 4096KB 4096KB >[29228.970119] ebitmap_node 1024KB 1024KB >[29228.970127] avtab_node 1023KB 1023KB >[29228.970135] avc_xperms_data 1024KB 1024KB >[29228.970143] selinux_file_security 1024KB 1024KB >[29228.970151] selinux_inode_security 1855KB 1855KB >[29228.970159] nsproxy 1023KB 1023KB >[29228.970167] vm_area_struct 1660KB 1660KB >[29228.970174] mm_struct 8056KB 8056KB >[29228.970180] files_cache 1211KB 1211KB >[29228.970186] signal_cache 1472KB 1472KB >[29228.970192] sighand_cache 2932KB 2932KB >[29228.970198] task_struct 12210KB 12210KB >[29228.970204] cred_jar 4800KB 4800KB >[29228.970211] thread_stack 17408KB 17408KB >[29228.970219] anon_vma 1086KB 1086KB >[29228.970226] pid 1280KB 1280KB >[29228.970233] numa_policy 383KB 383KB >[29228.970241] trace_event_file 255KB 255KB >[29228.970248] ftrace_event_field 447KB 447KB >[29228.970256] pool_workqueue 1024KB 1024KB >[29228.970264] task_group 1020KB 1020KB >[29228.970272] pgtable-2^11 9216KB 9216KB >[29228.970280] pgtable-2^8 4096KB 4096KB >[29228.970291] dma-kmalloc-512 128KB 128KB >[29228.970302] kmalloc-131072 35328KB 35328KB >[29228.970309] kmalloc-65536 9216KB 9216KB >[29228.970316] kmalloc-32768 8192KB 8192KB >[29228.970322] kmalloc-16384 18432KB 20480KB >[29228.970329] kmalloc-8192 8704KB 8704KB >[29228.970335] kmalloc-4096 8192KB 8192KB >[29228.970342] kmalloc-2048 4352KB 4352KB >[29228.970351] kmalloc-1024 19123KB 19456KB >[29228.970359] kmalloc-512 10176KB 11072KB >[29228.970365] kmalloc-256 1408KB 1408KB >[29228.970370] kmalloc-192 1790KB 1790KB >[29228.970375] kmalloc-128 1536KB 1536KB >[29228.970380] kmalloc-96 1470KB 1470KB >[29228.970386] kmalloc-64 1792KB 1792KB >[29228.970390] kmalloc-32 12992KB 12992KB >[29228.970394] kmalloc-16 1024KB 1024KB >[29228.970399] kmalloc-8 1024KB 1024KB >[29228.970403] kmem_cache_node 1024KB 1024KB >[29228.970408] kmem_cache 3315KB 3315KB >[29228.970412] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29228.970432] [ 716] 0 716 2668 0 69632 87 0 systemd-journal >[29228.970439] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29228.970447] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29228.970455] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29228.970462] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29228.970469] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29228.970476] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29228.970483] [ 2093] 0 2093 7324 0 63488 194 0 NetworkManager >[29228.970491] [ 2094] 0 2094 10692 0 122880 87 0 rsyslogd >[29228.970497] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29228.970504] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29228.970511] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29228.970517] [ 2121] 0 2121 307 0 51200 79 -1000 sshd >[29228.970524] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29228.970531] [ 2139] 0 2139 97 0 51200 37 0 atd >[29228.970537] [ 2140] 0 2140 3304 0 51200 51 0 crond >[29228.970544] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29228.970550] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29228.970556] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29228.970563] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29228.970569] [ 2337] 0 2337 449 0 53248 83 0 keepalived >[29228.970576] [ 2338] 0 2338 452 0 53248 89 0 keepalived >[29228.970583] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29228.970590] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29228.970597] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29228.970604] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29228.970611] [ 2697] 0 2697 87 0 49152 17 0 log >[29228.970618] [ 2699] 0 2699 103 0 51200 32 0 config >[29228.970625] [ 2711] 0 2711 398 0 51200 112 0 sendmail >[29228.970631] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29228.970637] [ 2723] 0 2723 699 0 53248 294 0 /usr/sbin/httpd >[29228.970645] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29228.970652] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29228.970659] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29228.970665] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29228.970672] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29228.970679] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29228.970686] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29228.970693] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29228.970701] [ 8357] 0 8357 4883 0 63488 300 0 php-fpm >[29228.970708] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29228.970717] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29228.970723] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29228.970729] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29228.970736] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29228.970742] [ 8907] 0 8907 410 0 51200 122 0 sendmail >[29228.970748] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29228.970754] [ 8909] 0 8909 3410 0 51200 66 0 ssh >[29228.970760] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29228.970767] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29228.970774] [ 8912] 0 8912 410 0 51200 120 0 sendmail >[29228.970780] [ 8913] 0 8913 166 0 40960 46 0 ps >[29228.970786] [ 8914] 0 8914 3249 0 53248 14 0 head >[29228.970793] [ 8918] 990 8918 159 0 51200 27 0 pop3-login >[29228.970800] [ 8919] 0 8919 142 8 51200 27 0 auth >[29228.970807] [ 8920] 990 8920 159 0 51200 30 0 pop3-login >[29228.970814] [ 8929] 0 8929 699 0 53248 300 0 /usr/sbin/httpd >[29228.970821] [ 8930] 0 8930 699 0 53248 306 0 /usr/sbin/httpd >[29228.970828] [ 8931] 0 8931 699 0 53248 306 0 /usr/sbin/httpd >[29228.970835] Out of memory: Kill process 8930 (/usr/sbin/httpd) score 4 or sacrifice child >[29228.970847] Killed process 8930 (/usr/sbin/httpd) total-vm:44736kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29229.900973] systemd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29229.901003] systemd cpuset=/ mems_allowed=0 >[29229.901026] CPU: 4 PID: 1 Comm: systemd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29229.901039] Call Trace: >[29229.901055] [c00000007e9ff380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29229.901077] [c00000007e9ff410] [c0000000002fb670] .dump_header+0x70/0x300 >[29229.901096] [c00000007e9ff4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29229.901116] [c00000007e9ff5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29229.901135] [c00000007e9ff660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29229.901156] [c00000007e9ff850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29229.909941] [c00000007e9ff8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29229.909968] [c00000007e9ff980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29229.910244] [c00000007e9ffa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29229.910264] [c00000007e9ffb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29229.910283] [c00000007e9ffb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29229.910302] [c00000007e9ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29229.910323] [c00000007e9ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29229.910342] [c00000007e9ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29229.910372] Mem-Info: >[29229.910394] active_anon:12 inactive_anon:70 isolated_anon:0 > active_file:37 inactive_file:20 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2803 slab_unreclaimable:15788 > mapped:51 shmem:0 pagetables:67 bounce:0 > free:1469 free_pcp:0 free_cma:0 >[29229.910449] Node 0 active_anon:768kB inactive_anon:4480kB active_file:2368kB inactive_file:1280kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3264kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29229.910480] Node 0 DMA free:94016kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:3008kB active_file:768kB inactive_file:1280kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9696kB pagetables:4288kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29229.910522] lowmem_reserve[]: 0 0 0 0 >[29229.910542] Node 0 DMA: 408*64kB (UME) 361*128kB (UM) 105*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102784kB >[29229.910601] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29229.910617] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29229.910630] 119 total pagecache pages >[29229.910640] 0 pages in swap cache >[29229.910650] Swap cache stats: add 2971229, delete 2971340, find 2521396/4275686 >[29229.910663] Free swap = 1749248kB >[29229.910672] Total swap = 2052032kB >[29229.910681] 32768 pages RAM >[29229.910689] 0 pages HighMem/MovableOnly >[29229.910698] 2684 pages reserved >[29229.910706] 0 pages cma reserved >[29229.910715] 0 pages hwpoisoned >[29229.910725] Unreclaimable slab info: >[29229.910734] Name Used Total >[29229.910945] xfs_efd_item 1017KB 1017KB >[29229.910959] xfs_buf_item 1020KB 1020KB >[29229.910971] xfs_trans 1022KB 1022KB >[29229.910984] xfs_da_state 1020KB 1020KB >[29229.910997] xfs_log_ticket 1023KB 1023KB >[29229.911016] scsi_sense_cache 1024KB 1024KB >[29229.911047] RAWv6 504KB 504KB >[29229.911060] UDPv6 1012KB 1012KB >[29229.911072] tw_sock_TCPv6 127KB 127KB >[29229.911085] TCPv6 1386KB 1386KB >[29229.911099] cfq_io_cq 1023KB 1023KB >[29229.911111] cfq_queue 1023KB 1023KB >[29229.911125] mqueue_inode_cache 64KB 64KB >[29229.911138] kioctx 63KB 63KB >[29229.919950] posix_timers_cache 1022KB 1022KB >[29229.919965] ip4-frags 1023KB 1023KB >[29229.919976] secpath_cache 768KB 768KB >[29229.919987] RAW 896KB 896KB >[29229.920000] UDP 1008KB 1008KB >[29229.920008] tw_sock_TCP 127KB 127KB >[29229.920015] request_sock_TCP 191KB 191KB >[29229.920022] TCP 2040KB 2040KB >[29229.920030] hugetlbfs_inode_cache 127KB 127KB >[29229.920037] eventpoll_pwq 1023KB 1023KB >[29229.920045] inotify_inode_mark 1023KB 1023KB >[29229.920053] request_queue 755KB 755KB >[29229.920060] blkdev_ioc 1023KB 1023KB >[29229.920067] biovec-128 256KB 256KB >[29229.920074] biovec-64 128KB 128KB >[29229.920081] skbuff_fclone_cache 1024KB 1024KB >[29229.920088] skbuff_head_cache 1600KB 1600KB >[29229.920095] file_lock_cache 1021KB 1021KB >[29229.920102] net_namespace 510KB 510KB >[29229.920109] shmem_inode_cache 1087KB 1087KB >[29229.920118] dtl 10040KB 11520KB >[29229.920125] taskstats 764KB 764KB >[29229.920132] proc_dir_entry 640KB 640KB >[29229.920139] pde_opener 1023KB 1023KB >[29229.920147] seq_file 1024KB 1024KB >[29229.920154] sigqueue 1022KB 1022KB >[29229.920162] kernfs_node_cache 5302KB 5302KB >[29229.920169] mnt_cache 2996KB 2996KB >[29229.920177] names_cache 4096KB 4096KB >[29229.920184] ebitmap_node 1024KB 1024KB >[29229.920191] avtab_node 1023KB 1023KB >[29229.920198] avc_xperms_data 1024KB 1024KB >[29229.920205] selinux_file_security 1024KB 1024KB >[29229.920213] selinux_inode_security 1855KB 1855KB >[29229.920221] nsproxy 1023KB 1023KB >[29229.920229] vm_area_struct 1660KB 1660KB >[29229.920236] mm_struct 8056KB 8056KB >[29229.920243] files_cache 1211KB 1211KB >[29229.920250] signal_cache 1472KB 1472KB >[29229.920257] sighand_cache 2932KB 2932KB >[29229.920265] task_struct 12210KB 12210KB >[29229.920272] cred_jar 4800KB 4800KB >[29229.920279] thread_stack 17408KB 17408KB >[29229.920285] anon_vma 1086KB 1086KB >[29229.920292] pid 1280KB 1280KB >[29229.920299] numa_policy 383KB 383KB >[29229.920307] trace_event_file 255KB 255KB >[29229.920313] ftrace_event_field 447KB 447KB >[29229.920321] pool_workqueue 1024KB 1024KB >[29229.920328] task_group 1020KB 1020KB >[29229.920335] pgtable-2^11 9216KB 9216KB >[29229.920342] pgtable-2^8 4096KB 4096KB >[29229.920353] dma-kmalloc-512 128KB 128KB >[29229.920367] kmalloc-131072 35328KB 35328KB >[29229.920377] kmalloc-65536 9216KB 9216KB >[29229.920386] kmalloc-32768 8192KB 8192KB >[29229.920397] kmalloc-16384 18432KB 20480KB >[29229.920406] kmalloc-8192 8704KB 8704KB >[29229.920416] kmalloc-4096 8192KB 8192KB >[29229.920425] kmalloc-2048 4352KB 4352KB >[29229.920436] kmalloc-1024 19123KB 19456KB >[29229.920448] kmalloc-512 10176KB 11072KB >[29229.920458] kmalloc-256 1408KB 1408KB >[29229.920467] kmalloc-192 1790KB 1790KB >[29229.920476] kmalloc-128 1536KB 1536KB >[29229.920485] kmalloc-96 1470KB 1470KB >[29229.920495] kmalloc-64 1792KB 1792KB >[29229.920505] kmalloc-32 12992KB 12992KB >[29229.920514] kmalloc-16 1024KB 1024KB >[29229.920523] kmalloc-8 1024KB 1024KB >[29229.920534] kmem_cache_node 1024KB 1024KB >[29229.920543] kmem_cache 3315KB 3315KB >[29229.920552] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29229.920575] [ 716] 0 716 2668 0 69632 94 0 systemd-journal >[29229.920591] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29229.920607] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29229.920620] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29229.920634] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29229.920648] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29229.920661] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29229.920674] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29229.920688] [ 2094] 0 2094 10692 0 122880 95 0 rsyslogd >[29229.920701] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29229.920714] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29229.920728] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29229.920742] [ 2121] 0 2121 307 2 51200 72 -1000 sshd >[29229.920755] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29229.920769] [ 2139] 0 2139 97 0 51200 37 0 atd >[29229.920782] [ 2140] 0 2140 3304 0 51200 46 0 crond >[29229.920795] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29229.920809] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29229.920823] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29229.920837] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29229.920850] [ 2337] 0 2337 449 1 53248 77 0 keepalived >[29229.920865] [ 2338] 0 2338 452 0 53248 81 0 keepalived >[29229.920880] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29229.920893] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29229.920906] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29229.920919] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29229.920933] [ 2697] 0 2697 87 0 49152 21 0 log >[29229.920946] [ 2699] 0 2699 103 0 51200 32 0 config >[29229.920960] [ 2711] 0 2711 398 0 51200 111 0 sendmail >[29229.920974] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29229.920987] [ 2723] 0 2723 699 0 53248 291 0 /usr/sbin/httpd >[29229.921002] [ 2724] 48 2724 703 2 53248 292 0 /usr/sbin/httpd >[29229.921016] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29229.921030] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29229.921043] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29229.921058] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29229.921072] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29229.921085] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29229.921098] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29229.921111] [ 8357] 0 8357 4883 1 63488 287 0 php-fpm >[29229.921124] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29229.921139] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29229.921152] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29229.921165] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29229.929956] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29229.929975] [ 8907] 0 8907 410 0 51200 129 0 sendmail >[29229.929986] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29229.929995] [ 8909] 0 8909 3410 0 51200 54 0 ssh >[29229.930006] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29229.930016] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29229.930027] [ 8912] 0 8912 410 0 51200 131 0 sendmail >[29229.930037] [ 8913] 0 8913 166 0 40960 46 0 ps >[29229.930047] [ 8914] 0 8914 3249 0 53248 14 0 head >[29229.930058] [ 8918] 990 8918 159 0 51200 31 0 pop3-login >[29229.930068] [ 8919] 0 8919 158 0 51200 33 0 auth >[29229.930079] [ 8920] 990 8920 159 0 51200 40 0 pop3-login >[29229.930090] [ 8929] 0 8929 699 0 53248 291 0 /usr/sbin/httpd >[29229.930100] [ 8931] 0 8931 699 0 53248 292 0 /usr/sbin/httpd >[29229.930111] [ 8932] 0 8932 307 0 51200 85 -1000 sshd >[29229.930120] Out of memory: Kill process 2724 (/usr/sbin/httpd) score 4 or sacrifice child >[29229.930142] Killed process 2724 (/usr/sbin/httpd) total-vm:44992kB, anon-rss:0kB, file-rss:128kB, shmem-rss:0kB >[29229.980513] oom_reaper: reaped process 2724 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29230.020947] crond invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29230.020965] crond cpuset=/ mems_allowed=0 >[29230.020984] CPU: 13 PID: 2140 Comm: crond Not tainted 4.18.7-200.fc28.ppc64 #1 >[29230.020997] Call Trace: >[29230.021010] [c00000005accb380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29230.021030] [c00000005accb410] [c0000000002fb670] .dump_header+0x70/0x300 >[29230.021047] [c00000005accb4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29230.021065] [c00000005accb5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29230.021084] [c00000005accb660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29230.021104] [c00000005accb850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29230.021123] [c00000005accb8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29230.021142] [c00000005accb980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29230.030198] [c00000005accba50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29230.030219] [c00000005accbb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29230.030238] [c00000005accbb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29230.030258] [c00000005accbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29230.030277] [c00000005accbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29230.030297] [c00000005accbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29230.030328] Mem-Info: >[29230.030348] active_anon:12 inactive_anon:70 isolated_anon:0 > active_file:37 inactive_file:20 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2803 slab_unreclaimable:15788 > mapped:51 shmem:0 pagetables:67 bounce:0 > free:1469 free_pcp:0 free_cma:0 >[29230.030400] Node 0 active_anon:768kB inactive_anon:4480kB active_file:2368kB inactive_file:1280kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3264kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29230.030430] Node 0 DMA free:94016kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:3008kB active_file:768kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9696kB pagetables:4288kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29230.030475] lowmem_reserve[]: 0 0 0 0 >[29230.030493] Node 0 DMA: 407*64kB (UME) 361*128kB (UM) 105*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102720kB >[29230.030551] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29230.030567] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29230.030581] 119 total pagecache pages >[29230.030590] 0 pages in swap cache >[29230.030602] Swap cache stats: add 2971230, delete 2971341, find 2521397/4275688 >[29230.030614] Free swap = 1753344kB >[29230.030623] Total swap = 2052032kB >[29230.030632] 32768 pages RAM >[29230.030640] 0 pages HighMem/MovableOnly >[29230.030649] 2684 pages reserved >[29230.030658] 0 pages cma reserved >[29230.030666] 0 pages hwpoisoned >[29230.030675] Unreclaimable slab info: >[29230.030684] Name Used Total >[29230.030797] xfs_efd_item 1017KB 1017KB >[29230.030810] xfs_buf_item 1020KB 1020KB >[29230.030823] xfs_trans 1022KB 1022KB >[29230.030836] xfs_da_state 1020KB 1020KB >[29230.030848] xfs_log_ticket 1023KB 1023KB >[29230.030863] scsi_sense_cache 1024KB 1024KB >[29230.030886] RAWv6 504KB 504KB >[29230.030899] UDPv6 1012KB 1012KB >[29230.030911] tw_sock_TCPv6 127KB 127KB >[29230.030924] TCPv6 1386KB 1386KB >[29230.030937] cfq_io_cq 1023KB 1023KB >[29230.030950] cfq_queue 1023KB 1023KB >[29230.030962] mqueue_inode_cache 64KB 64KB >[29230.030975] kioctx 63KB 63KB >[29230.030989] posix_timers_cache 1022KB 1022KB >[29230.031002] ip4-frags 1023KB 1023KB >[29230.031015] secpath_cache 768KB 768KB >[29230.031027] RAW 896KB 896KB >[29230.031039] UDP 1008KB 1008KB >[29230.031051] tw_sock_TCP 127KB 127KB >[29230.031063] request_sock_TCP 191KB 191KB >[29230.031074] TCP 2040KB 2040KB >[29230.031086] hugetlbfs_inode_cache 127KB 127KB >[29230.031099] eventpoll_pwq 1023KB 1023KB >[29230.031113] inotify_inode_mark 1023KB 1023KB >[29230.031126] request_queue 755KB 755KB >[29230.031140] blkdev_ioc 1023KB 1023KB >[29230.031152] biovec-128 256KB 256KB >[29230.031164] biovec-64 128KB 128KB >[29230.039949] skbuff_fclone_cache 1024KB 1024KB >[29230.039963] skbuff_head_cache 1600KB 1600KB >[29230.039975] file_lock_cache 1021KB 1021KB >[29230.039987] net_namespace 510KB 510KB >[29230.039999] shmem_inode_cache 1087KB 1087KB >[29230.040012] dtl 10040KB 11520KB >[29230.040025] taskstats 764KB 764KB >[29230.040037] proc_dir_entry 640KB 640KB >[29230.040049] pde_opener 1023KB 1023KB >[29230.040061] seq_file 1024KB 1024KB >[29230.040073] sigqueue 1022KB 1022KB >[29230.040085] kernfs_node_cache 5302KB 5302KB >[29230.040097] mnt_cache 2996KB 2996KB >[29230.040109] names_cache 4096KB 4096KB >[29230.040121] ebitmap_node 1024KB 1024KB >[29230.040133] avtab_node 1023KB 1023KB >[29230.040146] avc_xperms_data 1024KB 1024KB >[29230.040158] selinux_file_security 1024KB 1024KB >[29230.040170] selinux_inode_security 1855KB 1855KB >[29230.040183] nsproxy 1023KB 1023KB >[29230.040195] vm_area_struct 1660KB 1660KB >[29230.040207] mm_struct 8056KB 8056KB >[29230.040219] files_cache 1211KB 1211KB >[29230.040231] signal_cache 1472KB 1472KB >[29230.040242] sighand_cache 2932KB 2932KB >[29230.040254] task_struct 12210KB 12210KB >[29230.040267] cred_jar 4800KB 4800KB >[29230.040280] thread_stack 17408KB 17408KB >[29230.040292] anon_vma 1086KB 1086KB >[29230.040304] pid 1280KB 1280KB >[29230.040316] numa_policy 383KB 383KB >[29230.040328] trace_event_file 255KB 255KB >[29230.040340] ftrace_event_field 447KB 447KB >[29230.040353] pool_workqueue 1024KB 1024KB >[29230.040365] task_group 1020KB 1020KB >[29230.040377] pgtable-2^11 9216KB 9216KB >[29230.040389] pgtable-2^8 4096KB 4096KB >[29230.040406] dma-kmalloc-512 128KB 128KB >[29230.040423] kmalloc-131072 35328KB 35328KB >[29230.040435] kmalloc-65536 9216KB 9216KB >[29230.040447] kmalloc-32768 8192KB 8192KB >[29230.040460] kmalloc-16384 18432KB 20480KB >[29230.040472] kmalloc-8192 8704KB 8704KB >[29230.040483] kmalloc-4096 8192KB 8192KB >[29230.040495] kmalloc-2048 4352KB 4352KB >[29230.040508] kmalloc-1024 19123KB 19456KB >[29230.040522] kmalloc-512 10176KB 11072KB >[29230.040534] kmalloc-256 1408KB 1408KB >[29230.040546] kmalloc-192 1790KB 1790KB >[29230.040558] kmalloc-128 1536KB 1536KB >[29230.040570] kmalloc-96 1470KB 1470KB >[29230.040582] kmalloc-64 1792KB 1792KB >[29230.040594] kmalloc-32 12992KB 12992KB >[29230.040606] kmalloc-16 1024KB 1024KB >[29230.040618] kmalloc-8 1024KB 1024KB >[29230.040630] kmem_cache_node 1024KB 1024KB >[29230.040642] kmem_cache 3315KB 3315KB >[29230.040652] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29230.040679] [ 716] 0 716 2668 0 69632 94 0 systemd-journal >[29230.040698] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29230.040717] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29230.040735] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29230.040753] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29230.040771] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29230.040789] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29230.040808] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29230.040827] [ 2094] 0 2094 10692 0 122880 95 0 rsyslogd >[29230.040845] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29230.040862] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29230.040879] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29230.040896] [ 2121] 0 2121 307 2 51200 72 -1000 sshd >[29230.040914] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29230.040931] [ 2139] 0 2139 97 0 51200 37 0 atd >[29230.040948] [ 2140] 0 2140 3304 0 51200 46 0 crond >[29230.040965] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29230.040983] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29230.041000] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29230.041018] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29230.041035] [ 2337] 0 2337 449 1 53248 77 0 keepalived >[29230.041053] [ 2338] 0 2338 452 0 53248 81 0 keepalived >[29230.041071] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29230.041090] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29230.041108] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29230.041126] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29230.041143] [ 2697] 0 2697 87 0 49152 21 0 log >[29230.049960] [ 2699] 0 2699 103 0 51200 32 0 config >[29230.049979] [ 2711] 0 2711 398 0 51200 111 0 sendmail >[29230.049998] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29230.050015] [ 2723] 0 2723 699 0 53248 291 0 /usr/sbin/httpd >[29230.050034] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29230.050052] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29230.050069] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29230.050087] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29230.050104] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29230.050122] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29230.050140] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29230.050156] [ 8357] 0 8357 4883 1 63488 287 0 php-fpm >[29230.050173] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29230.050192] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29230.050210] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29230.050227] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29230.050245] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29230.050262] [ 8907] 0 8907 410 0 51200 129 0 sendmail >[29230.050280] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29230.050297] [ 8909] 0 8909 3410 0 51200 54 0 ssh >[29230.050316] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29230.050333] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29230.050352] [ 8912] 0 8912 410 0 51200 131 0 sendmail >[29230.050369] [ 8913] 0 8913 166 0 40960 46 0 ps >[29230.050386] [ 8914] 0 8914 3249 0 53248 14 0 head >[29230.050404] [ 8918] 990 8918 159 0 51200 31 0 pop3-login >[29230.050423] [ 8919] 0 8919 158 0 51200 33 0 auth >[29230.050441] [ 8920] 990 8920 159 0 51200 40 0 pop3-login >[29230.050459] [ 8929] 0 8929 699 0 53248 291 0 /usr/sbin/httpd >[29230.050477] [ 8931] 0 8931 699 0 53248 292 0 /usr/sbin/httpd >[29230.050495] [ 8932] 0 8932 307 0 51200 85 -1000 sshd >[29230.050510] Out of memory: Kill process 8931 (/usr/sbin/httpd) score 4 or sacrifice child >[29230.050530] Killed process 8931 (/usr/sbin/httpd) total-vm:44736kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29230.070525] oom_reaper: reaped process 8931 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29231.370744] ssh invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29231.370757] ssh cpuset=/ mems_allowed=0 >[29231.370765] CPU: 15 PID: 8909 Comm: ssh Not tainted 4.18.7-200.fc28.ppc64 #1 >[29231.370768] Call Trace: >[29231.370774] [c000000009f371a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29231.370781] [c000000009f37230] [c0000000002fb670] .dump_header+0x70/0x300 >[29231.370785] [c000000009f37300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29231.370790] [c000000009f373d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29231.370795] [c000000009f37480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29231.370801] [c000000009f37670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29231.370806] [c000000009f37730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29231.370811] [c000000009f377f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29231.370816] [c000000009f37880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29231.370821] [c000000009f37990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29231.370826] [c000000009f37ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29231.370831] [c000000009f37b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29231.370835] [c000000009f37cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29231.370841] [c000000009f37d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29231.370846] [c000000009f37e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29231.370850] Mem-Info: >[29231.370856] active_anon:0 inactive_anon:0 isolated_anon:0 > active_file:0 inactive_file:44 isolated_file:3 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2807 slab_unreclaimable:15791 > mapped:45 shmem:0 pagetables:67 bounce:0 > free:1493 free_pcp:1 free_cma:0 >[29231.370870] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:2816kB unevictable:0kB isolated(anon):0kB isolated(file):192kB mapped:2880kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29231.370878] Node 0 DMA free:95552kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:1344kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9712kB pagetables:4288kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29231.370889] lowmem_reserve[]: 0 0 0 0 >[29231.370894] Node 0 DMA: 403*64kB (UME) 360*128kB (UM) 104*256kB (UM) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102080kB >[29231.370910] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29231.370914] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29231.370918] 93 total pagecache pages >[29231.370920] 0 pages in swap cache >[29231.370923] Swap cache stats: add 2972389, delete 2972499, find 2522316/4277428 >[29231.370926] Free swap = 1753344kB >[29231.370928] Total swap = 2052032kB >[29231.370930] 32768 pages RAM >[29231.370932] 0 pages HighMem/MovableOnly >[29231.370935] 2684 pages reserved >[29231.370937] 0 pages cma reserved >[29231.370939] 0 pages hwpoisoned >[29231.370942] Unreclaimable slab info: >[29231.370944] Name Used Total >[29231.371122] xfs_efd_item 1017KB 1017KB >[29231.371126] xfs_buf_item 1020KB 1020KB >[29231.371129] xfs_trans 1022KB 1022KB >[29231.371132] xfs_da_state 1020KB 1020KB >[29231.371135] xfs_log_ticket 1023KB 1023KB >[29231.371144] scsi_sense_cache 1024KB 1024KB >[29231.379957] RAWv6 504KB 504KB >[29231.379964] UDPv6 1012KB 1012KB >[29231.379968] tw_sock_TCPv6 127KB 127KB >[29231.379971] TCPv6 1386KB 1386KB >[29231.379975] cfq_io_cq 1023KB 1023KB >[29231.379978] cfq_queue 1023KB 1023KB >[29231.379981] mqueue_inode_cache 64KB 64KB >[29231.379985] kioctx 63KB 63KB >[29231.379989] posix_timers_cache 1022KB 1022KB >[29231.379992] ip4-frags 1023KB 1023KB >[29231.379995] secpath_cache 768KB 768KB >[29231.379998] RAW 896KB 896KB >[29231.380001] UDP 1008KB 1008KB >[29231.380004] tw_sock_TCP 127KB 127KB >[29231.380007] request_sock_TCP 191KB 191KB >[29231.380010] TCP 2040KB 2040KB >[29231.380013] hugetlbfs_inode_cache 127KB 127KB >[29231.380016] eventpoll_pwq 1023KB 1023KB >[29231.380020] inotify_inode_mark 1023KB 1023KB >[29231.380023] request_queue 755KB 755KB >[29231.380026] blkdev_ioc 1023KB 1023KB >[29231.380029] biovec-128 256KB 256KB >[29231.380032] biovec-64 128KB 128KB >[29231.380035] skbuff_fclone_cache 1024KB 1024KB >[29231.380038] skbuff_head_cache 1600KB 1600KB >[29231.380041] file_lock_cache 1021KB 1021KB >[29231.380044] net_namespace 510KB 510KB >[29231.380047] shmem_inode_cache 1087KB 1087KB >[29231.380051] dtl 10040KB 11520KB >[29231.380054] taskstats 764KB 764KB >[29231.380057] proc_dir_entry 640KB 640KB >[29231.380060] pde_opener 1023KB 1023KB >[29231.380064] seq_file 1024KB 1024KB >[29231.380067] sigqueue 1022KB 1022KB >[29231.380070] kernfs_node_cache 5302KB 5302KB >[29231.380073] mnt_cache 2996KB 2996KB >[29231.380076] names_cache 4096KB 4096KB >[29231.380079] ebitmap_node 1024KB 1024KB >[29231.380082] avtab_node 1023KB 1023KB >[29231.380085] avc_xperms_data 1024KB 1024KB >[29231.380088] selinux_file_security 1024KB 1024KB >[29231.380091] selinux_inode_security 1855KB 1855KB >[29231.380095] nsproxy 1023KB 1023KB >[29231.380098] vm_area_struct 1660KB 1660KB >[29231.380101] mm_struct 8056KB 8056KB >[29231.380104] files_cache 1211KB 1211KB >[29231.380107] signal_cache 1472KB 1472KB >[29231.380110] sighand_cache 2932KB 2932KB >[29231.380113] task_struct 12210KB 12210KB >[29231.380115] cred_jar 4800KB 4800KB >[29231.380118] thread_stack 17408KB 17408KB >[29231.380121] anon_vma 1086KB 1086KB >[29231.380124] pid 1280KB 1280KB >[29231.380127] numa_policy 383KB 383KB >[29231.380130] trace_event_file 255KB 255KB >[29231.380133] ftrace_event_field 447KB 447KB >[29231.380137] pool_workqueue 1024KB 1024KB >[29231.380140] task_group 1020KB 1020KB >[29231.380143] pgtable-2^11 9216KB 9216KB >[29231.380146] pgtable-2^8 4096KB 4096KB >[29231.380149] dma-kmalloc-512 128KB 128KB >[29231.380153] kmalloc-131072 35328KB 35328KB >[29231.380156] kmalloc-65536 9216KB 9216KB >[29231.380159] kmalloc-32768 8192KB 8192KB >[29231.380163] kmalloc-16384 18432KB 20480KB >[29231.380166] kmalloc-8192 8704KB 8704KB >[29231.380168] kmalloc-4096 8192KB 8192KB >[29231.380171] kmalloc-2048 4352KB 4352KB >[29231.380176] kmalloc-1024 19123KB 19456KB >[29231.380180] kmalloc-512 10176KB 11072KB >[29231.380183] kmalloc-256 1408KB 1408KB >[29231.380186] kmalloc-192 1790KB 1790KB >[29231.380189] kmalloc-128 1536KB 1536KB >[29231.380192] kmalloc-96 1470KB 1470KB >[29231.380195] kmalloc-64 1792KB 1792KB >[29231.380198] kmalloc-32 12992KB 12992KB >[29231.380201] kmalloc-16 1024KB 1024KB >[29231.380203] kmalloc-8 1024KB 1024KB >[29231.380207] kmem_cache_node 1024KB 1024KB >[29231.380210] kmem_cache 3315KB 3315KB >[29231.380212] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29231.380226] [ 716] 0 716 2668 0 69632 93 0 systemd-journal >[29231.380231] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29231.380237] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29231.380242] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29231.380246] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29231.380251] [ 2069] 81 2069 208 0 51200 59 -900 dbus-daemon >[29231.380256] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29231.380261] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29231.380265] [ 2094] 0 2094 10692 0 122880 87 0 rsyslogd >[29231.380270] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29231.380275] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29231.380279] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29231.380284] [ 2121] 0 2121 307 0 51200 81 -1000 sshd >[29231.380288] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29231.380293] [ 2139] 0 2139 97 0 51200 37 0 atd >[29231.380298] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29231.380302] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29231.380307] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29231.380312] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29231.380316] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29231.380321] [ 2337] 0 2337 449 0 53248 93 0 keepalived >[29231.380326] [ 2338] 0 2338 452 0 53248 89 0 keepalived >[29231.380331] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29231.380336] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29231.380340] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29231.380345] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29231.380349] [ 2697] 0 2697 87 0 49152 28 0 log >[29231.380354] [ 2699] 0 2699 103 0 51200 32 0 config >[29231.380358] [ 2711] 0 2711 398 0 51200 126 0 sendmail >[29231.380363] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29231.380367] [ 2723] 0 2723 699 0 53248 307 0 /usr/sbin/httpd >[29231.380372] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29231.380377] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29231.380382] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29231.380386] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29231.380391] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29231.380395] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29231.380400] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29231.380405] [ 8357] 0 8357 4883 0 63488 295 0 php-fpm >[29231.380410] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29231.380415] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29231.380420] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29231.380425] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29231.380429] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29231.380434] [ 8907] 0 8907 410 0 51200 131 0 sendmail >[29231.380438] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29231.380443] [ 8909] 0 8909 3410 2 51200 48 0 ssh >[29231.380447] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29231.380452] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29231.380456] [ 8912] 0 8912 410 0 51200 130 0 sendmail >[29231.380461] [ 8913] 0 8913 166 1 40960 36 0 ps >[29231.380465] [ 8914] 0 8914 3249 0 53248 14 0 head >[29231.380470] [ 8918] 990 8918 159 0 51200 39 0 pop3-login >[29231.380474] [ 8919] 0 8919 158 1 51200 32 0 auth >[29231.380479] [ 8920] 990 8920 159 0 51200 44 0 pop3-login >[29231.380484] [ 8929] 0 8929 699 0 53248 302 0 /usr/sbin/httpd >[29231.380489] [ 8932] 0 8932 307 0 51200 88 0 sshd >[29231.380493] [ 8933] 0 8933 3304 0 51200 38 0 crond >[29231.380497] Out of memory: Kill process 2723 (/usr/sbin/httpd) score 4 or sacrifice child >[29231.380504] Killed process 8929 (/usr/sbin/httpd) total-vm:44736kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29231.400095] oom_reaper: reaped process 8929 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29231.960043] php-fpm invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29231.960073] php-fpm cpuset=/ mems_allowed=0 >[29231.960096] CPU: 7 PID: 8357 Comm: php-fpm Not tainted 4.18.7-200.fc28.ppc64 #1 >[29231.960110] Call Trace: >[29231.960127] [c00000000cd771a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29231.960149] [c00000000cd77230] [c0000000002fb670] .dump_header+0x70/0x300 >[29231.960167] [c00000000cd77300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29231.960185] [c00000000cd773d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29231.960205] [c00000000cd77480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29231.960225] [c00000000cd77670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29231.960245] [c00000000cd77730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29231.960265] [c00000000cd777f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29231.960285] [c00000000cd77880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29231.960305] [c00000000cd77990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29231.960325] [c00000000cd77ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29231.960343] [c00000000cd77b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29231.960362] [c00000000cd77cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29231.960383] [c00000000cd77d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29231.960404] [c00000000cd77e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29231.960419] Mem-Info: >[29231.960440] active_anon:24 inactive_anon:97 isolated_anon:1 > active_file:25 inactive_file:50 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2807 slab_unreclaimable:15799 > mapped:56 shmem:1 pagetables:68 bounce:0 > free:1467 free_pcp:0 free_cma:0 >[29231.960493] Node 0 active_anon:1536kB inactive_anon:6208kB active_file:1600kB inactive_file:3200kB unevictable:0kB isolated(anon):64kB isolated(file):0kB mapped:3584kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29231.960523] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1728kB active_file:1536kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9712kB pagetables:4352kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29231.960565] lowmem_reserve[]: 0 0 0 0 >[29231.960583] Node 0 DMA: 398*64kB (UME) 361*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 101632kB >[29231.960642] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29231.960659] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29231.960673] 129 total pagecache pages >[29231.960682] 0 pages in swap cache >[29231.960693] Swap cache stats: add 2972690, delete 2972800, find 2522540/4277860 >[29231.960705] Free swap = 1753344kB >[29231.960714] Total swap = 2052032kB >[29231.960724] 32768 pages RAM >[29231.960732] 0 pages HighMem/MovableOnly >[29231.960741] 2684 pages reserved >[29231.960750] 0 pages cma reserved >[29231.960758] 0 pages hwpoisoned >[29231.960767] Unreclaimable slab info: >[29231.960776] Name Used Total >[29231.960979] xfs_efd_item 1017KB 1017KB >[29231.960992] xfs_buf_item 1020KB 1020KB >[29231.961005] xfs_trans 1022KB 1022KB >[29231.961018] xfs_da_state 1020KB 1020KB >[29231.961030] xfs_log_ticket 1023KB 1023KB >[29231.961048] scsi_sense_cache 1024KB 1024KB >[29231.961078] RAWv6 504KB 504KB >[29231.961091] UDPv6 1012KB 1012KB >[29231.961103] tw_sock_TCPv6 127KB 127KB >[29231.961117] TCPv6 1386KB 1386KB >[29231.961131] cfq_io_cq 1023KB 1023KB >[29231.961144] cfq_queue 1023KB 1023KB >[29231.961156] mqueue_inode_cache 64KB 64KB >[29231.961169] kioctx 63KB 63KB >[29231.969963] posix_timers_cache 1022KB 1022KB >[29231.969977] ip4-frags 1023KB 1023KB >[29231.969990] secpath_cache 768KB 768KB >[29231.970003] RAW 896KB 896KB >[29231.970015] UDP 1008KB 1008KB >[29231.970027] tw_sock_TCP 127KB 127KB >[29231.970039] request_sock_TCP 191KB 191KB >[29231.970052] TCP 2040KB 2040KB >[29231.970064] hugetlbfs_inode_cache 127KB 127KB >[29231.970077] eventpoll_pwq 1023KB 1023KB >[29231.970089] inotify_inode_mark 1023KB 1023KB >[29231.970101] request_queue 755KB 755KB >[29231.970115] blkdev_ioc 1023KB 1023KB >[29231.970127] biovec-128 256KB 256KB >[29231.970140] biovec-64 128KB 128KB >[29231.970153] skbuff_fclone_cache 1024KB 1024KB >[29231.970165] skbuff_head_cache 1600KB 1600KB >[29231.970177] file_lock_cache 1021KB 1021KB >[29231.970189] net_namespace 510KB 510KB >[29231.970201] shmem_inode_cache 1087KB 1087KB >[29231.970214] dtl 10040KB 11520KB >[29231.970226] taskstats 764KB 764KB >[29231.970238] proc_dir_entry 640KB 640KB >[29231.970249] pde_opener 1023KB 1023KB >[29231.970261] seq_file 1024KB 1024KB >[29231.970274] sigqueue 1022KB 1022KB >[29231.970286] kernfs_node_cache 5302KB 5302KB >[29231.970298] mnt_cache 2996KB 2996KB >[29231.970309] names_cache 4096KB 4096KB >[29231.970321] ebitmap_node 1024KB 1024KB >[29231.970333] avtab_node 1023KB 1023KB >[29231.970345] avc_xperms_data 1024KB 1024KB >[29231.970355] selinux_file_security 1024KB 1024KB >[29231.970367] selinux_inode_security 1855KB 1855KB >[29231.970380] nsproxy 1023KB 1023KB >[29231.970391] vm_area_struct 1660KB 1660KB >[29231.970402] mm_struct 8056KB 8056KB >[29231.970413] files_cache 1211KB 1211KB >[29231.970424] signal_cache 1472KB 1472KB >[29231.970436] sighand_cache 2932KB 2932KB >[29231.970448] task_struct 12210KB 12210KB >[29231.970460] cred_jar 4800KB 4800KB >[29231.970472] thread_stack 17408KB 17408KB >[29231.970484] anon_vma 1086KB 1086KB >[29231.970496] pid 1280KB 1280KB >[29231.970508] numa_policy 383KB 383KB >[29231.970520] trace_event_file 255KB 255KB >[29231.970533] ftrace_event_field 447KB 447KB >[29231.970544] pool_workqueue 1024KB 1024KB >[29231.970556] task_group 1020KB 1020KB >[29231.970567] pgtable-2^11 9216KB 9216KB >[29231.970578] pgtable-2^8 4096KB 4096KB >[29231.970594] dma-kmalloc-512 128KB 128KB >[29231.970610] kmalloc-131072 35328KB 35328KB >[29231.970622] kmalloc-65536 9216KB 9216KB >[29231.970633] kmalloc-32768 8192KB 8192KB >[29231.970646] kmalloc-16384 18432KB 20480KB >[29231.970658] kmalloc-8192 8704KB 8704KB >[29231.970670] kmalloc-4096 8192KB 8192KB >[29231.970681] kmalloc-2048 4352KB 4352KB >[29231.970693] kmalloc-1024 19331KB 19456KB >[29231.970707] kmalloc-512 10176KB 11072KB >[29231.970718] kmalloc-256 1408KB 1408KB >[29231.970730] kmalloc-192 1790KB 1790KB >[29231.970742] kmalloc-128 1536KB 1536KB >[29231.970754] kmalloc-96 1470KB 1470KB >[29231.970766] kmalloc-64 1792KB 1792KB >[29231.970777] kmalloc-32 12992KB 12992KB >[29231.970789] kmalloc-16 1024KB 1024KB >[29231.970800] kmalloc-8 1024KB 1024KB >[29231.970812] kmem_cache_node 1024KB 1024KB >[29231.970823] kmem_cache 3315KB 3315KB >[29231.970832] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29231.970860] [ 716] 0 716 2668 0 69632 83 0 systemd-journal >[29231.970878] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29231.970897] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29231.970914] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29231.970931] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29231.970948] [ 2069] 81 2069 208 0 51200 63 -900 dbus-daemon >[29231.970965] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29231.970983] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29231.971001] [ 2094] 0 2094 10692 0 122880 93 0 rsyslogd >[29231.971018] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29231.971035] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29231.971051] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29231.971067] [ 2121] 0 2121 307 0 51200 82 -1000 sshd >[29231.971085] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29231.971102] [ 2139] 0 2139 97 0 51200 37 0 atd >[29231.971117] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29231.971133] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29231.979937] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29231.979969] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29231.979987] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29231.980007] [ 2337] 0 2337 449 0 53248 82 0 keepalived >[29231.980025] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29231.980042] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29231.980061] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29231.980078] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29231.980095] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29231.980112] [ 2697] 0 2697 87 0 49152 28 0 log >[29231.980129] [ 2699] 0 2699 103 0 51200 32 0 config >[29231.980146] [ 2711] 0 2711 398 0 51200 117 0 sendmail >[29231.980164] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29231.980181] [ 2723] 0 2723 699 0 53248 311 0 /usr/sbin/httpd >[29231.980201] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29231.980218] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29231.980235] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29231.980253] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29231.980271] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29231.980290] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29231.980307] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29231.980325] [ 8357] 0 8357 4883 1 63488 285 0 php-fpm >[29231.980342] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29231.980362] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29231.980378] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29231.980395] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29231.980411] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29231.980428] [ 8907] 0 8907 410 0 51200 118 0 sendmail >[29231.980445] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29231.980461] [ 8909] 0 8909 3410 0 51200 61 0 ssh >[29231.980477] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29231.980493] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29231.980509] [ 8912] 0 8912 410 0 51200 123 0 sendmail >[29231.980525] [ 8913] 0 8913 166 0 40960 43 0 ps >[29231.980541] [ 8914] 0 8914 3249 0 53248 14 0 head >[29231.980557] [ 8918] 990 8918 159 0 51200 26 0 pop3-login >[29231.980574] [ 8919] 0 8919 168 0 51200 38 0 auth >[29231.980590] [ 8920] 990 8920 159 0 51200 30 0 pop3-login >[29231.980608] [ 8932] 0 8932 38 0 38912 9 0 sshd >[29231.980625] [ 8933] 0 8933 3304 0 51200 46 0 crond >[29231.980638] Out of memory: Kill process 2723 (/usr/sbin/httpd) score 5 or sacrifice child >[29231.980661] Killed process 2723 (/usr/sbin/httpd) total-vm:44736kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29232.000769] oom_reaper: reaped process 2723 (/usr/sbin/httpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29232.019958] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=-1000 >[29232.019974] sshd cpuset=/ mems_allowed=0 >[29232.019993] CPU: 14 PID: 2121 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29232.020005] Call Trace: >[29232.020018] [c000000064d731a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29232.020039] [c000000064d73230] [c0000000002fb670] .dump_header+0x70/0x300 >[29232.020056] [c000000064d73300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29232.020074] [c000000064d733d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29232.020094] [c000000064d73480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29232.020114] [c000000064d73670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29232.020133] [c000000064d73730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29232.020153] [c000000064d737f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29232.020172] [c000000064d73880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29232.020191] [c000000064d73990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29232.020209] [c000000064d73ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29232.020227] [c000000064d73b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29232.020246] [c000000064d73cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29232.020266] [c000000064d73d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29232.020286] [c000000064d73e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29232.020300] Mem-Info: >[29232.020319] active_anon:24 inactive_anon:97 isolated_anon:1 > active_file:25 inactive_file:50 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2807 slab_unreclaimable:15799 > mapped:56 shmem:1 pagetables:68 bounce:0 > free:1467 free_pcp:2 free_cma:0 >[29232.020372] Node 0 active_anon:1536kB inactive_anon:6208kB active_file:1600kB inactive_file:3200kB unevictable:0kB isolated(anon):64kB isolated(file):0kB mapped:3584kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29232.020402] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1728kB active_file:1536kB inactive_file:960kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9712kB pagetables:4352kB bounce:0kB free_pcp:128kB local_pcp:0kB free_cma:0kB >[29232.020443] lowmem_reserve[]: 0 0 0 0 >[29232.020462] Node 0 DMA: 395*64kB (UME) 361*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 101440kB >[29232.020518] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29232.020535] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29232.020549] 129 total pagecache pages >[29232.020558] 0 pages in swap cache >[29232.020568] Swap cache stats: add 2972692, delete 2972802, find 2522541/4277862 >[29232.020581] Free swap = 1769728kB >[29232.020590] Total swap = 2052032kB >[29232.020599] 32768 pages RAM >[29232.020607] 0 pages HighMem/MovableOnly >[29232.020615] 2684 pages reserved >[29232.020624] 0 pages cma reserved >[29232.020633] 0 pages hwpoisoned >[29232.020642] Unreclaimable slab info: >[29232.020652] Name Used Total >[29232.020704] xfs_efd_item 1017KB 1017KB >[29232.020717] xfs_buf_item 1020KB 1020KB >[29232.020729] xfs_trans 1022KB 1022KB >[29232.020741] xfs_da_state 1020KB 1020KB >[29232.020752] xfs_log_ticket 1023KB 1023KB >[29232.020766] scsi_sense_cache 1024KB 1024KB >[29232.020784] RAWv6 504KB 504KB >[29232.020795] UDPv6 1012KB 1012KB >[29232.020807] tw_sock_TCPv6 127KB 127KB >[29232.020820] TCPv6 1386KB 1386KB >[29232.020833] cfq_io_cq 1023KB 1023KB >[29232.020845] cfq_queue 1023KB 1023KB >[29232.020858] mqueue_inode_cache 64KB 64KB >[29232.020871] kioctx 63KB 63KB >[29232.020886] posix_timers_cache 1022KB 1022KB >[29232.020898] ip4-frags 1023KB 1023KB >[29232.020910] secpath_cache 768KB 768KB >[29232.020923] RAW 896KB 896KB >[29232.020935] UDP 1008KB 1008KB >[29232.020948] tw_sock_TCP 127KB 127KB >[29232.020960] request_sock_TCP 191KB 191KB >[29232.020973] TCP 2040KB 2040KB >[29232.020986] hugetlbfs_inode_cache 127KB 127KB >[29232.020999] eventpoll_pwq 1023KB 1023KB >[29232.021012] inotify_inode_mark 1023KB 1023KB >[29232.021024] request_queue 755KB 755KB >[29232.021039] blkdev_ioc 1023KB 1023KB >[29232.021051] biovec-128 256KB 256KB >[29232.021063] biovec-64 128KB 128KB >[29232.021075] skbuff_fclone_cache 1024KB 1024KB >[29232.021087] skbuff_head_cache 1600KB 1600KB >[29232.021099] file_lock_cache 1021KB 1021KB >[29232.021111] net_namespace 510KB 510KB >[29232.021124] shmem_inode_cache 1087KB 1087KB >[29232.021137] dtl 10040KB 11520KB >[29232.021149] taskstats 764KB 764KB >[29232.021161] proc_dir_entry 640KB 640KB >[29232.021172] pde_opener 1023KB 1023KB >[29232.029961] seq_file 1024KB 1024KB >[29232.029974] sigqueue 1022KB 1022KB >[29232.029985] kernfs_node_cache 5302KB 5302KB >[29232.029998] mnt_cache 2996KB 2996KB >[29232.030010] names_cache 4096KB 4096KB >[29232.030023] ebitmap_node 1024KB 1024KB >[29232.030035] avtab_node 1023KB 1023KB >[29232.030047] avc_xperms_data 1024KB 1024KB >[29232.030060] selinux_file_security 1024KB 1024KB >[29232.030072] selinux_inode_security 1855KB 1855KB >[29232.030086] nsproxy 1023KB 1023KB >[29232.030099] vm_area_struct 1660KB 1660KB >[29232.030110] mm_struct 8056KB 8056KB >[29232.030122] files_cache 1211KB 1211KB >[29232.030134] signal_cache 1472KB 1472KB >[29232.030146] sighand_cache 2932KB 2932KB >[29232.030159] task_struct 12210KB 12210KB >[29232.030171] cred_jar 4800KB 4800KB >[29232.030183] thread_stack 17408KB 17408KB >[29232.030196] anon_vma 1086KB 1086KB >[29232.030209] pid 1280KB 1280KB >[29232.030222] numa_policy 383KB 383KB >[29232.030235] trace_event_file 255KB 255KB >[29232.030248] ftrace_event_field 447KB 447KB >[29232.030260] pool_workqueue 1024KB 1024KB >[29232.030273] task_group 1020KB 1020KB >[29232.030285] pgtable-2^11 9216KB 9216KB >[29232.030299] pgtable-2^8 4096KB 4096KB >[29232.030315] dma-kmalloc-512 128KB 128KB >[29232.030330] kmalloc-131072 35328KB 35328KB >[29232.030341] kmalloc-65536 9216KB 9216KB >[29232.030353] kmalloc-32768 8192KB 8192KB >[29232.030365] kmalloc-16384 18432KB 20480KB >[29232.030377] kmalloc-8192 8704KB 8704KB >[29232.030388] kmalloc-4096 8192KB 8192KB >[29232.030400] kmalloc-2048 4352KB 4352KB >[29232.030412] kmalloc-1024 19331KB 19456KB >[29232.030425] kmalloc-512 10176KB 11072KB >[29232.030436] kmalloc-256 1408KB 1408KB >[29232.030448] kmalloc-192 1790KB 1790KB >[29232.030461] kmalloc-128 1536KB 1536KB >[29232.030473] kmalloc-96 1470KB 1470KB >[29232.030485] kmalloc-64 1792KB 1792KB >[29232.030497] kmalloc-32 12992KB 12992KB >[29232.030510] kmalloc-16 1024KB 1024KB >[29232.030522] kmalloc-8 1024KB 1024KB >[29232.030535] kmem_cache_node 1024KB 1024KB >[29232.030547] kmem_cache 3315KB 3315KB >[29232.030558] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29232.030585] [ 716] 0 716 2668 0 69632 83 0 systemd-journal >[29232.030603] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29232.030622] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29232.030639] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29232.030658] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29232.030675] [ 2069] 81 2069 208 0 51200 63 -900 dbus-daemon >[29232.030693] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29232.030711] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29232.030730] [ 2094] 0 2094 10692 0 122880 93 0 rsyslogd >[29232.030746] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29232.030763] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29232.030780] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29232.030797] [ 2121] 0 2121 307 0 51200 82 -1000 sshd >[29232.030814] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29232.030831] [ 2139] 0 2139 97 0 51200 37 0 atd >[29232.030848] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29232.030865] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29232.030882] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29232.030899] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29232.030917] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29232.030935] [ 2337] 0 2337 449 0 53248 82 0 keepalived >[29232.030952] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29232.030969] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29232.030987] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29232.031004] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29232.031022] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29232.031040] [ 2697] 0 2697 87 0 49152 28 0 log >[29232.031056] [ 2699] 0 2699 103 0 51200 32 0 config >[29232.031073] [ 2711] 0 2711 398 0 51200 117 0 sendmail >[29232.031091] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29232.031109] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29232.031126] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29232.031145] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29232.039968] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29232.039987] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29232.040005] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29232.040022] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29232.040040] [ 8357] 0 8357 4883 1 63488 285 0 php-fpm >[29232.040058] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29232.040079] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29232.040097] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29232.040114] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29232.040133] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29232.040150] [ 8907] 0 8907 410 0 51200 118 0 sendmail >[29232.040167] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29232.040184] [ 8909] 0 8909 3410 0 51200 61 0 ssh >[29232.040202] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29232.040219] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29232.040236] [ 8912] 0 8912 410 0 51200 123 0 sendmail >[29232.040253] [ 8913] 0 8913 166 0 40960 43 0 ps >[29232.040269] [ 8914] 0 8914 3249 0 53248 14 0 head >[29232.040287] [ 8918] 990 8918 159 0 51200 26 0 pop3-login >[29232.040305] [ 8919] 0 8919 168 0 51200 38 0 auth >[29232.040323] [ 8920] 990 8920 159 0 51200 30 0 pop3-login >[29232.040342] [ 8932] 0 8932 38 0 38912 9 0 sshd >[29232.040359] [ 8933] 0 8933 3304 0 51200 46 0 crond >[29232.040374] Out of memory: Kill process 8357 (php-fpm) score 4 or sacrifice child >[29232.040395] Killed process 8357 (php-fpm) total-vm:312512kB, anon-rss:0kB, file-rss:64kB, shmem-rss:0kB >[29232.060826] oom_reaper: reaped process 8357 (php-fpm), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29232.110752] sendmail invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29232.110768] sendmail cpuset=/ mems_allowed=0 >[29232.110788] CPU: 6 PID: 2711 Comm: sendmail Not tainted 4.18.7-200.fc28.ppc64 #1 >[29232.110801] Call Trace: >[29232.110814] [c0000000153eb1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29232.110835] [c0000000153eb230] [c0000000002fb670] .dump_header+0x70/0x300 >[29232.110853] [c0000000153eb300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29232.110872] [c0000000153eb3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29232.110891] [c0000000153eb480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29232.110910] [c0000000153eb670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29232.110930] [c0000000153eb730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29232.110951] [c0000000153eb7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29232.110970] [c0000000153eb880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29232.110990] [c0000000153eb990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29232.111009] [c0000000153ebac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29232.111027] [c0000000153ebb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29232.111047] [c0000000153ebcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29232.111067] [c0000000153ebd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29232.111087] [c0000000153ebe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29232.111102] Mem-Info: >[29232.111121] active_anon:24 inactive_anon:97 isolated_anon:1 > active_file:25 inactive_file:50 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2807 slab_unreclaimable:15799 > mapped:56 shmem:1 pagetables:68 bounce:0 > free:1467 free_pcp:2 free_cma:0 >[29232.111171] Node 0 active_anon:1536kB inactive_anon:6208kB active_file:1600kB inactive_file:3200kB unevictable:0kB isolated(anon):64kB isolated(file):0kB mapped:3584kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29232.119974] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1728kB active_file:1536kB inactive_file:960kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9712kB pagetables:4352kB bounce:0kB free_pcp:128kB local_pcp:0kB free_cma:0kB >[29232.120020] lowmem_reserve[]: 0 0 0 0 >[29232.120040] Node 0 DMA: 394*64kB (UME) 361*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 101376kB >[29232.120099] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29232.120116] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29232.120130] 129 total pagecache pages >[29232.120139] 0 pages in swap cache >[29232.120150] Swap cache stats: add 2972694, delete 2972804, find 2522542/4277864 >[29232.120163] Free swap = 1790208kB >[29232.120172] Total swap = 2052032kB >[29232.120181] 32768 pages RAM >[29232.120188] 0 pages HighMem/MovableOnly >[29232.120198] 2684 pages reserved >[29232.120207] 0 pages cma reserved >[29232.120216] 0 pages hwpoisoned >[29232.120224] Unreclaimable slab info: >[29232.120233] Name Used Total >[29232.120342] xfs_efd_item 1017KB 1017KB >[29232.120355] xfs_buf_item 1020KB 1020KB >[29232.120368] xfs_trans 1022KB 1022KB >[29232.120381] xfs_da_state 1020KB 1020KB >[29232.120393] xfs_log_ticket 1023KB 1023KB >[29232.120409] scsi_sense_cache 1024KB 1024KB >[29232.120433] RAWv6 504KB 504KB >[29232.120446] UDPv6 1012KB 1012KB >[29232.120459] tw_sock_TCPv6 127KB 127KB >[29232.120471] TCPv6 1386KB 1386KB >[29232.120485] cfq_io_cq 1023KB 1023KB >[29232.120497] cfq_queue 1023KB 1023KB >[29232.120510] mqueue_inode_cache 64KB 64KB >[29232.120523] kioctx 63KB 63KB >[29232.120537] posix_timers_cache 1022KB 1022KB >[29232.120549] ip4-frags 1023KB 1023KB >[29232.120562] secpath_cache 768KB 768KB >[29232.120574] RAW 896KB 896KB >[29232.120586] UDP 1008KB 1008KB >[29232.120599] tw_sock_TCP 127KB 127KB >[29232.120611] request_sock_TCP 191KB 191KB >[29232.120624] TCP 2040KB 2040KB >[29232.120637] hugetlbfs_inode_cache 127KB 127KB >[29232.120650] eventpoll_pwq 1023KB 1023KB >[29232.120662] inotify_inode_mark 1023KB 1023KB >[29232.120674] request_queue 755KB 755KB >[29232.120688] blkdev_ioc 1023KB 1023KB >[29232.120701] biovec-128 256KB 256KB >[29232.120714] biovec-64 128KB 128KB >[29232.120726] skbuff_fclone_cache 1024KB 1024KB >[29232.120740] skbuff_head_cache 1600KB 1600KB >[29232.120753] file_lock_cache 1021KB 1021KB >[29232.120766] net_namespace 510KB 510KB >[29232.120778] shmem_inode_cache 1087KB 1087KB >[29232.120791] dtl 10040KB 11520KB >[29232.120803] taskstats 764KB 764KB >[29232.120816] proc_dir_entry 640KB 640KB >[29232.120829] pde_opener 1023KB 1023KB >[29232.120841] seq_file 1024KB 1024KB >[29232.120853] sigqueue 1022KB 1022KB >[29232.120865] kernfs_node_cache 5302KB 5302KB >[29232.120878] mnt_cache 2996KB 2996KB >[29232.120892] names_cache 4096KB 4096KB >[29232.120905] ebitmap_node 1024KB 1024KB >[29232.120918] avtab_node 1023KB 1023KB >[29232.120930] avc_xperms_data 1024KB 1024KB >[29232.120942] selinux_file_security 1024KB 1024KB >[29232.120954] selinux_inode_security 1855KB 1855KB >[29232.120968] nsproxy 1023KB 1023KB >[29232.120980] vm_area_struct 1660KB 1660KB >[29232.120992] mm_struct 8056KB 8056KB >[29232.121004] files_cache 1211KB 1211KB >[29232.121017] signal_cache 1472KB 1472KB >[29232.121029] sighand_cache 2932KB 2932KB >[29232.121042] task_struct 12210KB 12210KB >[29232.121055] cred_jar 4800KB 4800KB >[29232.121068] thread_stack 17408KB 17408KB >[29232.121081] anon_vma 1086KB 1086KB >[29232.121093] pid 1280KB 1280KB >[29232.121106] numa_policy 383KB 383KB >[29232.121119] trace_event_file 255KB 255KB >[29232.121131] ftrace_event_field 447KB 447KB >[29232.121144] pool_workqueue 1024KB 1024KB >[29232.129938] task_group 1020KB 1020KB >[29232.129962] pgtable-2^11 9216KB 9216KB >[29232.129975] pgtable-2^8 4096KB 4096KB >[29232.129992] dma-kmalloc-512 128KB 128KB >[29232.130008] kmalloc-131072 35328KB 35328KB >[29232.130020] kmalloc-65536 9216KB 9216KB >[29232.130032] kmalloc-32768 8192KB 8192KB >[29232.130045] kmalloc-16384 18432KB 20480KB >[29232.130057] kmalloc-8192 8704KB 8704KB >[29232.130069] kmalloc-4096 8192KB 8192KB >[29232.130081] kmalloc-2048 4352KB 4352KB >[29232.130093] kmalloc-1024 19331KB 19456KB >[29232.130107] kmalloc-512 10176KB 11072KB >[29232.130121] kmalloc-256 1408KB 1408KB >[29232.130134] kmalloc-192 1790KB 1790KB >[29232.130147] kmalloc-128 1536KB 1536KB >[29232.130160] kmalloc-96 1470KB 1470KB >[29232.130173] kmalloc-64 1792KB 1792KB >[29232.130186] kmalloc-32 12992KB 12992KB >[29232.130198] kmalloc-16 1024KB 1024KB >[29232.130210] kmalloc-8 1024KB 1024KB >[29232.130221] kmem_cache_node 1024KB 1024KB >[29232.130234] kmem_cache 3315KB 3315KB >[29232.130244] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29232.130271] [ 716] 0 716 2668 0 69632 83 0 systemd-journal >[29232.130290] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29232.130309] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29232.130326] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29232.130345] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29232.130362] [ 2069] 81 2069 208 0 51200 63 -900 dbus-daemon >[29232.130381] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29232.130399] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29232.130416] [ 2094] 0 2094 10692 0 122880 93 0 rsyslogd >[29232.130433] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29232.130450] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29232.130468] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29232.130485] [ 2121] 0 2121 307 0 51200 82 -1000 sshd >[29232.130502] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29232.130519] [ 2139] 0 2139 97 0 51200 37 0 atd >[29232.130536] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29232.130553] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29232.130571] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29232.130589] [ 2205] 995 2205 44922 0 133120 222 0 polkitd >[29232.130608] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29232.130626] [ 2337] 0 2337 449 0 53248 82 0 keepalived >[29232.130644] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29232.130662] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29232.130681] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29232.130698] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29232.130715] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29232.130732] [ 2697] 0 2697 87 0 49152 28 0 log >[29232.130748] [ 2699] 0 2699 103 0 51200 32 0 config >[29232.130766] [ 2711] 0 2711 398 0 51200 117 0 sendmail >[29232.130784] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29232.130801] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29232.130819] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29232.130836] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29232.130853] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29232.130871] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29232.130888] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29232.130905] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29232.130922] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29232.130941] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29232.130958] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29232.130975] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29232.130994] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29232.131012] [ 8907] 0 8907 410 0 51200 118 0 sendmail >[29232.131030] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29232.131047] [ 8909] 0 8909 3410 0 51200 61 0 ssh >[29232.131065] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29232.131081] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29232.131098] [ 8912] 0 8912 410 0 51200 123 0 sendmail >[29232.131115] [ 8913] 0 8913 166 0 40960 43 0 ps >[29232.131134] [ 8914] 0 8914 3249 0 53248 14 0 head >[29232.131152] [ 8918] 990 8918 159 0 51200 26 0 pop3-login >[29232.131170] [ 8919] 0 8919 168 0 51200 38 0 auth >[29232.139963] [ 8920] 990 8920 159 0 51200 30 0 pop3-login >[29232.139983] [ 8932] 0 8932 38 0 38912 9 0 sshd >[29232.140000] [ 8933] 0 8933 3304 0 51200 46 0 crond >[29232.140015] Out of memory: Kill process 2205 (polkitd) score 3 or sacrifice child >[29232.140182] Killed process 2205 (polkitd) total-vm:2875008kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29232.180918] oom_reaper: reaped process 2205 (polkitd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29232.210690] sendmail invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29232.210707] sendmail cpuset=/ mems_allowed=0 >[29232.210727] CPU: 7 PID: 8907 Comm: sendmail Not tainted 4.18.7-200.fc28.ppc64 #1 >[29232.210740] Call Trace: >[29232.210753] [c000000015c631a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29232.210773] [c000000015c63230] [c0000000002fb670] .dump_header+0x70/0x300 >[29232.210791] [c000000015c63300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29232.210810] [c000000015c633d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29232.210829] [c000000015c63480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29232.210848] [c000000015c63670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29232.210868] [c000000015c63730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29232.210888] [c000000015c637f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29232.210906] [c000000015c63880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29232.210927] [c000000015c63990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29232.210945] [c000000015c63ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29232.210963] [c000000015c63b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29232.210981] [c000000015c63cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29232.210999] [c000000015c63d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29232.211019] [c000000015c63e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29232.211034] Mem-Info: >[29232.211054] active_anon:12 inactive_anon:89 isolated_anon:1 > active_file:18 inactive_file:55 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2807 slab_unreclaimable:15799 > mapped:53 shmem:0 pagetables:68 bounce:0 > free:1466 free_pcp:8 free_cma:0 >[29232.211107] Node 0 active_anon:768kB inactive_anon:5696kB active_file:1152kB inactive_file:3520kB unevictable:0kB isolated(anon):64kB isolated(file):0kB mapped:3392kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29232.211137] Node 0 DMA free:93824kB min:94144kB low:117632kB high:141120kB active_anon:704kB inactive_anon:1216kB active_file:1088kB inactive_file:1280kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9712kB pagetables:4352kB bounce:0kB free_pcp:512kB local_pcp:0kB free_cma:0kB >[29232.219954] lowmem_reserve[]: 0 0 0 0 >[29232.219977] Node 0 DMA: 393*64kB (UME) 357*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100800kB >[29232.220036] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29232.220052] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29232.220066] 113 total pagecache pages >[29232.220075] 0 pages in swap cache >[29232.220086] Swap cache stats: add 2972745, delete 2972855, find 2522569/4277914 >[29232.220099] Free swap = 1806592kB >[29232.220108] Total swap = 2052032kB >[29232.220118] 32768 pages RAM >[29232.220125] 0 pages HighMem/MovableOnly >[29232.220134] 2684 pages reserved >[29232.220143] 0 pages cma reserved >[29232.220152] 0 pages hwpoisoned >[29232.220161] Unreclaimable slab info: >[29232.220170] Name Used Total >[29232.220252] xfs_efd_item 1017KB 1017KB >[29232.220265] xfs_buf_item 1020KB 1020KB >[29232.220276] xfs_trans 1022KB 1022KB >[29232.220288] xfs_da_state 1020KB 1020KB >[29232.220300] xfs_log_ticket 1023KB 1023KB >[29232.220315] scsi_sense_cache 1024KB 1024KB >[29232.220334] RAWv6 504KB 504KB >[29232.220347] UDPv6 1012KB 1012KB >[29232.220359] tw_sock_TCPv6 127KB 127KB >[29232.220372] TCPv6 1386KB 1386KB >[29232.220385] cfq_io_cq 1023KB 1023KB >[29232.220398] cfq_queue 1023KB 1023KB >[29232.220410] mqueue_inode_cache 64KB 64KB >[29232.220423] kioctx 63KB 63KB >[29232.220437] posix_timers_cache 1022KB 1022KB >[29232.220449] ip4-frags 1023KB 1023KB >[29232.220461] secpath_cache 768KB 768KB >[29232.220474] RAW 896KB 896KB >[29232.220486] UDP 1008KB 1008KB >[29232.220498] tw_sock_TCP 127KB 127KB >[29232.220511] request_sock_TCP 191KB 191KB >[29232.220523] TCP 2040KB 2040KB >[29232.220536] hugetlbfs_inode_cache 127KB 127KB >[29232.220548] eventpoll_pwq 1023KB 1023KB >[29232.220560] inotify_inode_mark 1023KB 1023KB >[29232.220573] request_queue 755KB 755KB >[29232.220587] blkdev_ioc 1023KB 1023KB >[29232.220599] biovec-128 256KB 256KB >[29232.220611] biovec-64 128KB 128KB >[29232.220623] skbuff_fclone_cache 1024KB 1024KB >[29232.220635] skbuff_head_cache 1600KB 1600KB >[29232.220647] file_lock_cache 1021KB 1021KB >[29232.220660] net_namespace 510KB 510KB >[29232.220672] shmem_inode_cache 1087KB 1087KB >[29232.220684] dtl 10040KB 11520KB >[29232.220697] taskstats 764KB 764KB >[29232.220709] proc_dir_entry 640KB 640KB >[29232.220721] pde_opener 1023KB 1023KB >[29232.220734] seq_file 1024KB 1024KB >[29232.220746] sigqueue 1022KB 1022KB >[29232.220758] kernfs_node_cache 5302KB 5302KB >[29232.220770] mnt_cache 2996KB 2996KB >[29232.220782] names_cache 4096KB 4096KB >[29232.220794] ebitmap_node 1024KB 1024KB >[29232.220806] avtab_node 1023KB 1023KB >[29232.220818] avc_xperms_data 1024KB 1024KB >[29232.220831] selinux_file_security 1024KB 1024KB >[29232.220843] selinux_inode_security 1855KB 1855KB >[29232.220857] nsproxy 1023KB 1023KB >[29232.220868] vm_area_struct 1660KB 1660KB >[29232.220880] mm_struct 8056KB 8056KB >[29232.220892] files_cache 1211KB 1211KB >[29232.220905] signal_cache 1472KB 1472KB >[29232.220917] sighand_cache 2932KB 2932KB >[29232.220929] task_struct 12210KB 12210KB >[29232.220941] cred_jar 4800KB 4800KB >[29232.220952] thread_stack 17408KB 17408KB >[29232.220964] anon_vma 1086KB 1086KB >[29232.220976] pid 1280KB 1280KB >[29232.220988] numa_policy 383KB 383KB >[29232.220999] trace_event_file 255KB 255KB >[29232.221011] ftrace_event_field 447KB 447KB >[29232.221023] pool_workqueue 1024KB 1024KB >[29232.221037] task_group 1020KB 1020KB >[29232.221049] pgtable-2^11 9216KB 9216KB >[29232.221061] pgtable-2^8 4096KB 4096KB >[29232.221078] dma-kmalloc-512 128KB 128KB >[29232.221096] kmalloc-131072 35328KB 35328KB >[29232.221109] kmalloc-65536 9216KB 9216KB >[29232.221121] kmalloc-32768 8192KB 8192KB >[29232.221134] kmalloc-16384 18432KB 20480KB >[29232.221146] kmalloc-8192 8704KB 8704KB >[29232.229960] kmalloc-4096 8192KB 8192KB >[29232.229974] kmalloc-2048 4352KB 4352KB >[29232.229987] kmalloc-1024 19331KB 19456KB >[29232.230001] kmalloc-512 10176KB 11072KB >[29232.230013] kmalloc-256 1408KB 1408KB >[29232.230025] kmalloc-192 1790KB 1790KB >[29232.230036] kmalloc-128 1536KB 1536KB >[29232.230048] kmalloc-96 1470KB 1470KB >[29232.230060] kmalloc-64 1792KB 1792KB >[29232.230071] kmalloc-32 12992KB 12992KB >[29232.230083] kmalloc-16 1024KB 1024KB >[29232.230095] kmalloc-8 1024KB 1024KB >[29232.230106] kmem_cache_node 1024KB 1024KB >[29232.230118] kmem_cache 3315KB 3315KB >[29232.230128] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29232.230154] [ 716] 0 716 2668 0 69632 83 0 systemd-journal >[29232.230173] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29232.230191] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29232.230208] [ 2067] 0 2067 328 0 51200 87 0 systemd-logind >[29232.230226] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29232.230243] [ 2069] 81 2069 208 0 51200 63 -900 dbus-daemon >[29232.230262] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29232.230279] [ 2093] 0 2093 7324 0 63488 198 0 NetworkManager >[29232.230297] [ 2094] 0 2094 10692 0 122880 93 0 rsyslogd >[29232.230313] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29232.230330] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29232.230347] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29232.230364] [ 2121] 0 2121 307 0 51200 82 -1000 sshd >[29232.230380] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29232.230397] [ 2139] 0 2139 97 0 51200 37 0 atd >[29232.230414] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29232.230431] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29232.230449] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29232.230467] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29232.230484] [ 2337] 0 2337 449 0 53248 82 0 keepalived >[29232.230501] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29232.230519] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29232.230537] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29232.230553] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29232.230570] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29232.230587] [ 2697] 0 2697 87 0 49152 28 0 log >[29232.230604] [ 2699] 0 2699 103 0 51200 32 0 config >[29232.230621] [ 2711] 0 2711 398 0 51200 117 0 sendmail >[29232.230639] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29232.230656] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29232.230672] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29232.230689] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29232.230705] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29232.230722] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29232.230739] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29232.230757] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29232.230775] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29232.230793] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29232.230810] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29232.230827] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29232.230845] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29232.230863] [ 8907] 0 8907 410 0 51200 118 0 sendmail >[29232.230880] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29232.230898] [ 8909] 0 8909 3410 0 51200 61 0 ssh >[29232.230915] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29232.230932] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29232.230949] [ 8912] 0 8912 410 0 51200 123 0 sendmail >[29232.230966] [ 8913] 0 8913 166 0 40960 43 0 ps >[29232.230984] [ 8914] 0 8914 3249 0 53248 14 0 head >[29232.231002] [ 8918] 990 8918 159 0 51200 26 0 pop3-login >[29232.231020] [ 8919] 0 8919 168 0 51200 38 0 auth >[29232.231037] [ 8920] 990 8920 159 0 51200 30 0 pop3-login >[29232.231057] [ 8932] 0 8932 38 0 38912 9 0 sshd >[29232.231072] [ 8933] 0 8933 3304 0 51200 46 0 crond >[29232.231086] Out of memory: Kill process 2093 (NetworkManager) score 3 or sacrifice child >[29232.231124] Killed process 2093 (NetworkManager) total-vm:468736kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29232.270372] oom_reaper: reaped process 2093 (NetworkManager), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29235.050999] systemd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29235.051021] systemd cpuset=/ mems_allowed=0 >[29235.051036] CPU: 10 PID: 1 Comm: systemd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29235.051044] Call Trace: >[29235.051055] [c00000007e9ff1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29235.051068] [c00000007e9ff230] [c0000000002fb670] .dump_header+0x70/0x300 >[29235.051078] [c00000007e9ff300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29235.051089] [c00000007e9ff3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29235.051099] [c00000007e9ff480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29235.051112] [c00000007e9ff670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29235.051123] [c00000007e9ff730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29235.051134] [c00000007e9ff7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29235.051146] [c00000007e9ff880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29235.051157] [c00000007e9ff990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29235.051169] [c00000007e9ffac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29235.059952] [c00000007e9ffb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29235.059971] [c00000007e9ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29235.059985] [c00000007e9ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29235.059997] [c00000007e9ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29235.060015] Mem-Info: >[29235.060028] active_anon:11 inactive_anon:42 isolated_anon:0 > active_file:21 inactive_file:44 isolated_file:0 > unevictable:0 dirty:2 writeback:2 unstable:0 > slab_reclaimable:2793 slab_unreclaimable:15799 > mapped:65 shmem:0 pagetables:65 bounce:0 > free:1468 free_pcp:0 free_cma:0 >[29235.060059] Node 0 active_anon:704kB inactive_anon:2688kB active_file:1344kB inactive_file:2816kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4160kB dirty:128kB writeback:128kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29235.060076] Node 0 DMA free:93952kB min:94144kB low:117632kB high:141120kB active_anon:704kB inactive_anon:1216kB active_file:1344kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9296kB pagetables:4160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29235.060101] lowmem_reserve[]: 0 0 0 0 >[29235.060112] Node 0 DMA: 417*64kB (UME) 364*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103232kB >[29235.060147] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29235.060156] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29235.060164] 86 total pagecache pages >[29235.060170] 0 pages in swap cache >[29235.060176] Swap cache stats: add 2975564, delete 2975677, find 2524848/4282136 >[29235.060184] Free swap = 1827072kB >[29235.060189] Total swap = 2052032kB >[29235.060194] 32768 pages RAM >[29235.060198] 0 pages HighMem/MovableOnly >[29235.060204] 2684 pages reserved >[29235.060209] 0 pages cma reserved >[29235.060214] 0 pages hwpoisoned >[29235.060219] Unreclaimable slab info: >[29235.060224] Name Used Total >[29235.060405] xfs_efd_item 1017KB 1017KB >[29235.060413] xfs_buf_item 1020KB 1020KB >[29235.060420] xfs_trans 1022KB 1022KB >[29235.060427] xfs_da_state 1020KB 1020KB >[29235.060434] xfs_log_ticket 1023KB 1023KB >[29235.060447] scsi_sense_cache 1024KB 1024KB >[29235.060470] RAWv6 504KB 504KB >[29235.060478] UDPv6 1012KB 1012KB >[29235.060485] tw_sock_TCPv6 127KB 127KB >[29235.060493] TCPv6 1386KB 1386KB >[29235.060501] cfq_io_cq 1023KB 1023KB >[29235.060509] cfq_queue 1023KB 1023KB >[29235.060517] mqueue_inode_cache 64KB 64KB >[29235.060525] kioctx 63KB 63KB >[29235.060533] posix_timers_cache 1022KB 1022KB >[29235.060541] ip4-frags 1023KB 1023KB >[29235.060548] secpath_cache 768KB 768KB >[29235.060555] RAW 896KB 896KB >[29235.060562] UDP 1008KB 1008KB >[29235.060569] tw_sock_TCP 127KB 127KB >[29235.060575] request_sock_TCP 191KB 191KB >[29235.060583] TCP 2040KB 2040KB >[29235.060589] hugetlbfs_inode_cache 127KB 127KB >[29235.060597] eventpoll_pwq 1023KB 1023KB >[29235.060604] inotify_inode_mark 1023KB 1023KB >[29235.060611] request_queue 755KB 755KB >[29235.060619] blkdev_ioc 1023KB 1023KB >[29235.060626] biovec-128 256KB 256KB >[29235.060633] biovec-64 128KB 128KB >[29235.060640] skbuff_fclone_cache 1024KB 1024KB >[29235.060647] skbuff_head_cache 1600KB 1600KB >[29235.060654] file_lock_cache 1021KB 1021KB >[29235.060661] net_namespace 510KB 510KB >[29235.060668] shmem_inode_cache 1087KB 1087KB >[29235.060676] dtl 10040KB 11520KB >[29235.060683] taskstats 764KB 764KB >[29235.060690] proc_dir_entry 640KB 640KB >[29235.060697] pde_opener 1023KB 1023KB >[29235.060705] seq_file 1024KB 1024KB >[29235.060712] sigqueue 1022KB 1022KB >[29235.060719] kernfs_node_cache 5302KB 5302KB >[29235.060726] mnt_cache 2996KB 2996KB >[29235.060733] names_cache 4096KB 4096KB >[29235.060741] ebitmap_node 1024KB 1024KB >[29235.060748] avtab_node 1023KB 1023KB >[29235.060755] avc_xperms_data 1024KB 1024KB >[29235.060762] selinux_file_security 1024KB 1024KB >[29235.060769] selinux_inode_security 1855KB 1855KB >[29235.060776] nsproxy 1023KB 1023KB >[29235.060783] vm_area_struct 1660KB 1660KB >[29235.060790] mm_struct 8056KB 8056KB >[29235.060797] files_cache 1211KB 1211KB >[29235.060804] signal_cache 1472KB 1472KB >[29235.060811] sighand_cache 2932KB 2932KB >[29235.060817] task_struct 12210KB 12210KB >[29235.060824] cred_jar 4800KB 4800KB >[29235.060831] thread_stack 17408KB 17408KB >[29235.060838] anon_vma 1086KB 1086KB >[29235.060845] pid 1280KB 1280KB >[29235.060852] numa_policy 383KB 383KB >[29235.060859] trace_event_file 255KB 255KB >[29235.060865] ftrace_event_field 447KB 447KB >[29235.060872] pool_workqueue 1024KB 1024KB >[29235.060879] task_group 1020KB 1020KB >[29235.060886] pgtable-2^11 9216KB 9216KB >[29235.060893] pgtable-2^8 4096KB 4096KB >[29235.060902] dma-kmalloc-512 128KB 128KB >[29235.060912] kmalloc-131072 35328KB 35328KB >[29235.060919] kmalloc-65536 9216KB 9216KB >[29235.060925] kmalloc-32768 8192KB 8192KB >[29235.060933] kmalloc-16384 18432KB 20480KB >[29235.060941] kmalloc-8192 8704KB 8704KB >[29235.060947] kmalloc-4096 8192KB 8192KB >[29235.060954] kmalloc-2048 4352KB 4352KB >[29235.060962] kmalloc-1024 19331KB 19456KB >[29235.060970] kmalloc-512 10176KB 11072KB >[29235.060976] kmalloc-256 1408KB 1408KB >[29235.060982] kmalloc-192 1790KB 1790KB >[29235.060988] kmalloc-128 1536KB 1536KB >[29235.060994] kmalloc-96 1470KB 1470KB >[29235.061000] kmalloc-64 1792KB 1792KB >[29235.061006] kmalloc-32 12992KB 12992KB >[29235.061012] kmalloc-16 1024KB 1024KB >[29235.061017] kmalloc-8 1024KB 1024KB >[29235.061024] kmem_cache_node 1024KB 1024KB >[29235.061030] kmem_cache 3315KB 3315KB >[29235.061036] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29235.061059] [ 716] 0 716 2668 0 69632 92 0 systemd-journal >[29235.061069] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29235.061080] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29235.061090] [ 2067] 0 2067 328 1 51200 73 0 systemd-logind >[29235.061099] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29235.061108] [ 2069] 81 2069 208 0 51200 70 -900 dbus-daemon >[29235.061118] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29235.061127] [ 2094] 0 2094 10692 0 122880 86 0 rsyslogd >[29235.061137] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29235.061146] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29235.069951] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29235.069970] [ 2121] 0 2121 307 0 51200 71 -1000 sshd >[29235.069986] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29235.070002] [ 2139] 0 2139 97 0 51200 37 0 atd >[29235.070018] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29235.070034] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29235.070050] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29235.070067] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29235.070084] [ 2337] 0 2337 449 0 53248 82 0 keepalived >[29235.070101] [ 2338] 0 2338 452 0 53248 93 0 keepalived >[29235.070118] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29235.070135] [ 2607] 177 2607 534 0 53248 182 0 dhcpd >[29235.070151] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29235.070168] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29235.070184] [ 2697] 0 2697 87 0 49152 29 0 log >[29235.070201] [ 2699] 0 2699 103 0 51200 32 0 config >[29235.070217] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29235.070233] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29235.070250] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29235.070267] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29235.070283] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29235.070300] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29235.070317] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29235.070333] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29235.070350] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29235.070368] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29235.070385] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29235.070400] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29235.070416] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29235.070433] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29235.070449] [ 8907] 0 8907 410 0 51200 134 0 sendmail >[29235.070465] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29235.070481] [ 8909] 0 8909 3410 0 51200 50 0 ssh >[29235.070497] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29235.070513] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29235.070529] [ 8912] 0 8912 410 0 51200 130 0 sendmail >[29235.070545] [ 8913] 0 8913 166 0 40960 39 0 ps >[29235.070561] [ 8914] 0 8914 3249 0 53248 14 0 head >[29235.070578] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29235.070595] [ 8919] 0 8919 168 0 51200 41 0 auth >[29235.070612] [ 8920] 990 8920 159 0 51200 33 0 pop3-login >[29235.070629] [ 8932] 0 8932 118 0 38912 21 0 sshd >[29235.070644] [ 8933] 0 8933 3307 0 51200 52 0 crond >[29235.070658] Out of memory: Kill process 2607 (dhcpd) score 2 or sacrifice child >[29235.070688] Killed process 2607 (dhcpd) total-vm:34176kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29235.090467] oom_reaper: reaped process 2607 (dhcpd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29235.140806] sendmail invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29235.140823] sendmail cpuset=/ mems_allowed=0 >[29235.140842] CPU: 11 PID: 8912 Comm: sendmail Not tainted 4.18.7-200.fc28.ppc64 #1 >[29235.140854] Call Trace: >[29235.140867] [c0000000067ff1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29235.140886] [c0000000067ff230] [c0000000002fb670] .dump_header+0x70/0x300 >[29235.140903] [c0000000067ff300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29235.140921] [c0000000067ff3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29235.140940] [c0000000067ff480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29235.140959] [c0000000067ff670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29235.140978] [c0000000067ff730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29235.140996] [c0000000067ff7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29235.141016] [c0000000067ff880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29235.141036] [c0000000067ff990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29235.141054] [c0000000067ffac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29235.141072] [c0000000067ffb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29235.141091] [c0000000067ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29235.141110] [c0000000067ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29235.141130] [c0000000067ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29235.141144] Mem-Info: >[29235.141162] active_anon:11 inactive_anon:42 isolated_anon:0 > active_file:21 inactive_file:44 isolated_file:0 > unevictable:0 dirty:2 writeback:2 unstable:0 > slab_reclaimable:2793 slab_unreclaimable:15799 > mapped:65 shmem:0 pagetables:65 bounce:0 > free:1468 free_pcp:1 free_cma:0 >[29235.149995] Node 0 active_anon:704kB inactive_anon:2688kB active_file:1344kB inactive_file:2816kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4160kB dirty:128kB writeback:128kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29235.150026] Node 0 DMA free:93952kB min:94144kB low:117632kB high:141120kB active_anon:704kB inactive_anon:1216kB active_file:1344kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9296kB pagetables:4160kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29235.150069] lowmem_reserve[]: 0 0 0 0 >[29235.150087] Node 0 DMA: 415*64kB (UME) 364*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103104kB >[29235.150143] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29235.150159] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29235.150172] 86 total pagecache pages >[29235.150181] 0 pages in swap cache >[29235.150191] Swap cache stats: add 2975565, delete 2975678, find 2524849/4282138 >[29235.150203] Free swap = 1839360kB >[29235.150212] Total swap = 2052032kB >[29235.150222] 32768 pages RAM >[29235.150230] 0 pages HighMem/MovableOnly >[29235.150238] 2684 pages reserved >[29235.150246] 0 pages cma reserved >[29235.150255] 0 pages hwpoisoned >[29235.150263] Unreclaimable slab info: >[29235.150272] Name Used Total >[29235.150327] xfs_efd_item 1017KB 1017KB >[29235.150340] xfs_buf_item 1020KB 1020KB >[29235.150352] xfs_trans 1022KB 1022KB >[29235.150364] xfs_da_state 1020KB 1020KB >[29235.150376] xfs_log_ticket 1023KB 1023KB >[29235.150389] scsi_sense_cache 1024KB 1024KB >[29235.150405] RAWv6 504KB 504KB >[29235.150417] UDPv6 1012KB 1012KB >[29235.150429] tw_sock_TCPv6 127KB 127KB >[29235.150442] TCPv6 1386KB 1386KB >[29235.150455] cfq_io_cq 1023KB 1023KB >[29235.150467] cfq_queue 1023KB 1023KB >[29235.150479] mqueue_inode_cache 64KB 64KB >[29235.150491] kioctx 63KB 63KB >[29235.150504] posix_timers_cache 1022KB 1022KB >[29235.150516] ip4-frags 1023KB 1023KB >[29235.150528] secpath_cache 768KB 768KB >[29235.150540] RAW 896KB 896KB >[29235.150552] UDP 1008KB 1008KB >[29235.150564] tw_sock_TCP 127KB 127KB >[29235.150576] request_sock_TCP 191KB 191KB >[29235.150588] TCP 2040KB 2040KB >[29235.150600] hugetlbfs_inode_cache 127KB 127KB >[29235.150612] eventpoll_pwq 1023KB 1023KB >[29235.150624] inotify_inode_mark 1023KB 1023KB >[29235.150636] request_queue 755KB 755KB >[29235.150649] blkdev_ioc 1023KB 1023KB >[29235.150660] biovec-128 256KB 256KB >[29235.150672] biovec-64 128KB 128KB >[29235.150684] skbuff_fclone_cache 1024KB 1024KB >[29235.150696] skbuff_head_cache 1600KB 1600KB >[29235.150708] file_lock_cache 1021KB 1021KB >[29235.150720] net_namespace 510KB 510KB >[29235.150732] shmem_inode_cache 1087KB 1087KB >[29235.150745] dtl 10040KB 11520KB >[29235.150758] taskstats 764KB 764KB >[29235.150771] proc_dir_entry 640KB 640KB >[29235.150783] pde_opener 1023KB 1023KB >[29235.150795] seq_file 1024KB 1024KB >[29235.150807] sigqueue 1022KB 1022KB >[29235.150819] kernfs_node_cache 5302KB 5302KB >[29235.150831] mnt_cache 2996KB 2996KB >[29235.150843] names_cache 4096KB 4096KB >[29235.150855] ebitmap_node 1024KB 1024KB >[29235.150867] avtab_node 1023KB 1023KB >[29235.150879] avc_xperms_data 1024KB 1024KB >[29235.150892] selinux_file_security 1024KB 1024KB >[29235.150904] selinux_inode_security 1855KB 1855KB >[29235.150916] nsproxy 1023KB 1023KB >[29235.150927] vm_area_struct 1660KB 1660KB >[29235.150939] mm_struct 8056KB 8056KB >[29235.150950] files_cache 1211KB 1211KB >[29235.150962] signal_cache 1472KB 1472KB >[29235.150973] sighand_cache 2932KB 2932KB >[29235.150986] task_struct 12210KB 12210KB >[29235.150997] cred_jar 4800KB 4800KB >[29235.151009] thread_stack 17408KB 17408KB >[29235.151021] anon_vma 1086KB 1086KB >[29235.151032] pid 1280KB 1280KB >[29235.151044] numa_policy 383KB 383KB >[29235.151056] trace_event_file 255KB 255KB >[29235.151068] ftrace_event_field 447KB 447KB >[29235.151079] pool_workqueue 1024KB 1024KB >[29235.151090] task_group 1020KB 1020KB >[29235.151102] pgtable-2^11 9216KB 9216KB >[29235.151114] pgtable-2^8 4096KB 4096KB >[29235.151130] dma-kmalloc-512 128KB 128KB >[29235.151146] kmalloc-131072 35328KB 35328KB >[29235.151158] kmalloc-65536 9216KB 9216KB >[29235.151170] kmalloc-32768 8192KB 8192KB >[29235.159952] kmalloc-16384 18432KB 20480KB >[29235.159965] kmalloc-8192 8704KB 8704KB >[29235.159977] kmalloc-4096 8192KB 8192KB >[29235.159989] kmalloc-2048 4352KB 4352KB >[29235.160002] kmalloc-1024 19331KB 19456KB >[29235.160014] kmalloc-512 10176KB 11072KB >[29235.160026] kmalloc-256 1408KB 1408KB >[29235.160038] kmalloc-192 1790KB 1790KB >[29235.160050] kmalloc-128 1536KB 1536KB >[29235.160062] kmalloc-96 1470KB 1470KB >[29235.160074] kmalloc-64 1792KB 1792KB >[29235.160085] kmalloc-32 12992KB 12992KB >[29235.160097] kmalloc-16 1024KB 1024KB >[29235.160109] kmalloc-8 1024KB 1024KB >[29235.160120] kmem_cache_node 1024KB 1024KB >[29235.160132] kmem_cache 3315KB 3315KB >[29235.160142] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29235.160177] [ 716] 0 716 2668 0 69632 92 0 systemd-journal >[29235.160196] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29235.160215] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29235.160232] [ 2067] 0 2067 328 1 51200 73 0 systemd-logind >[29235.160249] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29235.160266] [ 2069] 81 2069 208 0 51200 70 -900 dbus-daemon >[29235.160284] [ 2089] 0 2089 1288 0 51200 51 0 irqbalance >[29235.160302] [ 2094] 0 2094 10692 0 122880 86 0 rsyslogd >[29235.160319] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29235.160335] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29235.160352] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29235.160368] [ 2121] 0 2121 307 0 51200 71 -1000 sshd >[29235.160385] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29235.160402] [ 2139] 0 2139 97 0 51200 37 0 atd >[29235.160419] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29235.160437] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29235.160453] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29235.160472] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29235.160489] [ 2337] 0 2337 449 0 53248 82 0 keepalived >[29235.160507] [ 2338] 0 2338 452 0 53248 93 0 keepalived >[29235.160525] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29235.160544] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29235.160560] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29235.160576] [ 2697] 0 2697 87 0 49152 29 0 log >[29235.160593] [ 2699] 0 2699 103 0 51200 32 0 config >[29235.160609] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29235.160626] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29235.160645] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29235.160662] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29235.160678] [ 5050] 0 5050 380 0 51200 87 0 systemd >[29235.160695] [ 5051] 0 5051 2886 0 59392 176 0 (sd-pam) >[29235.160712] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29235.160728] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29235.160745] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29235.160763] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29235.160783] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29235.160800] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29235.160817] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29235.160836] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29235.160852] [ 8907] 0 8907 410 0 51200 134 0 sendmail >[29235.160869] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29235.160885] [ 8909] 0 8909 3410 0 51200 50 0 ssh >[29235.160902] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29235.160918] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29235.160934] [ 8912] 0 8912 410 0 51200 130 0 sendmail >[29235.160950] [ 8913] 0 8913 166 0 40960 39 0 ps >[29235.160967] [ 8914] 0 8914 3249 0 53248 14 0 head >[29235.160984] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29235.161002] [ 8919] 0 8919 168 0 51200 41 0 auth >[29235.161019] [ 8920] 990 8920 159 0 51200 33 0 pop3-login >[29235.161038] [ 8932] 0 8932 118 0 38912 21 0 sshd >[29235.161055] [ 8933] 0 8933 3307 0 51200 52 0 crond >[29235.161070] Out of memory: Kill process 5051 ((sd-pam)) score 2 or sacrifice child >[29235.161098] Killed process 5051 ((sd-pam)) total-vm:184704kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29235.180118] oom_reaper: reaped process 5051 ((sd-pam)), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29237.210752] ps invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29237.210765] ps cpuset=/ mems_allowed=0 >[29237.210774] CPU: 15 PID: 8913 Comm: ps Not tainted 4.18.7-200.fc28.ppc64 #1 >[29237.210777] Call Trace: >[29237.210783] [c00000005867b1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29237.210790] [c00000005867b230] [c0000000002fb670] .dump_header+0x70/0x300 >[29237.210794] [c00000005867b300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29237.210799] [c00000005867b3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29237.210804] [c00000005867b480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29237.210810] [c00000005867b670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29237.210815] [c00000005867b730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29237.210820] [c00000005867b7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29237.210825] [c00000005867b880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29237.210830] [c00000005867b990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29237.210835] [c00000005867bac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29237.210839] [c00000005867bb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29237.210844] [c00000005867bcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29237.210850] [c00000005867bd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29237.210855] [c00000005867be30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29237.210859] Mem-Info: >[29237.210865] active_anon:0 inactive_anon:101 isolated_anon:0 > active_file:0 inactive_file:84 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2791 slab_unreclaimable:15803 > mapped:71 shmem:0 pagetables:65 bounce:0 > free:1472 free_pcp:0 free_cma:0 >[29237.210880] Node 0 active_anon:0kB inactive_anon:6464kB active_file:0kB inactive_file:5376kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:4544kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29237.210889] Node 0 DMA free:94208kB min:94144kB low:117632kB high:141120kB active_anon:2432kB inactive_anon:3392kB active_file:0kB inactive_file:448kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9312kB pagetables:4160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29237.210903] lowmem_reserve[]: 0 0 0 0 >[29237.210909] Node 0 DMA: 412*64kB (UME) 364*128kB (UME) 103*256kB (UME) 4*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102400kB >[29237.210934] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29237.210942] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29237.210948] 125 total pagecache pages >[29237.210952] 14 pages in swap cache >[29237.210958] Swap cache stats: add 2977964, delete 2978062, find 2526809/4285761 >[29237.210964] Free swap = 1831168kB >[29237.210969] Total swap = 2052032kB >[29237.210972] 32768 pages RAM >[29237.210975] 0 pages HighMem/MovableOnly >[29237.210977] 2684 pages reserved >[29237.210979] 0 pages cma reserved >[29237.210982] 0 pages hwpoisoned >[29237.210984] Unreclaimable slab info: >[29237.210987] Name Used Total >[29237.211146] xfs_efd_item 1017KB 1017KB >[29237.211150] xfs_buf_item 1020KB 1020KB >[29237.211153] xfs_trans 1022KB 1022KB >[29237.211156] xfs_da_state 1020KB 1020KB >[29237.211159] xfs_log_ticket 1023KB 1023KB >[29237.211167] scsi_sense_cache 1024KB 1024KB >[29237.219968] RAWv6 504KB 504KB >[29237.219975] UDPv6 1012KB 1012KB >[29237.219978] tw_sock_TCPv6 127KB 127KB >[29237.219982] TCPv6 1386KB 1386KB >[29237.219986] cfq_io_cq 1023KB 1023KB >[29237.219989] cfq_queue 1023KB 1023KB >[29237.219992] mqueue_inode_cache 64KB 64KB >[29237.219996] kioctx 63KB 63KB >[29237.220000] posix_timers_cache 1022KB 1022KB >[29237.220003] ip4-frags 1023KB 1023KB >[29237.220006] secpath_cache 768KB 768KB >[29237.220009] RAW 896KB 896KB >[29237.220012] UDP 1008KB 1008KB >[29237.220015] tw_sock_TCP 127KB 127KB >[29237.220018] request_sock_TCP 191KB 191KB >[29237.220021] TCP 2040KB 2040KB >[29237.220024] hugetlbfs_inode_cache 127KB 127KB >[29237.220028] eventpoll_pwq 1023KB 1023KB >[29237.220032] inotify_inode_mark 1023KB 1023KB >[29237.220037] request_queue 755KB 755KB >[29237.220044] blkdev_ioc 1023KB 1023KB >[29237.220053] biovec-128 256KB 256KB >[29237.220062] biovec-64 128KB 128KB >[29237.220071] skbuff_fclone_cache 1024KB 1024KB >[29237.220081] skbuff_head_cache 1600KB 1600KB >[29237.220089] file_lock_cache 1021KB 1021KB >[29237.220096] net_namespace 510KB 510KB >[29237.220103] shmem_inode_cache 1087KB 1087KB >[29237.220111] dtl 10040KB 11520KB >[29237.220119] taskstats 764KB 764KB >[29237.220128] proc_dir_entry 640KB 640KB >[29237.220135] pde_opener 1023KB 1023KB >[29237.220141] seq_file 1024KB 1024KB >[29237.220146] sigqueue 1022KB 1022KB >[29237.220150] kernfs_node_cache 5302KB 5302KB >[29237.220155] mnt_cache 2996KB 2996KB >[29237.220160] names_cache 4096KB 4096KB >[29237.220165] ebitmap_node 1024KB 1024KB >[29237.220172] avtab_node 1023KB 1023KB >[29237.220177] avc_xperms_data 1024KB 1024KB >[29237.220183] selinux_file_security 1024KB 1024KB >[29237.220189] selinux_inode_security 1855KB 1855KB >[29237.220193] nsproxy 1023KB 1023KB >[29237.220198] vm_area_struct 1660KB 1660KB >[29237.220203] mm_struct 8056KB 8056KB >[29237.220208] files_cache 1211KB 1211KB >[29237.220213] signal_cache 1472KB 1472KB >[29237.220218] sighand_cache 2932KB 2932KB >[29237.220223] task_struct 12210KB 12210KB >[29237.220228] cred_jar 4800KB 4800KB >[29237.220233] thread_stack 17408KB 17408KB >[29237.220238] anon_vma 1086KB 1086KB >[29237.220243] pid 1280KB 1280KB >[29237.220248] numa_policy 383KB 383KB >[29237.220253] trace_event_file 255KB 255KB >[29237.220258] ftrace_event_field 447KB 447KB >[29237.220264] pool_workqueue 1024KB 1024KB >[29237.220269] task_group 1020KB 1020KB >[29237.220274] pgtable-2^11 9216KB 9216KB >[29237.220278] pgtable-2^8 4096KB 4096KB >[29237.220286] dma-kmalloc-512 128KB 128KB >[29237.220293] kmalloc-131072 35328KB 35328KB >[29237.220297] kmalloc-65536 9216KB 9216KB >[29237.220301] kmalloc-32768 8192KB 8192KB >[29237.220305] kmalloc-16384 18432KB 20480KB >[29237.220309] kmalloc-8192 8704KB 8704KB >[29237.220313] kmalloc-4096 8192KB 8192KB >[29237.220317] kmalloc-2048 4352KB 4352KB >[29237.220323] kmalloc-1024 19331KB 19456KB >[29237.220328] kmalloc-512 10176KB 11072KB >[29237.220332] kmalloc-256 1408KB 1408KB >[29237.220336] kmalloc-192 1790KB 1790KB >[29237.220340] kmalloc-128 1536KB 1536KB >[29237.220344] kmalloc-96 1470KB 1470KB >[29237.220348] kmalloc-64 1792KB 1792KB >[29237.220352] kmalloc-32 12992KB 12992KB >[29237.220356] kmalloc-16 1024KB 1024KB >[29237.220360] kmalloc-8 1024KB 1024KB >[29237.220364] kmem_cache_node 1024KB 1024KB >[29237.220367] kmem_cache 3315KB 3315KB >[29237.220370] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29237.220383] [ 716] 0 716 2668 0 69632 80 0 systemd-journal >[29237.220388] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29237.220394] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29237.220399] [ 2067] 0 2067 328 0 51200 77 0 systemd-logind >[29237.220403] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29237.220409] [ 2069] 81 2069 208 0 51200 57 -900 dbus-daemon >[29237.220415] [ 2089] 0 2089 1288 0 51200 39 0 irqbalance >[29237.220421] [ 2094] 0 2094 10692 0 122880 102 0 rsyslogd >[29237.220427] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29237.220434] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29237.220442] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29237.220451] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29237.220459] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29237.220467] [ 2139] 0 2139 97 0 51200 37 0 atd >[29237.220473] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29237.220479] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29237.220486] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29237.220491] [ 2325] 65 2325 6129 0 67584 126 0 nslcd >[29237.220496] [ 2337] 0 2337 449 1 53248 76 0 keepalived >[29237.220500] [ 2338] 0 2338 452 0 53248 96 0 keepalived >[29237.220505] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29237.220510] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29237.220515] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29237.220519] [ 2697] 0 2697 87 0 49152 30 0 log >[29237.220524] [ 2699] 0 2699 103 0 51200 32 0 config >[29237.220528] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29237.220533] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29237.220538] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29237.220544] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29237.220549] [ 5050] 0 5050 380 0 51200 98 0 systemd >[29237.220555] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29237.220562] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29237.220568] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29237.220575] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29237.220583] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29237.220589] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29237.220593] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29237.220598] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29237.220603] [ 8907] 0 8907 410 2 51200 120 0 sendmail >[29237.220608] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29237.220614] [ 8909] 0 8909 3410 4 51200 46 0 ssh >[29237.220619] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29237.220625] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29237.220631] [ 8912] 0 8912 410 0 51200 121 0 sendmail >[29237.220637] [ 8913] 0 8913 166 0 40960 42 0 ps >[29237.220643] [ 8914] 0 8914 3249 0 53248 14 0 head >[29237.220650] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29237.220656] [ 8919] 0 8919 168 0 51200 53 0 auth >[29237.220662] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29237.220669] [ 8932] 0 8932 142 0 38912 24 0 sshd >[29237.220674] [ 8933] 0 8933 3307 0 51200 51 0 crond >[29237.220679] Out of memory: Kill process 2325 (nslcd) score 2 or sacrifice child >[29237.220719] Killed process 2325 (nslcd) total-vm:392256kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29237.240295] oom_reaper: reaped process 2325 (nslcd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29237.911002] systemd-logind invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29237.911027] systemd-logind cpuset=/ mems_allowed=0 >[29237.911048] CPU: 12 PID: 2067 Comm: systemd-logind Not tainted 4.18.7-200.fc28.ppc64 #1 >[29237.911060] Call Trace: >[29237.911074] [c00000000c4fb380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29237.911093] [c00000000c4fb410] [c0000000002fb670] .dump_header+0x70/0x300 >[29237.911108] [c00000000c4fb4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29237.911124] [c00000000c4fb5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29237.911140] [c00000000c4fb660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29237.919953] [c00000000c4fb850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29237.919975] [c00000000c4fb8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29237.919995] [c00000000c4fb980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29237.920247] [c00000000c4fba50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29237.920264] [c00000000c4fbb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29237.920280] [c00000000c4fbb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29237.920296] [c00000000c4fbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29237.920314] [c00000000c4fbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29237.920331] [c00000000c4fbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29237.920361] Mem-Info: >[29237.920379] active_anon:10 inactive_anon:31 isolated_anon:0 > active_file:11 inactive_file:21 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2791 slab_unreclaimable:15804 > mapped:39 shmem:1 pagetables:65 bounce:0 > free:1463 free_pcp:0 free_cma:0 >[29237.920423] Node 0 active_anon:640kB inactive_anon:1984kB active_file:704kB inactive_file:1344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2496kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29237.920448] Node 0 DMA free:93632kB min:94144kB low:117632kB high:141120kB active_anon:2112kB inactive_anon:448kB active_file:0kB inactive_file:2816kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9232kB pagetables:4160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29237.920483] lowmem_reserve[]: 0 0 0 0 >[29237.920500] Node 0 DMA: 424*64kB (UME) 366*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103936kB >[29237.920550] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29237.920563] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29237.920575] 93 total pagecache pages >[29237.920583] 0 pages in swap cache >[29237.920593] Swap cache stats: add 2978188, delete 2978300, find 2526989/4286110 >[29237.920603] Free swap = 1847552kB >[29237.920611] Total swap = 2052032kB >[29237.920620] 32768 pages RAM >[29237.920626] 0 pages HighMem/MovableOnly >[29237.920634] 2684 pages reserved >[29237.920641] 0 pages cma reserved >[29237.920648] 0 pages hwpoisoned >[29237.920657] Unreclaimable slab info: >[29237.920665] Name Used Total >[29237.920832] xfs_efd_item 1017KB 1017KB >[29237.920843] xfs_buf_item 1020KB 1020KB >[29237.920854] xfs_trans 1022KB 1022KB >[29237.920865] xfs_da_state 1020KB 1020KB >[29237.920875] xfs_log_ticket 1023KB 1023KB >[29237.920890] scsi_sense_cache 1024KB 1024KB >[29237.920917] RAWv6 504KB 504KB >[29237.920928] UDPv6 1012KB 1012KB >[29237.920938] tw_sock_TCPv6 127KB 127KB >[29237.920949] TCPv6 1386KB 1386KB >[29237.920961] cfq_io_cq 1023KB 1023KB >[29237.920971] cfq_queue 1023KB 1023KB >[29237.920982] mqueue_inode_cache 64KB 64KB >[29237.920993] kioctx 63KB 63KB >[29237.921006] posix_timers_cache 1022KB 1022KB >[29237.921017] ip4-frags 1023KB 1023KB >[29237.921028] secpath_cache 768KB 768KB >[29237.921040] RAW 896KB 896KB >[29237.921050] UDP 1008KB 1008KB >[29237.921059] tw_sock_TCP 127KB 127KB >[29237.921070] request_sock_TCP 191KB 191KB >[29237.921080] TCP 2040KB 2040KB >[29237.921090] hugetlbfs_inode_cache 127KB 127KB >[29237.921098] eventpoll_pwq 1023KB 1023KB >[29237.921108] inotify_inode_mark 1023KB 1023KB >[29237.921116] request_queue 755KB 755KB >[29237.921126] blkdev_ioc 1023KB 1023KB >[29237.921135] biovec-128 256KB 256KB >[29237.921143] biovec-64 128KB 128KB >[29237.929935] skbuff_fclone_cache 1024KB 1024KB >[29237.929958] skbuff_head_cache 1600KB 1600KB >[29237.929969] file_lock_cache 1021KB 1021KB >[29237.929979] net_namespace 510KB 510KB >[29237.929989] shmem_inode_cache 1087KB 1087KB >[29237.930001] dtl 10040KB 11520KB >[29237.930011] taskstats 828KB 828KB >[29237.930022] proc_dir_entry 640KB 640KB >[29237.930033] pde_opener 1023KB 1023KB >[29237.930044] seq_file 1024KB 1024KB >[29237.930056] sigqueue 1022KB 1022KB >[29237.930067] kernfs_node_cache 5302KB 5302KB >[29237.930077] mnt_cache 2996KB 2996KB >[29237.930087] names_cache 4096KB 4096KB >[29237.930099] ebitmap_node 1024KB 1024KB >[29237.930110] avtab_node 1023KB 1023KB >[29237.930121] avc_xperms_data 1024KB 1024KB >[29237.930132] selinux_file_security 1024KB 1024KB >[29237.930143] selinux_inode_security 1855KB 1855KB >[29237.930155] nsproxy 1023KB 1023KB >[29237.930166] vm_area_struct 1660KB 1660KB >[29237.930176] mm_struct 8056KB 8056KB >[29237.930186] files_cache 1211KB 1211KB >[29237.930197] signal_cache 1472KB 1472KB >[29237.930208] sighand_cache 2932KB 2932KB >[29237.930219] task_struct 12210KB 12210KB >[29237.930230] cred_jar 4800KB 4800KB >[29237.930241] thread_stack 17408KB 17408KB >[29237.930252] anon_vma 1086KB 1086KB >[29237.930263] pid 1280KB 1280KB >[29237.930274] numa_policy 383KB 383KB >[29237.930286] trace_event_file 255KB 255KB >[29237.930296] ftrace_event_field 447KB 447KB >[29237.930307] pool_workqueue 1024KB 1024KB >[29237.930317] task_group 1020KB 1020KB >[29237.930326] pgtable-2^11 9216KB 9216KB >[29237.930336] pgtable-2^8 4096KB 4096KB >[29237.930349] dma-kmalloc-512 128KB 128KB >[29237.930362] kmalloc-131072 35328KB 35328KB >[29237.930371] kmalloc-65536 9216KB 9216KB >[29237.930381] kmalloc-32768 8192KB 8192KB >[29237.930391] kmalloc-16384 18432KB 20480KB >[29237.930401] kmalloc-8192 8704KB 8704KB >[29237.930411] kmalloc-4096 8192KB 8192KB >[29237.930420] kmalloc-2048 4352KB 4352KB >[29237.930431] kmalloc-1024 19331KB 19456KB >[29237.930443] kmalloc-512 10176KB 11072KB >[29237.930452] kmalloc-256 1408KB 1408KB >[29237.930462] kmalloc-192 1790KB 1790KB >[29237.930471] kmalloc-128 1536KB 1536KB >[29237.930481] kmalloc-96 1470KB 1470KB >[29237.930490] kmalloc-64 1792KB 1792KB >[29237.930499] kmalloc-32 12992KB 12992KB >[29237.930508] kmalloc-16 1024KB 1024KB >[29237.930517] kmalloc-8 1024KB 1024KB >[29237.930527] kmem_cache_node 1024KB 1024KB >[29237.930537] kmem_cache 3315KB 3315KB >[29237.930545] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29237.930575] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29237.930591] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29237.930606] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29237.930620] [ 2067] 0 2067 328 0 51200 75 0 systemd-logind >[29237.930634] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29237.930648] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29237.930662] [ 2089] 0 2089 1288 0 51200 43 0 irqbalance >[29237.930677] [ 2094] 0 2094 10692 0 122880 115 0 rsyslogd >[29237.930691] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29237.930704] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29237.930718] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29237.930731] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29237.930745] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29237.930759] [ 2139] 0 2139 97 0 51200 37 0 atd >[29237.930772] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29237.930786] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29237.930799] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29237.930815] [ 2337] 0 2337 449 1 53248 76 0 keepalived >[29237.930829] [ 2338] 0 2338 452 0 53248 96 0 keepalived >[29237.930842] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29237.930858] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29237.930871] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29237.930885] [ 2697] 0 2697 87 0 49152 30 0 log >[29237.930898] [ 2699] 0 2699 103 0 51200 32 0 config >[29237.930911] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29237.930925] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29237.930939] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29237.930954] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29237.930968] [ 5050] 0 5050 380 0 51200 103 0 systemd >[29237.930981] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29237.930995] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29237.931009] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29237.931024] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29237.931040] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29237.931054] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29237.931068] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29237.931082] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29237.931096] [ 8907] 0 8907 410 0 51200 131 0 sendmail >[29237.931110] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29237.931124] [ 8909] 0 8909 3409 1 51200 48 0 ssh >[29237.931138] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29237.931152] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29237.931165] [ 8912] 0 8912 410 0 51200 129 0 sendmail >[29237.939954] [ 8913] 0 8913 166 0 40960 37 0 ps >[29237.939973] [ 8914] 0 8914 3249 0 53248 14 0 head >[29237.939992] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29237.940009] [ 8919] 0 8919 168 0 51200 38 0 auth >[29237.940026] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29237.940045] [ 8932] 0 8932 164 0 38912 40 0 sshd >[29237.940063] [ 8933] 0 8933 3307 0 51200 41 0 crond >[29237.940078] Out of memory: Kill process 8907 (sendmail) score 2 or sacrifice child >[29237.940099] Killed process 8907 (sendmail) total-vm:26240kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29237.960188] oom_reaper: reaped process 8907 (sendmail), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29237.990568] auth invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29237.990585] auth cpuset=/ mems_allowed=0 >[29237.990604] CPU: 4 PID: 8919 Comm: auth Not tainted 4.18.7-200.fc28.ppc64 #1 >[29237.990616] Call Trace: >[29237.990629] [c00000003801f1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29237.990649] [c00000003801f230] [c0000000002fb670] .dump_header+0x70/0x300 >[29237.990668] [c00000003801f300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29237.990686] [c00000003801f3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29237.990705] [c00000003801f480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29237.990724] [c00000003801f670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29237.990743] [c00000003801f730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29237.990764] [c00000003801f7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29237.990783] [c00000003801f880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29237.990801] [c00000003801f990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29237.990821] [c00000003801fac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29237.990839] [c00000003801fb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29237.990857] [c00000003801fcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29237.990876] [c00000003801fd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29237.990896] [c00000003801fe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29237.990910] Mem-Info: >[29237.990928] active_anon:10 inactive_anon:31 isolated_anon:0 > active_file:11 inactive_file:21 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2791 slab_unreclaimable:15804 > mapped:39 shmem:1 pagetables:65 bounce:0 > free:1463 free_pcp:0 free_cma:0 >[29237.990981] Node 0 active_anon:640kB inactive_anon:1984kB active_file:704kB inactive_file:1344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2496kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29237.991011] Node 0 DMA free:93632kB min:94144kB low:117632kB high:141120kB active_anon:2112kB inactive_anon:448kB active_file:0kB inactive_file:2816kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9232kB pagetables:4160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29237.991053] lowmem_reserve[]: 0 0 0 0 >[29237.991072] Node 0 DMA: 423*64kB (UME) 366*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103872kB >[29237.991130] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29237.991146] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29237.991159] 93 total pagecache pages >[29237.991168] 0 pages in swap cache >[29237.999955] Swap cache stats: add 2978190, delete 2978302, find 2526990/4286114 >[29237.999969] Free swap = 1847552kB >[29237.999978] Total swap = 2052032kB >[29237.999988] 32768 pages RAM >[29237.999996] 0 pages HighMem/MovableOnly >[29238.000005] 2684 pages reserved >[29238.000014] 0 pages cma reserved >[29238.000022] 0 pages hwpoisoned >[29238.000031] Unreclaimable slab info: >[29238.000039] Name Used Total >[29238.000142] xfs_efd_item 1017KB 1017KB >[29238.000155] xfs_buf_item 1020KB 1020KB >[29238.000167] xfs_trans 1022KB 1022KB >[29238.000180] xfs_da_state 1020KB 1020KB >[29238.000192] xfs_log_ticket 1023KB 1023KB >[29238.000208] scsi_sense_cache 1024KB 1024KB >[29238.000231] RAWv6 504KB 504KB >[29238.000244] UDPv6 1012KB 1012KB >[29238.000256] tw_sock_TCPv6 127KB 127KB >[29238.000269] TCPv6 1386KB 1386KB >[29238.000283] cfq_io_cq 1023KB 1023KB >[29238.000295] cfq_queue 1023KB 1023KB >[29238.000308] mqueue_inode_cache 64KB 64KB >[29238.000321] kioctx 63KB 63KB >[29238.000335] posix_timers_cache 1022KB 1022KB >[29238.000347] ip4-frags 1023KB 1023KB >[29238.000359] secpath_cache 768KB 768KB >[29238.000372] RAW 896KB 896KB >[29238.000385] UDP 1008KB 1008KB >[29238.000397] tw_sock_TCP 127KB 127KB >[29238.000409] request_sock_TCP 191KB 191KB >[29238.000421] TCP 2040KB 2040KB >[29238.000434] hugetlbfs_inode_cache 127KB 127KB >[29238.000445] eventpoll_pwq 1023KB 1023KB >[29238.000458] inotify_inode_mark 1023KB 1023KB >[29238.000471] request_queue 755KB 755KB >[29238.000485] blkdev_ioc 1023KB 1023KB >[29238.000497] biovec-128 256KB 256KB >[29238.000509] biovec-64 128KB 128KB >[29238.000522] skbuff_fclone_cache 1024KB 1024KB >[29238.000534] skbuff_head_cache 1600KB 1600KB >[29238.000547] file_lock_cache 1021KB 1021KB >[29238.000560] net_namespace 510KB 510KB >[29238.000572] shmem_inode_cache 1087KB 1087KB >[29238.000584] dtl 10040KB 11520KB >[29238.000596] taskstats 828KB 828KB >[29238.000608] proc_dir_entry 640KB 640KB >[29238.000621] pde_opener 1023KB 1023KB >[29238.000633] seq_file 1024KB 1024KB >[29238.000646] sigqueue 1022KB 1022KB >[29238.000658] kernfs_node_cache 5302KB 5302KB >[29238.000671] mnt_cache 2996KB 2996KB >[29238.000683] names_cache 4096KB 4096KB >[29238.000694] ebitmap_node 1024KB 1024KB >[29238.000706] avtab_node 1023KB 1023KB >[29238.000717] avc_xperms_data 1024KB 1024KB >[29238.000730] selinux_file_security 1024KB 1024KB >[29238.000742] selinux_inode_security 1855KB 1855KB >[29238.000756] nsproxy 1023KB 1023KB >[29238.000768] vm_area_struct 1660KB 1660KB >[29238.000781] mm_struct 8056KB 8056KB >[29238.000793] files_cache 1211KB 1211KB >[29238.000805] signal_cache 1472KB 1472KB >[29238.000817] sighand_cache 2932KB 2932KB >[29238.000830] task_struct 12210KB 12210KB >[29238.000841] cred_jar 4800KB 4800KB >[29238.000853] thread_stack 17408KB 17408KB >[29238.000864] anon_vma 1086KB 1086KB >[29238.000876] pid 1280KB 1280KB >[29238.000888] numa_policy 383KB 383KB >[29238.000900] trace_event_file 255KB 255KB >[29238.000912] ftrace_event_field 447KB 447KB >[29238.000924] pool_workqueue 1024KB 1024KB >[29238.000936] task_group 1020KB 1020KB >[29238.000948] pgtable-2^11 9216KB 9216KB >[29238.000960] pgtable-2^8 4096KB 4096KB >[29238.000976] dma-kmalloc-512 128KB 128KB >[29238.000994] kmalloc-131072 35328KB 35328KB >[29238.001006] kmalloc-65536 9216KB 9216KB >[29238.001019] kmalloc-32768 8192KB 8192KB >[29238.001031] kmalloc-16384 18432KB 20480KB >[29238.001043] kmalloc-8192 8704KB 8704KB >[29238.001055] kmalloc-4096 8192KB 8192KB >[29238.001067] kmalloc-2048 4352KB 4352KB >[29238.001081] kmalloc-1024 19331KB 19456KB >[29238.001094] kmalloc-512 10176KB 11072KB >[29238.001106] kmalloc-256 1408KB 1408KB >[29238.001118] kmalloc-192 1790KB 1790KB >[29238.001130] kmalloc-128 1536KB 1536KB >[29238.001142] kmalloc-96 1470KB 1470KB >[29238.009954] kmalloc-64 1792KB 1792KB >[29238.009968] kmalloc-32 12992KB 12992KB >[29238.009981] kmalloc-16 1024KB 1024KB >[29238.009993] kmalloc-8 1024KB 1024KB >[29238.010005] kmem_cache_node 1024KB 1024KB >[29238.010017] kmem_cache 3315KB 3315KB >[29238.010028] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29238.010053] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29238.010072] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29238.010089] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29238.010106] [ 2067] 0 2067 328 0 51200 75 0 systemd-logind >[29238.010122] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29238.010139] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29238.010157] [ 2089] 0 2089 1288 0 51200 43 0 irqbalance >[29238.010175] [ 2094] 0 2094 10692 0 122880 115 0 rsyslogd >[29238.010191] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29238.010208] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29238.010225] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29238.010242] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29238.010260] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29238.010277] [ 2139] 0 2139 97 0 51200 37 0 atd >[29238.010293] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29238.010310] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29238.010327] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29238.010345] [ 2337] 0 2337 449 1 53248 76 0 keepalived >[29238.010364] [ 2338] 0 2338 452 0 53248 96 0 keepalived >[29238.010382] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29238.010401] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29238.010418] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29238.010436] [ 2697] 0 2697 87 0 49152 30 0 log >[29238.010453] [ 2699] 0 2699 103 0 51200 32 0 config >[29238.010470] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29238.010487] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29238.010504] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29238.010522] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29238.010538] [ 5050] 0 5050 380 0 51200 103 0 systemd >[29238.010555] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29238.010572] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29238.010589] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29238.010606] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29238.010624] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29238.010642] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29238.010658] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29238.010677] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29238.010694] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29238.010711] [ 8909] 0 8909 3409 1 51200 48 0 ssh >[29238.010728] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29238.010745] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29238.010762] [ 8912] 0 8912 410 0 51200 129 0 sendmail >[29238.010779] [ 8913] 0 8913 166 0 40960 37 0 ps >[29238.010796] [ 8914] 0 8914 3249 0 53248 14 0 head >[29238.010813] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29238.010830] [ 8919] 0 8919 168 0 51200 38 0 auth >[29238.010849] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29238.010868] [ 8932] 0 8932 164 0 38912 40 0 sshd >[29238.010886] [ 8933] 0 8933 3307 0 51200 41 0 crond >[29238.010901] Out of memory: Kill process 8912 (sendmail) score 2 or sacrifice child >[29238.010921] Killed process 8912 (sendmail) total-vm:26240kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29238.030220] oom_reaper: reaped process 8912 (sendmail), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29238.040023] systemd-journal invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29238.040040] systemd-journal cpuset=/ mems_allowed=0 >[29238.040058] CPU: 0 PID: 716 Comm: systemd-journal Not tainted 4.18.7-200.fc28.ppc64 #1 >[29238.040069] Call Trace: >[29238.040081] [c0000000602731a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29238.040100] [c000000060273230] [c0000000002fb670] .dump_header+0x70/0x300 >[29238.040116] [c000000060273300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29238.040132] [c0000000602733d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29238.040150] [c000000060273480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29238.040168] [c000000060273670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29238.040187] [c000000060273730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29238.040205] [c0000000602737f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29238.040224] [c000000060273880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29238.040243] [c000000060273990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29238.040262] [c000000060273ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29238.040279] [c000000060273b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29238.040298] [c000000060273cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29238.040315] [c000000060273d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29238.040332] [c000000060273e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29238.040346] Mem-Info: >[29238.040364] active_anon:10 inactive_anon:31 isolated_anon:0 > active_file:11 inactive_file:21 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2791 slab_unreclaimable:15804 > mapped:39 shmem:1 pagetables:65 bounce:0 > free:1463 free_pcp:0 free_cma:0 >[29238.040412] Node 0 active_anon:640kB inactive_anon:1984kB active_file:704kB inactive_file:1344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2496kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29238.040439] Node 0 DMA free:93632kB min:94144kB low:117632kB high:141120kB active_anon:2112kB inactive_anon:448kB active_file:0kB inactive_file:2816kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9232kB pagetables:4160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29238.040478] lowmem_reserve[]: 0 0 0 0 >[29238.040496] Node 0 DMA: 422*64kB (UME) 366*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103808kB >[29238.040549] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29238.040565] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29238.040578] 93 total pagecache pages >[29238.040587] 0 pages in swap cache >[29238.040597] Swap cache stats: add 2978192, delete 2978304, find 2526991/4286116 >[29238.040609] Free swap = 1851648kB >[29238.040617] Total swap = 2052032kB >[29238.040626] 32768 pages RAM >[29238.040633] 0 pages HighMem/MovableOnly >[29238.040641] 2684 pages reserved >[29238.040648] 0 pages cma reserved >[29238.040656] 0 pages hwpoisoned >[29238.040664] Unreclaimable slab info: >[29238.040673] Name Used Total >[29238.040776] xfs_efd_item 1017KB 1017KB >[29238.040789] xfs_buf_item 1020KB 1020KB >[29238.040800] xfs_trans 1022KB 1022KB >[29238.040812] xfs_da_state 1020KB 1020KB >[29238.040823] xfs_log_ticket 1023KB 1023KB >[29238.040837] scsi_sense_cache 1024KB 1024KB >[29238.040858] RAWv6 504KB 504KB >[29238.040869] UDPv6 1012KB 1012KB >[29238.040880] tw_sock_TCPv6 127KB 127KB >[29238.040892] TCPv6 1386KB 1386KB >[29238.040904] cfq_io_cq 1023KB 1023KB >[29238.040915] cfq_queue 1023KB 1023KB >[29238.040927] mqueue_inode_cache 64KB 64KB >[29238.040939] kioctx 63KB 63KB >[29238.040952] posix_timers_cache 1022KB 1022KB >[29238.040964] ip4-frags 1023KB 1023KB >[29238.040975] secpath_cache 768KB 768KB >[29238.040986] RAW 896KB 896KB >[29238.040997] UDP 1008KB 1008KB >[29238.041008] tw_sock_TCP 127KB 127KB >[29238.041019] request_sock_TCP 191KB 191KB >[29238.041030] TCP 2040KB 2040KB >[29238.041042] hugetlbfs_inode_cache 127KB 127KB >[29238.041054] eventpoll_pwq 1023KB 1023KB >[29238.041065] inotify_inode_mark 1023KB 1023KB >[29238.041076] request_queue 755KB 755KB >[29238.041088] blkdev_ioc 1023KB 1023KB >[29238.041100] biovec-128 256KB 256KB >[29238.041112] biovec-64 128KB 128KB >[29238.041123] skbuff_fclone_cache 1024KB 1024KB >[29238.041134] skbuff_head_cache 1600KB 1600KB >[29238.041145] file_lock_cache 1021KB 1021KB >[29238.041157] net_namespace 510KB 510KB >[29238.041168] shmem_inode_cache 1087KB 1087KB >[29238.049955] dtl 10040KB 11520KB >[29238.049968] taskstats 828KB 828KB >[29238.049979] proc_dir_entry 640KB 640KB >[29238.049991] pde_opener 1023KB 1023KB >[29238.050002] seq_file 1024KB 1024KB >[29238.050013] sigqueue 1022KB 1022KB >[29238.050024] kernfs_node_cache 5302KB 5302KB >[29238.050035] mnt_cache 2996KB 2996KB >[29238.050046] names_cache 4096KB 4096KB >[29238.050058] ebitmap_node 1024KB 1024KB >[29238.050069] avtab_node 1023KB 1023KB >[29238.050080] avc_xperms_data 1024KB 1024KB >[29238.050092] selinux_file_security 1024KB 1024KB >[29238.050103] selinux_inode_security 1855KB 1855KB >[29238.050116] nsproxy 1023KB 1023KB >[29238.050126] vm_area_struct 1660KB 1660KB >[29238.050136] mm_struct 8056KB 8056KB >[29238.050147] files_cache 1211KB 1211KB >[29238.050159] signal_cache 1472KB 1472KB >[29238.050170] sighand_cache 2932KB 2932KB >[29238.050181] task_struct 12210KB 12210KB >[29238.050192] cred_jar 4800KB 4800KB >[29238.050204] thread_stack 17408KB 17408KB >[29238.050215] anon_vma 1086KB 1086KB >[29238.050227] pid 1280KB 1280KB >[29238.050238] numa_policy 383KB 383KB >[29238.050248] trace_event_file 255KB 255KB >[29238.050260] ftrace_event_field 447KB 447KB >[29238.050271] pool_workqueue 1024KB 1024KB >[29238.050282] task_group 1020KB 1020KB >[29238.050293] pgtable-2^11 9216KB 9216KB >[29238.050304] pgtable-2^8 4096KB 4096KB >[29238.050320] dma-kmalloc-512 128KB 128KB >[29238.050335] kmalloc-131072 35328KB 35328KB >[29238.050346] kmalloc-65536 9216KB 9216KB >[29238.050357] kmalloc-32768 8192KB 8192KB >[29238.050368] kmalloc-16384 18432KB 20480KB >[29238.050379] kmalloc-8192 8704KB 8704KB >[29238.050390] kmalloc-4096 8192KB 8192KB >[29238.050401] kmalloc-2048 4352KB 4352KB >[29238.050412] kmalloc-1024 19331KB 19456KB >[29238.050424] kmalloc-512 10176KB 11072KB >[29238.050436] kmalloc-256 1408KB 1408KB >[29238.050447] kmalloc-192 1790KB 1790KB >[29238.050459] kmalloc-128 1536KB 1536KB >[29238.050470] kmalloc-96 1470KB 1470KB >[29238.050481] kmalloc-64 1792KB 1792KB >[29238.050492] kmalloc-32 12992KB 12992KB >[29238.050503] kmalloc-16 1024KB 1024KB >[29238.050514] kmalloc-8 1024KB 1024KB >[29238.050526] kmem_cache_node 1024KB 1024KB >[29238.050537] kmem_cache 3315KB 3315KB >[29238.050547] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29238.050582] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29238.050601] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29238.050619] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29238.050635] [ 2067] 0 2067 328 0 51200 75 0 systemd-logind >[29238.050653] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29238.050669] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29238.050686] [ 2089] 0 2089 1288 0 51200 43 0 irqbalance >[29238.050703] [ 2094] 0 2094 10692 0 122880 115 0 rsyslogd >[29238.050719] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29238.050735] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29238.050752] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29238.050768] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29238.050783] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29238.050800] [ 2139] 0 2139 97 0 51200 37 0 atd >[29238.050816] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29238.050832] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29238.050849] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29238.050868] [ 2337] 0 2337 449 1 53248 76 0 keepalived >[29238.050885] [ 2338] 0 2338 452 0 53248 96 0 keepalived >[29238.050901] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29238.050920] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29238.050936] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29238.050952] [ 2697] 0 2697 87 0 49152 30 0 log >[29238.050968] [ 2699] 0 2699 103 0 51200 32 0 config >[29238.050984] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29238.051000] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29238.051017] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29238.051034] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29238.051050] [ 5050] 0 5050 380 0 51200 103 0 systemd >[29238.051066] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29238.051082] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29238.051098] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29238.051115] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29238.051135] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29238.051151] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29238.051168] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29238.059956] [ 8900] 0 8900 411 0 51200 123 0 sendmail >[29238.059974] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29238.059991] [ 8909] 0 8909 3409 1 51200 48 0 ssh >[29238.060008] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29238.060025] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29238.060042] [ 8913] 0 8913 166 0 40960 37 0 ps >[29238.060060] [ 8914] 0 8914 3249 0 53248 14 0 head >[29238.060078] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29238.060097] [ 8919] 0 8919 168 0 51200 38 0 auth >[29238.060115] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29238.060134] [ 8932] 0 8932 164 0 38912 40 0 sshd >[29238.060152] [ 8933] 0 8933 3307 0 51200 41 0 crond >[29238.060166] Out of memory: Kill process 8900 (sendmail) score 1 or sacrifice child >[29238.060195] Killed process 8900 (sendmail) total-vm:26304kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29238.080162] oom_reaper: reaped process 8900 (sendmail), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29238.111089] keepalived invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29238.111107] keepalived cpuset=/ mems_allowed=0 >[29238.111126] CPU: 15 PID: 2337 Comm: keepalived Not tainted 4.18.7-200.fc28.ppc64 #1 >[29238.111139] Call Trace: >[29238.111151] [c00000000c1e2c00] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29238.111171] [c00000000c1e2c90] [c0000000002fb670] .dump_header+0x70/0x300 >[29238.119956] [c00000000c1e2d60] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29238.119975] [c00000000c1e2e30] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29238.119995] [c00000000c1e2ee0] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29238.120016] [c00000000c1e30d0] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29238.120035] [c00000000c1e3190] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29238.120054] [c00000000c1e3250] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29238.120074] [c00000000c1e32e0] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29238.120094] [c00000000c1e33f0] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29238.120112] [c00000000c1e3520] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29238.120130] [c00000000c1e35f0] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29238.120149] [c00000000c1e3720] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29238.120168] [c00000000c1e37b0] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29238.120188] [c00000000c1e3890] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29238.120212] --- interrupt: 301 at .__copy_tofrom_user_power7+0x218/0x7ac > LR = .core_sys_select+0x2cc/0x430 >[29238.120234] [c00000000c1e3b80] [c000000000413418] .core_sys_select+0x2b8/0x430 (unreliable) >[29238.120255] [c00000000c1e3d70] [c000000000413668] .__se_sys_select+0xd8/0x140 >[29238.120274] [c00000000c1e3e30] [c00000000000bb64] system_call+0x5c/0x70 >[29238.120303] Mem-Info: >[29238.120323] active_anon:10 inactive_anon:31 isolated_anon:0 > active_file:11 inactive_file:21 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2791 slab_unreclaimable:15804 > mapped:39 shmem:1 pagetables:65 bounce:0 > free:1463 free_pcp:0 free_cma:0 >[29238.120374] Node 0 active_anon:640kB inactive_anon:1984kB active_file:704kB inactive_file:1344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2496kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29238.120404] Node 0 DMA free:93632kB min:94144kB low:117632kB high:141120kB active_anon:2112kB inactive_anon:448kB active_file:0kB inactive_file:1344kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9232kB pagetables:4160kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29238.120447] lowmem_reserve[]: 0 0 0 0 >[29238.120466] Node 0 DMA: 420*64kB (UME) 366*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103680kB >[29238.120523] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29238.120539] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29238.120552] 93 total pagecache pages >[29238.120562] 0 pages in swap cache >[29238.120572] Swap cache stats: add 2978193, delete 2978305, find 2526992/4286118 >[29238.120585] Free swap = 1851648kB >[29238.120594] Total swap = 2052032kB >[29238.120603] 32768 pages RAM >[29238.120611] 0 pages HighMem/MovableOnly >[29238.120619] 2684 pages reserved >[29238.120627] 0 pages cma reserved >[29238.120636] 0 pages hwpoisoned >[29238.120644] Unreclaimable slab info: >[29238.120652] Name Used Total >[29238.120756] xfs_efd_item 1017KB 1017KB >[29238.120769] xfs_buf_item 1020KB 1020KB >[29238.120781] xfs_trans 1022KB 1022KB >[29238.120793] xfs_da_state 1020KB 1020KB >[29238.120805] xfs_log_ticket 1023KB 1023KB >[29238.120820] scsi_sense_cache 1024KB 1024KB >[29238.120842] RAWv6 504KB 504KB >[29238.120855] UDPv6 1012KB 1012KB >[29238.120868] tw_sock_TCPv6 127KB 127KB >[29238.120880] TCPv6 1386KB 1386KB >[29238.120893] cfq_io_cq 1023KB 1023KB >[29238.120905] cfq_queue 1023KB 1023KB >[29238.120917] mqueue_inode_cache 64KB 64KB >[29238.120930] kioctx 63KB 63KB >[29238.120943] posix_timers_cache 1022KB 1022KB >[29238.120955] ip4-frags 1023KB 1023KB >[29238.120967] secpath_cache 768KB 768KB >[29238.120979] RAW 896KB 896KB >[29238.120990] UDP 1008KB 1008KB >[29238.121002] tw_sock_TCP 127KB 127KB >[29238.121014] request_sock_TCP 191KB 191KB >[29238.121027] TCP 2040KB 2040KB >[29238.121039] hugetlbfs_inode_cache 127KB 127KB >[29238.121051] eventpoll_pwq 1023KB 1023KB >[29238.121064] inotify_inode_mark 1023KB 1023KB >[29238.121076] request_queue 755KB 755KB >[29238.121089] blkdev_ioc 1023KB 1023KB >[29238.121102] biovec-128 256KB 256KB >[29238.121113] biovec-64 128KB 128KB >[29238.121126] skbuff_fclone_cache 1024KB 1024KB >[29238.121138] skbuff_head_cache 1600KB 1600KB >[29238.129935] file_lock_cache 1021KB 1021KB >[29238.129962] net_namespace 510KB 510KB >[29238.129974] shmem_inode_cache 1087KB 1087KB >[29238.129986] dtl 10040KB 11520KB >[29238.129999] taskstats 828KB 828KB >[29238.130011] proc_dir_entry 640KB 640KB >[29238.130025] pde_opener 1023KB 1023KB >[29238.130037] seq_file 1024KB 1024KB >[29238.130048] sigqueue 1022KB 1022KB >[29238.130060] kernfs_node_cache 5302KB 5302KB >[29238.130071] mnt_cache 2996KB 2996KB >[29238.130084] names_cache 4096KB 4096KB >[29238.130097] ebitmap_node 1024KB 1024KB >[29238.130109] avtab_node 1023KB 1023KB >[29238.130121] avc_xperms_data 1024KB 1024KB >[29238.130133] selinux_file_security 1024KB 1024KB >[29238.130145] selinux_inode_security 1855KB 1855KB >[29238.130160] nsproxy 1023KB 1023KB >[29238.130171] vm_area_struct 1660KB 1660KB >[29238.130183] mm_struct 8056KB 8056KB >[29238.130193] files_cache 1211KB 1211KB >[29238.130205] signal_cache 1472KB 1472KB >[29238.130213] sighand_cache 2932KB 2932KB >[29238.130221] task_struct 12210KB 12210KB >[29238.130229] cred_jar 4800KB 4800KB >[29238.130237] thread_stack 17408KB 17408KB >[29238.130246] anon_vma 1086KB 1086KB >[29238.130253] pid 1280KB 1280KB >[29238.130262] numa_policy 383KB 383KB >[29238.130270] trace_event_file 255KB 255KB >[29238.130278] ftrace_event_field 447KB 447KB >[29238.130287] pool_workqueue 1024KB 1024KB >[29238.130295] task_group 1020KB 1020KB >[29238.130303] pgtable-2^11 9216KB 9216KB >[29238.130311] pgtable-2^8 4096KB 4096KB >[29238.130322] dma-kmalloc-512 128KB 128KB >[29238.130333] kmalloc-131072 35328KB 35328KB >[29238.130341] kmalloc-65536 9216KB 9216KB >[29238.130349] kmalloc-32768 8192KB 8192KB >[29238.130358] kmalloc-16384 18432KB 20480KB >[29238.130366] kmalloc-8192 8704KB 8704KB >[29238.130373] kmalloc-4096 8192KB 8192KB >[29238.130381] kmalloc-2048 4352KB 4352KB >[29238.130390] kmalloc-1024 19331KB 19456KB >[29238.130398] kmalloc-512 10176KB 11072KB >[29238.130407] kmalloc-256 1408KB 1408KB >[29238.130414] kmalloc-192 1790KB 1790KB >[29238.130422] kmalloc-128 1536KB 1536KB >[29238.130430] kmalloc-96 1470KB 1470KB >[29238.130438] kmalloc-64 1792KB 1792KB >[29238.130445] kmalloc-32 12992KB 12992KB >[29238.130453] kmalloc-16 1024KB 1024KB >[29238.130461] kmalloc-8 1024KB 1024KB >[29238.130469] kmem_cache_node 1024KB 1024KB >[29238.130477] kmem_cache 3315KB 3315KB >[29238.130484] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29238.130503] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29238.130516] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29238.130528] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29238.130540] [ 2067] 0 2067 328 0 51200 75 0 systemd-logind >[29238.130552] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29238.130563] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29238.130576] [ 2089] 0 2089 1288 0 51200 43 0 irqbalance >[29238.130588] [ 2094] 0 2094 10692 0 122880 115 0 rsyslogd >[29238.130599] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29238.130611] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29238.130622] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29238.130633] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29238.130644] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29238.130655] [ 2139] 0 2139 97 0 51200 37 0 atd >[29238.130667] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29238.130678] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29238.130690] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29238.130702] [ 2337] 0 2337 449 1 53248 76 0 keepalived >[29238.130714] [ 2338] 0 2338 452 0 53248 96 0 keepalived >[29238.130726] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29238.130739] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29238.130750] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29238.130761] [ 2697] 0 2697 87 0 49152 30 0 log >[29238.130772] [ 2699] 0 2699 103 0 51200 32 0 config >[29238.130783] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29238.130795] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29238.130807] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29238.130818] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29238.130829] [ 5050] 0 5050 380 0 51200 103 0 systemd >[29238.130841] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29238.130851] [ 5058] 0 5058 3308 0 55296 45 0 bash >[29238.130863] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29238.130875] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29238.130888] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29238.130899] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29238.130911] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29238.130923] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29238.130934] [ 8909] 0 8909 3409 1 51200 48 0 ssh >[29238.130945] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29238.130956] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29238.130968] [ 8913] 0 8913 166 0 40960 37 0 ps >[29238.130979] [ 8914] 0 8914 3249 0 53248 14 0 head >[29238.130990] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29238.131002] [ 8919] 0 8919 168 0 51200 38 0 auth >[29238.131014] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29238.131026] [ 8932] 0 8932 164 0 38912 40 0 sshd >[29238.131038] [ 8933] 0 8933 3307 0 51200 41 0 crond >[29238.131048] Out of memory: Kill process 5057 (sshd) score 1 or sacrifice child >[29238.131069] Killed process 5058 (bash) total-vm:211712kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29238.150642] crond invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29238.150659] crond cpuset=/ mems_allowed=0 >[29238.150678] CPU: 2 PID: 8933 Comm: crond Not tainted 4.18.7-200.fc28.ppc64 #1 >[29238.150691] Call Trace: >[29238.150704] [c0000000099471a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29238.150724] [c000000009947230] [c0000000002fb670] .dump_header+0x70/0x300 >[29238.150742] [c000000009947300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29238.150760] [c0000000099473d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29238.150778] [c000000009947480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29238.150798] [c000000009947670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29238.150818] [c000000009947730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29238.150837] [c0000000099477f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29238.150857] [c000000009947880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29238.150876] [c000000009947990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29238.150895] [c000000009947ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29238.150912] [c000000009947b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29238.150931] [c000000009947cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29238.150949] [c000000009947d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29238.150968] [c000000009947e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29238.150982] Mem-Info: >[29238.151000] active_anon:10 inactive_anon:31 isolated_anon:0 > active_file:11 inactive_file:21 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2791 slab_unreclaimable:15804 > mapped:39 shmem:1 pagetables:65 bounce:0 > free:1463 free_pcp:1 free_cma:0 >[29238.151052] Node 0 active_anon:640kB inactive_anon:1984kB active_file:704kB inactive_file:1344kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2496kB dirty:0kB writeback:0kB shmem:64kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29238.151083] Node 0 DMA free:93632kB min:94144kB low:117632kB high:141120kB active_anon:2112kB inactive_anon:448kB active_file:0kB inactive_file:1344kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9232kB pagetables:4160kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29238.151123] lowmem_reserve[]: 0 0 0 0 >[29238.151143] Node 0 DMA: 419*64kB (UME) 366*128kB (UME) 103*256kB (UME) 5*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103616kB >[29238.159998] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29238.160014] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29238.160027] 93 total pagecache pages >[29238.160036] 0 pages in swap cache >[29238.160061] Swap cache stats: add 2978193, delete 2978305, find 2526992/4286120 >[29238.160090] Free swap = 1855744kB >[29238.160100] Total swap = 2052032kB >[29238.160123] 32768 pages RAM >[29238.160131] 0 pages HighMem/MovableOnly >[29238.160153] 2684 pages reserved >[29238.160162] 0 pages cma reserved >[29238.160182] 0 pages hwpoisoned >[29238.160191] Unreclaimable slab info: >[29238.160200] Name Used Total >[29238.160378] xfs_efd_item 1017KB 1017KB >[29238.160391] xfs_buf_item 1020KB 1020KB >[29238.160403] xfs_trans 1022KB 1022KB >[29238.160416] xfs_da_state 1020KB 1020KB >[29238.160428] xfs_log_ticket 1023KB 1023KB >[29238.160443] scsi_sense_cache 1024KB 1024KB >[29238.160466] RAWv6 504KB 504KB >[29238.160478] UDPv6 1012KB 1012KB >[29238.160490] tw_sock_TCPv6 127KB 127KB >[29238.160503] TCPv6 1386KB 1386KB >[29238.160517] cfq_io_cq 1023KB 1023KB >[29238.160530] cfq_queue 1023KB 1023KB >[29238.160543] mqueue_inode_cache 64KB 64KB >[29238.160555] kioctx 63KB 63KB >[29238.160569] posix_timers_cache 1022KB 1022KB >[29238.160581] ip4-frags 1023KB 1023KB >[29238.160592] secpath_cache 768KB 768KB >[29238.160606] RAW 896KB 896KB >[29238.160617] UDP 1008KB 1008KB >[29238.160629] tw_sock_TCP 127KB 127KB >[29238.160642] request_sock_TCP 191KB 191KB >[29238.160653] TCP 2040KB 2040KB >[29238.160665] hugetlbfs_inode_cache 127KB 127KB >[29238.160677] eventpoll_pwq 1023KB 1023KB >[29238.160689] inotify_inode_mark 1023KB 1023KB >[29238.160701] request_queue 755KB 755KB >[29238.160714] blkdev_ioc 1023KB 1023KB >[29238.160725] biovec-128 256KB 256KB >[29238.160739] biovec-64 128KB 128KB >[29238.160751] skbuff_fclone_cache 1024KB 1024KB >[29238.160762] skbuff_head_cache 1600KB 1600KB >[29238.160775] file_lock_cache 1021KB 1021KB >[29238.160787] net_namespace 510KB 510KB >[29238.160799] shmem_inode_cache 1087KB 1087KB >[29238.160811] dtl 10040KB 11520KB >[29238.160823] taskstats 828KB 828KB >[29238.160834] proc_dir_entry 640KB 640KB >[29238.160846] pde_opener 1023KB 1023KB >[29238.160859] seq_file 1024KB 1024KB >[29238.160870] sigqueue 1022KB 1022KB >[29238.160882] kernfs_node_cache 5302KB 5302KB >[29238.160895] mnt_cache 2996KB 2996KB >[29238.160908] names_cache 4096KB 4096KB >[29238.160921] ebitmap_node 1024KB 1024KB >[29238.160933] avtab_node 1023KB 1023KB >[29238.160945] avc_xperms_data 1024KB 1024KB >[29238.160958] selinux_file_security 1024KB 1024KB >[29238.160970] selinux_inode_security 1855KB 1855KB >[29238.160983] nsproxy 1023KB 1023KB >[29238.160995] vm_area_struct 1660KB 1660KB >[29238.161007] mm_struct 8056KB 8056KB >[29238.161019] files_cache 1211KB 1211KB >[29238.161031] signal_cache 1472KB 1472KB >[29238.161043] sighand_cache 2932KB 2932KB >[29238.161055] task_struct 12210KB 12210KB >[29238.161067] cred_jar 4800KB 4800KB >[29238.161079] thread_stack 17408KB 17408KB >[29238.161091] anon_vma 1086KB 1086KB >[29238.161104] pid 1280KB 1280KB >[29238.161116] numa_policy 383KB 383KB >[29238.161128] trace_event_file 255KB 255KB >[29238.161139] ftrace_event_field 447KB 447KB >[29238.169948] pool_workqueue 1024KB 1024KB >[29238.169961] task_group 1020KB 1020KB >[29238.169973] pgtable-2^11 9216KB 9216KB >[29238.169985] pgtable-2^8 4096KB 4096KB >[29238.170002] dma-kmalloc-512 128KB 128KB >[29238.170018] kmalloc-131072 35328KB 35328KB >[29238.170030] kmalloc-65536 9216KB 9216KB >[29238.170042] kmalloc-32768 8192KB 8192KB >[29238.170054] kmalloc-16384 18432KB 20480KB >[29238.170066] kmalloc-8192 8704KB 8704KB >[29238.170078] kmalloc-4096 8192KB 8192KB >[29238.170090] kmalloc-2048 4352KB 4352KB >[29238.170102] kmalloc-1024 19331KB 19456KB >[29238.170115] kmalloc-512 10176KB 11072KB >[29238.170127] kmalloc-256 1408KB 1408KB >[29238.170138] kmalloc-192 1790KB 1790KB >[29238.170151] kmalloc-128 1536KB 1536KB >[29238.170163] kmalloc-96 1470KB 1470KB >[29238.170175] kmalloc-64 1792KB 1792KB >[29238.170187] kmalloc-32 12992KB 12992KB >[29238.170199] kmalloc-16 1024KB 1024KB >[29238.170211] kmalloc-8 1024KB 1024KB >[29238.170222] kmem_cache_node 1024KB 1024KB >[29238.170235] kmem_cache 3315KB 3315KB >[29238.170245] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29238.170282] [ 716] 0 716 2668 0 69632 91 0 systemd-journal >[29238.170301] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29238.170320] [ 2033] 0 2033 346 0 49152 62 -1000 auditd >[29238.170338] [ 2067] 0 2067 328 0 51200 75 0 systemd-logind >[29238.170357] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29238.170375] [ 2069] 81 2069 208 0 51200 65 -900 dbus-daemon >[29238.170393] [ 2089] 0 2089 1288 0 51200 43 0 irqbalance >[29238.170411] [ 2094] 0 2094 10692 0 122880 115 0 rsyslogd >[29238.170428] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29238.170446] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29238.170463] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29238.170480] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29238.170498] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29238.170516] [ 2139] 0 2139 97 0 51200 37 0 atd >[29238.170533] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29238.170550] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29238.170567] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29238.170585] [ 2337] 0 2337 449 1 53248 76 0 keepalived >[29238.170604] [ 2338] 0 2338 452 0 53248 96 0 keepalived >[29238.170623] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29238.170642] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29238.170660] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29238.170677] [ 2697] 0 2697 87 0 49152 30 0 log >[29238.170694] [ 2699] 0 2699 103 0 51200 32 0 config >[29238.170712] [ 2711] 0 2711 398 0 51200 113 0 sendmail >[29238.170729] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29238.170747] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29238.170764] [ 5044] 0 5044 405 1 53248 111 0 sshd >[29238.170781] [ 5050] 0 5050 380 0 51200 103 0 systemd >[29238.170799] [ 5057] 0 5057 405 0 53248 121 0 sshd >[29238.170817] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29238.170835] [ 8375] 0 8375 3282 1 51200 32 0 watch >[29238.170855] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29238.170873] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29238.170891] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29238.170911] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29238.170925] [ 8909] 0 8909 3409 1 51200 48 0 ssh >[29238.170938] [ 8910] 0 8910 3282 0 49152 44 0 watch >[29238.170949] [ 8911] 0 8911 3282 0 53248 28 0 sh >[29238.170962] [ 8913] 0 8913 166 0 40960 37 0 ps >[29238.170974] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29238.170987] [ 8919] 0 8919 168 0 51200 38 0 auth >[29238.171001] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29238.171016] [ 8932] 0 8932 164 0 38912 40 0 sshd >[29238.171027] [ 8933] 0 8933 3307 0 51200 41 0 crond >[29238.171035] Out of memory: Kill process 5057 (sshd) score 1 or sacrifice child >[29238.171048] Killed process 5057 (sshd) total-vm:25920kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29238.171120] sshd[5057]: bad frame in setup_rt_frame: 00007ffffc61c650 nip 00007fff8ac40d98 lr 000000013b461940 >[29238.190202] oom_reaper: reaped process 5057 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29241.150588] crond invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29241.150610] crond cpuset=/ mems_allowed=0 >[29241.150625] CPU: 2 PID: 8933 Comm: crond Not tainted 4.18.7-200.fc28.ppc64 #1 >[29241.150633] Call Trace: >[29241.150644] [c000000009947380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29241.150658] [c000000009947410] [c0000000002fb670] .dump_header+0x70/0x300 >[29241.150668] [c0000000099474e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29241.150679] [c0000000099475b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29241.150690] [c000000009947660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29241.150702] [c000000009947850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29241.150714] [c0000000099478f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29241.150725] [c000000009947980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29241.150872] [c000000009947a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29241.150884] [c000000009947b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29241.150895] [c000000009947b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29241.150905] [c000000009947cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29241.150917] [c000000009947d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29241.150929] [c000000009947e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29241.150938] Mem-Info: >[29241.150950] active_anon:18 inactive_anon:18 isolated_anon:0 > active_file:36 inactive_file:44 isolated_file:0 > unevictable:0 dirty:3 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15817 > mapped:79 shmem:0 pagetables:56 bounce:0 > free:1471 free_pcp:0 free_cma:0 >[29241.150981] Node 0 active_anon:1152kB inactive_anon:1152kB active_file:2304kB inactive_file:2816kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:5056kB dirty:192kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29241.150998] Node 0 DMA free:94144kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:3968kB active_file:2304kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9152kB pagetables:3584kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29241.151022] lowmem_reserve[]: 0 0 0 0 >[29241.151033] Node 0 DMA: 431*64kB (UME) 363*128kB (UME) 104*256kB (UME) 6*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103744kB >[29241.151065] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29241.151075] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29241.151083] 86 total pagecache pages >[29241.151088] 0 pages in swap cache >[29241.151094] Swap cache stats: add 2981570, delete 2981683, find 2529754/4291148 >[29241.151101] Free swap = 1876224kB >[29241.151106] Total swap = 2052032kB >[29241.151112] 32768 pages RAM >[29241.151116] 0 pages HighMem/MovableOnly >[29241.151121] 2684 pages reserved >[29241.151126] 0 pages cma reserved >[29241.151132] 0 pages hwpoisoned >[29241.151138] Unreclaimable slab info: >[29241.151143] Name Used Total >[29241.160112] xfs_efd_item 1017KB 1017KB >[29241.160123] xfs_buf_item 1020KB 1020KB >[29241.160132] xfs_trans 1022KB 1022KB >[29241.160139] xfs_da_state 1020KB 1020KB >[29241.160146] xfs_log_ticket 1023KB 1023KB >[29241.160159] scsi_sense_cache 1024KB 1024KB >[29241.160182] RAWv6 504KB 504KB >[29241.160190] UDPv6 1012KB 1012KB >[29241.160197] tw_sock_TCPv6 127KB 127KB >[29241.160204] TCPv6 1386KB 1386KB >[29241.160212] cfq_io_cq 1023KB 1023KB >[29241.160219] cfq_queue 1023KB 1023KB >[29241.160227] mqueue_inode_cache 64KB 64KB >[29241.160235] kioctx 63KB 63KB >[29241.160243] posix_timers_cache 1022KB 1022KB >[29241.160250] ip4-frags 1023KB 1023KB >[29241.160257] secpath_cache 768KB 768KB >[29241.160264] RAW 896KB 896KB >[29241.160271] UDP 1008KB 1008KB >[29241.160278] tw_sock_TCP 127KB 127KB >[29241.160284] request_sock_TCP 191KB 191KB >[29241.160291] TCP 2040KB 2040KB >[29241.160298] hugetlbfs_inode_cache 127KB 127KB >[29241.160306] eventpoll_pwq 1023KB 1023KB >[29241.160313] inotify_inode_mark 1023KB 1023KB >[29241.160319] request_queue 755KB 755KB >[29241.160326] blkdev_ioc 1023KB 1023KB >[29241.160333] biovec-128 256KB 256KB >[29241.160340] biovec-64 128KB 128KB >[29241.160347] skbuff_fclone_cache 1024KB 1024KB >[29241.160354] skbuff_head_cache 1600KB 1600KB >[29241.160361] file_lock_cache 1021KB 1021KB >[29241.160367] net_namespace 510KB 510KB >[29241.160374] shmem_inode_cache 1087KB 1087KB >[29241.160382] dtl 10040KB 11520KB >[29241.160389] taskstats 828KB 828KB >[29241.160396] proc_dir_entry 640KB 640KB >[29241.160403] pde_opener 1023KB 1023KB >[29241.160411] seq_file 1024KB 1024KB >[29241.160418] sigqueue 1022KB 1022KB >[29241.160425] kernfs_node_cache 5302KB 5302KB >[29241.160432] mnt_cache 2996KB 2996KB >[29241.160439] names_cache 4096KB 4096KB >[29241.160446] ebitmap_node 1024KB 1024KB >[29241.160453] avtab_node 1023KB 1023KB >[29241.160460] avc_xperms_data 1024KB 1024KB >[29241.160466] selinux_file_security 1024KB 1024KB >[29241.160473] selinux_inode_security 1855KB 1855KB >[29241.160481] nsproxy 1023KB 1023KB >[29241.160488] vm_area_struct 1660KB 1660KB >[29241.160495] mm_struct 8056KB 8056KB >[29241.160501] files_cache 1211KB 1211KB >[29241.160508] signal_cache 1472KB 1472KB >[29241.160516] sighand_cache 2932KB 2932KB >[29241.160523] task_struct 12210KB 12210KB >[29241.160529] cred_jar 4800KB 4800KB >[29241.160536] thread_stack 17408KB 17408KB >[29241.160543] anon_vma 1086KB 1086KB >[29241.160549] pid 1280KB 1280KB >[29241.160556] numa_policy 383KB 383KB >[29241.160563] trace_event_file 255KB 255KB >[29241.160570] ftrace_event_field 447KB 447KB >[29241.160576] pool_workqueue 1024KB 1024KB >[29241.160583] task_group 1020KB 1020KB >[29241.160590] pgtable-2^11 9216KB 9216KB >[29241.160597] pgtable-2^8 4096KB 4096KB >[29241.160607] dma-kmalloc-512 128KB 128KB >[29241.160617] kmalloc-131072 35328KB 35328KB >[29241.160624] kmalloc-65536 9216KB 9216KB >[29241.160630] kmalloc-32768 8192KB 8192KB >[29241.160638] kmalloc-16384 18432KB 20480KB >[29241.160645] kmalloc-8192 8704KB 8704KB >[29241.160651] kmalloc-4096 8192KB 8192KB >[29241.160659] kmalloc-2048 4352KB 4352KB >[29241.160666] kmalloc-1024 19331KB 19456KB >[29241.160675] kmalloc-512 10176KB 11072KB >[29241.160683] kmalloc-256 1408KB 1408KB >[29241.160690] kmalloc-192 1790KB 1790KB >[29241.160697] kmalloc-128 1536KB 1536KB >[29241.160704] kmalloc-96 1470KB 1470KB >[29241.160711] kmalloc-64 1792KB 1792KB >[29241.160718] kmalloc-32 12992KB 12992KB >[29241.160724] kmalloc-16 1024KB 1024KB >[29241.160731] kmalloc-8 1024KB 1024KB >[29241.160739] kmem_cache_node 1024KB 1024KB >[29241.160746] kmem_cache 3315KB 3315KB >[29241.160751] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29241.160772] [ 716] 0 716 2668 0 69632 84 0 systemd-journal >[29241.160782] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29241.160791] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29241.160800] [ 2067] 0 2067 328 0 51200 91 0 systemd-logind >[29241.160809] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29241.160817] [ 2069] 81 2069 208 0 51200 68 -900 dbus-daemon >[29241.160826] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29241.160835] [ 2094] 0 2094 10692 0 122880 94 0 rsyslogd >[29241.160844] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29241.160852] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29241.160861] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29241.160869] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29241.160878] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29241.160886] [ 2139] 0 2139 97 0 51200 37 0 atd >[29241.160895] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29241.160903] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29241.160912] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29241.160921] [ 2337] 0 2337 449 0 53248 80 0 keepalived >[29241.160930] [ 2338] 0 2338 452 0 53248 85 0 keepalived >[29241.160939] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29241.160948] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29241.160956] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29241.160965] [ 2697] 0 2697 87 0 49152 30 0 log >[29241.160973] [ 2699] 0 2699 103 0 51200 32 0 config >[29241.160981] [ 2711] 0 2711 398 0 51200 115 0 sendmail >[29241.160990] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29241.160999] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29241.161008] [ 5044] 0 5044 405 0 53248 115 0 sshd >[29241.161016] [ 5050] 0 5050 380 0 51200 105 0 systemd >[29241.161026] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29241.161037] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29241.161046] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29241.161054] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29241.161063] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29241.161072] [ 8909] 0 8909 3409 0 51200 55 0 ssh >[29241.161081] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29241.161090] [ 8919] 0 8919 168 0 51200 50 0 auth >[29241.161098] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29241.161108] [ 8932] 0 8932 210 0 40960 42 0 sshd >[29241.161116] [ 8933] 0 8933 3329 0 51200 52 0 crond >[29241.161124] Out of memory: Kill process 2711 (sendmail) score 1 or sacrifice child >[29241.161143] Killed process 2711 (sendmail) total-vm:25472kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29241.190192] oom_reaper: reaped process 2711 (sendmail), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29241.210702] in:imjournal invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29241.210716] in:imjournal cpuset=/ mems_allowed=0 >[29241.210728] CPU: 3 PID: 2104 Comm: in:imjournal Not tainted 4.18.7-200.fc28.ppc64 #1 >[29241.210733] Call Trace: >[29241.210739] [c00000003f3631a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29241.210747] [c00000003f363230] [c0000000002fb670] .dump_header+0x70/0x300 >[29241.210754] [c00000003f363300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29241.210762] [c00000003f3633d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29241.210770] [c00000003f363480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29241.210785] [c00000003f363670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29241.210799] [c00000003f363730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29241.210808] [c00000003f3637f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29241.210818] [c00000003f363880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29241.210828] [c00000003f363990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29241.210839] [c00000003f363ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29241.210850] [c00000003f363b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29241.210860] [c00000003f363cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29241.210871] [c00000003f363d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29241.210883] [c00000003f363e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29241.210891] Mem-Info: >[29241.210902] active_anon:0 inactive_anon:7 isolated_anon:0 > active_file:49 inactive_file:6 isolated_file:0 > unevictable:0 dirty:3 writeback:10 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15817 > mapped:49 shmem:0 pagetables:56 bounce:0 > free:1490 free_pcp:0 free_cma:0 >[29241.210923] Node 0 active_anon:0kB inactive_anon:448kB active_file:3136kB inactive_file:384kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3136kB dirty:192kB writeback:640kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29241.210935] Node 0 DMA free:95360kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:3264kB active_file:3136kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9152kB pagetables:3584kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29241.210953] lowmem_reserve[]: 0 0 0 0 >[29241.210961] Node 0 DMA: 422*64kB (UE) 359*128kB (UME) 104*256kB (UME) 6*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102656kB >[29241.210985] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29241.210992] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29241.210997] 90 total pagecache pages >[29241.211001] 7 pages in swap cache >[29241.211005] Swap cache stats: add 2981579, delete 2981685, find 2529761/4291157 >[29241.211010] Free swap = 1880320kB >[29241.211013] Total swap = 2052032kB >[29241.211018] 32768 pages RAM >[29241.211022] 0 pages HighMem/MovableOnly >[29241.211025] 2684 pages reserved >[29241.211029] 0 pages cma reserved >[29241.211032] 0 pages hwpoisoned >[29241.211036] Unreclaimable slab info: >[29241.211039] Name Used Total >[29241.211123] xfs_efd_item 1017KB 1017KB >[29241.211131] xfs_buf_item 1020KB 1020KB >[29241.211137] xfs_trans 1022KB 1022KB >[29241.211143] xfs_da_state 1020KB 1020KB >[29241.211149] xfs_log_ticket 1023KB 1023KB >[29241.219849] scsi_sense_cache 1024KB 1024KB >[29241.219866] RAWv6 504KB 504KB >[29241.219875] UDPv6 1012KB 1012KB >[29241.219882] tw_sock_TCPv6 127KB 127KB >[29241.219890] TCPv6 1386KB 1386KB >[29241.219899] cfq_io_cq 1023KB 1023KB >[29241.219906] cfq_queue 1023KB 1023KB >[29241.219915] mqueue_inode_cache 64KB 64KB >[29241.219922] kioctx 63KB 63KB >[29241.219929] posix_timers_cache 1022KB 1022KB >[29241.219936] ip4-frags 1023KB 1023KB >[29241.219943] secpath_cache 768KB 768KB >[29241.219950] RAW 896KB 896KB >[29241.219956] UDP 1008KB 1008KB >[29241.219963] tw_sock_TCP 127KB 127KB >[29241.219970] request_sock_TCP 191KB 191KB >[29241.219978] TCP 2040KB 2040KB >[29241.219985] hugetlbfs_inode_cache 127KB 127KB >[29241.219992] eventpoll_pwq 1023KB 1023KB >[29241.219998] inotify_inode_mark 1023KB 1023KB >[29241.220005] request_queue 755KB 755KB >[29241.220012] blkdev_ioc 1023KB 1023KB >[29241.220018] biovec-128 256KB 256KB >[29241.220024] biovec-64 128KB 128KB >[29241.220033] skbuff_fclone_cache 1024KB 1024KB >[29241.220041] skbuff_head_cache 1600KB 1600KB >[29241.220050] file_lock_cache 1021KB 1021KB >[29241.220057] net_namespace 510KB 510KB >[29241.220064] shmem_inode_cache 1087KB 1087KB >[29241.220073] dtl 10040KB 11520KB >[29241.220082] taskstats 828KB 828KB >[29241.220091] proc_dir_entry 640KB 640KB >[29241.220099] pde_opener 1023KB 1023KB >[29241.220108] seq_file 1024KB 1024KB >[29241.220117] sigqueue 1022KB 1022KB >[29241.220125] kernfs_node_cache 5302KB 5302KB >[29241.220132] mnt_cache 2996KB 2996KB >[29241.220139] names_cache 4096KB 4096KB >[29241.220145] ebitmap_node 1024KB 1024KB >[29241.220151] avtab_node 1023KB 1023KB >[29241.220156] avc_xperms_data 1024KB 1024KB >[29241.220163] selinux_file_security 1024KB 1024KB >[29241.220169] selinux_inode_security 1855KB 1855KB >[29241.220175] nsproxy 1023KB 1023KB >[29241.220179] vm_area_struct 1660KB 1660KB >[29241.220183] mm_struct 8056KB 8056KB >[29241.220187] files_cache 1211KB 1211KB >[29241.220191] signal_cache 1472KB 1472KB >[29241.220197] sighand_cache 2932KB 2932KB >[29241.220203] task_struct 12210KB 12210KB >[29241.220210] cred_jar 4800KB 4800KB >[29241.220215] thread_stack 17408KB 17408KB >[29241.220218] anon_vma 1086KB 1086KB >[29241.220222] pid 1280KB 1280KB >[29241.220226] numa_policy 383KB 383KB >[29241.220230] trace_event_file 255KB 255KB >[29241.220234] ftrace_event_field 447KB 447KB >[29241.220240] pool_workqueue 1024KB 1024KB >[29241.220246] task_group 1020KB 1020KB >[29241.220251] pgtable-2^11 9216KB 9216KB >[29241.220256] pgtable-2^8 4096KB 4096KB >[29241.220263] dma-kmalloc-512 128KB 128KB >[29241.220271] kmalloc-131072 35328KB 35328KB >[29241.220277] kmalloc-65536 9216KB 9216KB >[29241.220283] kmalloc-32768 8192KB 8192KB >[29241.220290] kmalloc-16384 18432KB 20480KB >[29241.220296] kmalloc-8192 8704KB 8704KB >[29241.220303] kmalloc-4096 8192KB 8192KB >[29241.220310] kmalloc-2048 4352KB 4352KB >[29241.220318] kmalloc-1024 19331KB 19456KB >[29241.220325] kmalloc-512 10176KB 11072KB >[29241.220330] kmalloc-256 1408KB 1408KB >[29241.220337] kmalloc-192 1790KB 1790KB >[29241.220345] kmalloc-128 1536KB 1536KB >[29241.220352] kmalloc-96 1470KB 1470KB >[29241.220357] kmalloc-64 1792KB 1792KB >[29241.220362] kmalloc-32 12992KB 12992KB >[29241.220367] kmalloc-16 1024KB 1024KB >[29241.220371] kmalloc-8 1024KB 1024KB >[29241.220376] kmem_cache_node 1024KB 1024KB >[29241.220382] kmem_cache 3315KB 3315KB >[29241.220387] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29241.220400] [ 716] 0 716 2668 0 69632 84 0 systemd-journal >[29241.220407] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29241.220417] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29241.220425] [ 2067] 0 2067 328 0 51200 91 0 systemd-logind >[29241.220432] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29241.220439] [ 2069] 81 2069 208 0 51200 68 -900 dbus-daemon >[29241.220449] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29241.220456] [ 2094] 0 2094 10692 0 122880 94 0 rsyslogd >[29241.220463] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29241.220470] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29241.220479] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29241.220487] [ 2121] 0 2121 307 1 51200 70 -1000 sshd >[29241.220495] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29241.220503] [ 2139] 0 2139 97 0 51200 37 0 atd >[29241.220510] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29241.220518] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29241.220527] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29241.220536] [ 2337] 0 2337 449 0 53248 80 0 keepalived >[29241.220545] [ 2338] 0 2338 452 0 53248 85 0 keepalived >[29241.220552] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29241.220562] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29241.220569] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29241.220577] [ 2697] 0 2697 87 0 49152 30 0 log >[29241.220585] [ 2699] 0 2699 103 0 51200 32 0 config >[29241.220593] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29241.220602] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29241.220611] [ 5044] 0 5044 405 0 53248 115 0 sshd >[29241.220620] [ 5050] 0 5050 380 0 51200 105 0 systemd >[29241.220628] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29241.220636] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29241.220645] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29241.220652] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29241.220659] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29241.220665] [ 8909] 0 8909 3409 0 51200 55 0 ssh >[29241.220674] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29241.220683] [ 8919] 0 8919 168 0 51200 50 0 auth >[29241.220690] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29241.220699] [ 8932] 0 8932 210 0 40960 42 0 sshd >[29241.220706] [ 8933] 0 8933 3329 0 51200 52 0 crond >[29241.220713] Out of memory: Kill process 5044 (sshd) score 1 or sacrifice child >[29241.220731] Killed process 5044 (sshd) total-vm:25920kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29241.250156] oom_reaper: reaped process 5044 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29244.810628] systemd-journal invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29244.810653] systemd-journal cpuset=/ mems_allowed=0 >[29244.810671] CPU: 0 PID: 716 Comm: systemd-journal Not tainted 4.18.7-200.fc28.ppc64 #1 >[29244.810680] Call Trace: >[29244.810693] [c000000060273380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29244.810709] [c000000060273410] [c0000000002fb670] .dump_header+0x70/0x300 >[29244.810721] [c0000000602734e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29244.810734] [c0000000602735b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29244.810747] [c000000060273660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29244.810762] [c000000060273850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29244.810775] [c0000000602738f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29244.810789] [c000000060273980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29244.810977] [c000000060273a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29244.810991] [c000000060273b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29244.811004] [c000000060273b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29244.811016] [c000000060273cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29244.811031] [c000000060273d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29244.811046] [c000000060273e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29244.811056] Mem-Info: >[29244.811070] active_anon:16 inactive_anon:0 isolated_anon:0 > active_file:0 inactive_file:13 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15846 > mapped:57 shmem:0 pagetables:57 bounce:0 > free:1471 free_pcp:0 free_cma:0 >[29244.811106] Node 0 active_anon:1024kB inactive_anon:0kB active_file:0kB inactive_file:832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3648kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29244.811125] Node 0 DMA free:94144kB min:94144kB low:117632kB high:141120kB active_anon:1024kB inactive_anon:1152kB active_file:1088kB inactive_file:2240kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9168kB pagetables:3648kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29244.819958] lowmem_reserve[]: 0 0 0 0 >[29244.819980] Node 0 DMA: 426*64kB (UME) 365*128kB (UME) 99*256kB (UME) 3*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100864kB >[29244.820021] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29244.820032] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29244.820042] 105 total pagecache pages >[29244.820048] 5 pages in swap cache >[29244.820056] Swap cache stats: add 2985439, delete 2985552, find 2532935/4296949 >[29244.820065] Free swap = 1868032kB >[29244.820071] Total swap = 2052032kB >[29244.820078] 32768 pages RAM >[29244.820083] 0 pages HighMem/MovableOnly >[29244.820089] 2684 pages reserved >[29244.820095] 0 pages cma reserved >[29244.820101] 0 pages hwpoisoned >[29244.820108] Unreclaimable slab info: >[29244.820114] Name Used Total >[29244.820292] xfs_efd_item 1017KB 1017KB >[29244.820301] xfs_buf_item 1020KB 1020KB >[29244.820310] xfs_trans 1022KB 1022KB >[29244.820318] xfs_da_state 1020KB 1020KB >[29244.820327] xfs_log_ticket 1023KB 1023KB >[29244.820340] scsi_sense_cache 1024KB 1024KB >[29244.820367] RAWv6 504KB 504KB >[29244.820376] UDPv6 1012KB 1012KB >[29244.820384] tw_sock_TCPv6 127KB 127KB >[29244.820393] TCPv6 1386KB 1386KB >[29244.820403] cfq_io_cq 1023KB 1023KB >[29244.820412] cfq_queue 1023KB 1023KB >[29244.820421] mqueue_inode_cache 64KB 64KB >[29244.820430] kioctx 63KB 63KB >[29244.820441] posix_timers_cache 1022KB 1022KB >[29244.820449] ip4-frags 1023KB 1023KB >[29244.820457] secpath_cache 768KB 768KB >[29244.820466] RAW 896KB 896KB >[29244.820474] UDP 1008KB 1008KB >[29244.820483] tw_sock_TCP 127KB 127KB >[29244.820491] request_sock_TCP 191KB 191KB >[29244.820500] TCP 2040KB 2040KB >[29244.820507] hugetlbfs_inode_cache 127KB 127KB >[29244.820516] eventpoll_pwq 1023KB 1023KB >[29244.820524] inotify_inode_mark 1023KB 1023KB >[29244.820533] request_queue 755KB 755KB >[29244.820541] blkdev_ioc 1023KB 1023KB >[29244.820549] biovec-128 256KB 256KB >[29244.820557] biovec-64 128KB 128KB >[29244.820566] skbuff_fclone_cache 1024KB 1024KB >[29244.820574] skbuff_head_cache 1600KB 1600KB >[29244.820582] file_lock_cache 1021KB 1021KB >[29244.820591] net_namespace 510KB 510KB >[29244.820600] shmem_inode_cache 1087KB 1087KB >[29244.820609] dtl 10040KB 11520KB >[29244.820618] taskstats 828KB 828KB >[29244.820627] proc_dir_entry 640KB 640KB >[29244.820635] pde_opener 1023KB 1023KB >[29244.820644] seq_file 1024KB 1024KB >[29244.820653] sigqueue 1022KB 1022KB >[29244.820661] kernfs_node_cache 5302KB 5302KB >[29244.820669] mnt_cache 2996KB 2996KB >[29244.820677] names_cache 4096KB 4096KB >[29244.820686] ebitmap_node 1024KB 1024KB >[29244.820693] avtab_node 1023KB 1023KB >[29244.820701] avc_xperms_data 1024KB 1024KB >[29244.820710] selinux_file_security 1024KB 1024KB >[29244.820718] selinux_inode_security 1855KB 1855KB >[29244.820728] nsproxy 1023KB 1023KB >[29244.820737] vm_area_struct 1660KB 1660KB >[29244.820745] mm_struct 8056KB 8056KB >[29244.820753] files_cache 1211KB 1211KB >[29244.820762] signal_cache 1472KB 1472KB >[29244.820770] sighand_cache 2932KB 2932KB >[29244.820779] task_struct 12210KB 12210KB >[29244.820787] cred_jar 4800KB 4800KB >[29244.820795] thread_stack 17408KB 17408KB >[29244.820804] anon_vma 1086KB 1086KB >[29244.820813] pid 1280KB 1280KB >[29244.820821] numa_policy 383KB 383KB >[29244.820829] trace_event_file 255KB 255KB >[29244.820836] ftrace_event_field 447KB 447KB >[29244.820844] pool_workqueue 1024KB 1024KB >[29244.820851] task_group 1020KB 1020KB >[29244.820859] pgtable-2^11 9216KB 9216KB >[29244.820867] pgtable-2^8 4096KB 4096KB >[29244.820877] dma-kmalloc-512 128KB 128KB >[29244.820888] kmalloc-131072 35328KB 35328KB >[29244.820895] kmalloc-65536 9216KB 9216KB >[29244.820903] kmalloc-32768 8192KB 8192KB >[29244.820912] kmalloc-16384 18432KB 20480KB >[29244.820920] kmalloc-8192 8704KB 8704KB >[29244.820927] kmalloc-4096 8192KB 8192KB >[29244.820935] kmalloc-2048 4352KB 4352KB >[29244.820942] kmalloc-1024 19520KB 19520KB >[29244.820953] kmalloc-512 10176KB 11072KB >[29244.820960] kmalloc-256 1408KB 1408KB >[29244.820968] kmalloc-192 1790KB 1790KB >[29244.820976] kmalloc-128 1536KB 1536KB >[29244.820984] kmalloc-96 1470KB 1470KB >[29244.820992] kmalloc-64 1792KB 1792KB >[29244.820999] kmalloc-32 12992KB 12992KB >[29244.821007] kmalloc-16 1024KB 1024KB >[29244.821014] kmalloc-8 1024KB 1024KB >[29244.821023] kmem_cache_node 1024KB 1024KB >[29244.821031] kmem_cache 3315KB 3315KB >[29244.821037] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29244.821062] [ 716] 0 716 2668 0 69632 83 0 systemd-journal >[29244.821075] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29244.821087] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29244.821098] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29244.821110] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29244.821121] [ 2069] 81 2069 208 0 51200 69 -900 dbus-daemon >[29244.821133] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29244.821145] [ 2094] 0 2094 10692 0 122880 86 0 rsyslogd >[29244.821157] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29244.829957] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29244.829972] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29244.829987] [ 2121] 0 2121 307 0 51200 71 -1000 sshd >[29244.830000] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29244.830011] [ 2139] 0 2139 97 0 51200 37 0 atd >[29244.830021] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29244.830033] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29244.830044] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29244.830057] [ 2337] 0 2337 449 0 53248 78 0 keepalived >[29244.830069] [ 2338] 0 2338 452 0 53248 89 0 keepalived >[29244.830081] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29244.830093] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29244.830105] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29244.830116] [ 2697] 0 2697 87 0 49152 30 0 log >[29244.830128] [ 2699] 0 2699 103 0 51200 32 0 config >[29244.830140] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29244.830152] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29244.830164] [ 5050] 0 5050 380 0 51200 105 0 systemd >[29244.830176] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29244.830190] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29244.830201] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29244.830213] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29244.830225] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29244.830236] [ 8909] 0 8909 3409 0 51200 51 0 ssh >[29244.830248] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29244.830260] [ 8919] 0 8919 168 0 51200 48 0 auth >[29244.830271] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29244.830284] [ 8932] 0 8932 264 0 40960 51 0 sshd >[29244.830294] [ 8933] 0 8933 3424 0 53248 60 0 crond >[29244.830305] [ 8937] 0 8937 42 3 38912 8 0 sshd >[29244.830315] Out of memory: Kill process 5050 (systemd) score 1 or sacrifice child >[29244.830339] Killed process 5050 (systemd) total-vm:24320kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29244.850147] oom_reaper: reaped process 5050 (systemd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29244.860249] keepalived invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29244.860261] keepalived cpuset=/ mems_allowed=0 >[29244.860273] CPU: 3 PID: 2338 Comm: keepalived Not tainted 4.18.7-200.fc28.ppc64 #1 >[29244.860280] Call Trace: >[29244.860288] [c000000018f4ec00] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29244.860300] [c000000018f4ec90] [c0000000002fb670] .dump_header+0x70/0x300 >[29244.860311] [c000000018f4ed60] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29244.860323] [c000000018f4ee30] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29244.860334] [c000000018f4eee0] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29244.860346] [c000000018f4f0d0] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29244.860358] [c000000018f4f190] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29244.860370] [c000000018f4f250] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29244.860381] [c000000018f4f2e0] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29244.860393] [c000000018f4f3f0] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29244.860405] [c000000018f4f520] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29244.860416] [c000000018f4f5f0] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29244.860427] [c000000018f4f720] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29244.860439] [c000000018f4f7b0] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29244.860450] [c000000018f4f890] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29244.860465] --- interrupt: 301 at .__copy_tofrom_user_power7+0x218/0x7ac > LR = .core_sys_select+0x2cc/0x430 >[29244.860478] [c000000018f4fb80] [c000000000413418] .core_sys_select+0x2b8/0x430 (unreliable) >[29244.860490] [c000000018f4fd70] [c000000000413668] .__se_sys_select+0xd8/0x140 >[29244.860502] [c000000018f4fe30] [c00000000000bb64] system_call+0x5c/0x70 >[29244.860510] Mem-Info: >[29244.860522] active_anon:16 inactive_anon:0 isolated_anon:0 > active_file:0 inactive_file:13 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15846 > mapped:57 shmem:0 pagetables:57 bounce:0 > free:1471 free_pcp:1 free_cma:0 >[29244.860552] Node 0 active_anon:1024kB inactive_anon:0kB active_file:0kB inactive_file:832kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3648kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29244.860574] Node 0 DMA free:94144kB min:94144kB low:117632kB high:141120kB active_anon:1024kB inactive_anon:1152kB active_file:1088kB inactive_file:2240kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9168kB pagetables:3648kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29244.860606] lowmem_reserve[]: 0 0 0 0 >[29244.860621] Node 0 DMA: 424*64kB (UME) 365*128kB (UME) 99*256kB (UME) 3*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100736kB >[29244.860664] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29244.860677] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29244.860687] 105 total pagecache pages >[29244.860694] 5 pages in swap cache >[29244.860703] Swap cache stats: add 2985440, delete 2985553, find 2532936/4296952 >[29244.860712] Free swap = 1872128kB >[29244.860719] Total swap = 2052032kB >[29244.860727] 32768 pages RAM >[29244.860733] 0 pages HighMem/MovableOnly >[29244.860740] 2684 pages reserved >[29244.860746] 0 pages cma reserved >[29244.860753] 0 pages hwpoisoned >[29244.860759] Unreclaimable slab info: >[29244.860765] Name Used Total >[29244.860859] xfs_efd_item 1017KB 1017KB >[29244.860869] xfs_buf_item 1020KB 1020KB >[29244.860878] xfs_trans 1022KB 1022KB >[29244.860887] xfs_da_state 1020KB 1020KB >[29244.860897] xfs_log_ticket 1023KB 1023KB >[29244.860909] scsi_sense_cache 1024KB 1024KB >[29244.860927] RAWv6 504KB 504KB >[29244.860936] UDPv6 1012KB 1012KB >[29244.860946] tw_sock_TCPv6 127KB 127KB >[29244.860956] TCPv6 1386KB 1386KB >[29244.860967] cfq_io_cq 1023KB 1023KB >[29244.860976] cfq_queue 1023KB 1023KB >[29244.860986] mqueue_inode_cache 64KB 64KB >[29244.860996] kioctx 63KB 63KB >[29244.861006] posix_timers_cache 1022KB 1022KB >[29244.861016] ip4-frags 1023KB 1023KB >[29244.861025] secpath_cache 768KB 768KB >[29244.861035] RAW 896KB 896KB >[29244.861044] UDP 1008KB 1008KB >[29244.861053] tw_sock_TCP 127KB 127KB >[29244.861063] request_sock_TCP 191KB 191KB >[29244.861072] TCP 2040KB 2040KB >[29244.861081] hugetlbfs_inode_cache 127KB 127KB >[29244.861091] eventpoll_pwq 1023KB 1023KB >[29244.861100] inotify_inode_mark 1023KB 1023KB >[29244.861109] request_queue 755KB 755KB >[29244.861119] blkdev_ioc 1023KB 1023KB >[29244.861128] biovec-128 256KB 256KB >[29244.861137] biovec-64 128KB 128KB >[29244.861146] skbuff_fclone_cache 1024KB 1024KB >[29244.869952] skbuff_head_cache 1600KB 1600KB >[29244.869962] file_lock_cache 1021KB 1021KB >[29244.869969] net_namespace 510KB 510KB >[29244.869976] shmem_inode_cache 1087KB 1087KB >[29244.869984] dtl 10040KB 11520KB >[29244.869991] taskstats 828KB 828KB >[29244.869998] proc_dir_entry 640KB 640KB >[29244.870005] pde_opener 1023KB 1023KB >[29244.870012] seq_file 1024KB 1024KB >[29244.870019] sigqueue 1022KB 1022KB >[29244.870026] kernfs_node_cache 5302KB 5302KB >[29244.870032] mnt_cache 2996KB 2996KB >[29244.870039] names_cache 4096KB 4096KB >[29244.870047] ebitmap_node 1024KB 1024KB >[29244.870054] avtab_node 1023KB 1023KB >[29244.870061] avc_xperms_data 1024KB 1024KB >[29244.870068] selinux_file_security 1024KB 1024KB >[29244.870075] selinux_inode_security 1855KB 1855KB >[29244.870082] nsproxy 1023KB 1023KB >[29244.870089] vm_area_struct 1660KB 1660KB >[29244.870096] mm_struct 8056KB 8056KB >[29244.870103] files_cache 1211KB 1211KB >[29244.870110] signal_cache 1472KB 1472KB >[29244.870117] sighand_cache 2932KB 2932KB >[29244.870124] task_struct 12210KB 12210KB >[29244.870131] cred_jar 4800KB 4800KB >[29244.870138] thread_stack 17408KB 17408KB >[29244.870145] anon_vma 1086KB 1086KB >[29244.870152] pid 1280KB 1280KB >[29244.870159] numa_policy 383KB 383KB >[29244.870166] trace_event_file 255KB 255KB >[29244.870173] ftrace_event_field 447KB 447KB >[29244.870180] pool_workqueue 1024KB 1024KB >[29244.870187] task_group 1020KB 1020KB >[29244.870195] pgtable-2^11 9216KB 9216KB >[29244.870202] pgtable-2^8 4096KB 4096KB >[29244.870212] dma-kmalloc-512 128KB 128KB >[29244.870222] kmalloc-131072 35328KB 35328KB >[29244.870229] kmalloc-65536 9216KB 9216KB >[29244.870236] kmalloc-32768 8192KB 8192KB >[29244.870243] kmalloc-16384 18432KB 20480KB >[29244.870250] kmalloc-8192 8704KB 8704KB >[29244.870257] kmalloc-4096 8192KB 8192KB >[29244.870264] kmalloc-2048 4352KB 4352KB >[29244.870271] kmalloc-1024 19520KB 19520KB >[29244.870279] kmalloc-512 10176KB 11072KB >[29244.870286] kmalloc-256 1408KB 1408KB >[29244.870293] kmalloc-192 1790KB 1790KB >[29244.870300] kmalloc-128 1536KB 1536KB >[29244.870308] kmalloc-96 1470KB 1470KB >[29244.870317] kmalloc-64 1792KB 1792KB >[29244.870326] kmalloc-32 12992KB 12992KB >[29244.870336] kmalloc-16 1024KB 1024KB >[29244.870345] kmalloc-8 1024KB 1024KB >[29244.870354] kmem_cache_node 1024KB 1024KB >[29244.870363] kmem_cache 3315KB 3315KB >[29244.870371] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29244.870400] [ 716] 0 716 2668 0 69632 83 0 systemd-journal >[29244.870414] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29244.870423] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29244.870431] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29244.870440] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29244.870451] [ 2069] 81 2069 208 0 51200 69 -900 dbus-daemon >[29244.870459] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29244.870466] [ 2094] 0 2094 10692 0 122880 86 0 rsyslogd >[29244.870472] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29244.870478] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29244.870484] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29244.870490] [ 2121] 0 2121 307 0 51200 71 -1000 sshd >[29244.870496] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29244.870502] [ 2139] 0 2139 97 0 51200 37 0 atd >[29244.870508] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29244.870515] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29244.870523] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29244.870530] [ 2337] 0 2337 449 0 53248 78 0 keepalived >[29244.870538] [ 2338] 0 2338 452 0 53248 89 0 keepalived >[29244.870545] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29244.870553] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29244.870561] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29244.870570] [ 2697] 0 2697 87 0 49152 30 0 log >[29244.870579] [ 2699] 0 2699 103 0 51200 32 0 config >[29244.870588] [ 2719] 51 2719 377 2 51200 99 0 sendmail >[29244.870598] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29244.870608] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29244.870622] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29244.870631] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29244.870642] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29244.870651] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29244.870658] [ 8909] 0 8909 3409 0 51200 51 0 ssh >[29244.870665] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29244.870672] [ 8919] 0 8919 168 0 51200 48 0 auth >[29244.870678] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29244.870686] [ 8932] 0 8932 264 0 40960 51 0 sshd >[29244.870693] [ 8933] 0 8933 3424 0 53248 60 0 crond >[29244.870700] [ 8937] 0 8937 42 3 38912 8 0 sshd >[29244.870706] Out of memory: Kill process 2719 (sendmail) score 1 or sacrifice child >[29244.870719] Killed process 2719 (sendmail) total-vm:24128kB, anon-rss:0kB, file-rss:128kB, shmem-rss:0kB >[29244.890121] oom_reaper: reaped process 2719 (sendmail), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29245.220171] keepalived invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29245.220189] keepalived cpuset=/ mems_allowed=0 >[29245.220206] CPU: 6 PID: 2338 Comm: keepalived Not tainted 4.18.7-200.fc28.ppc64 #1 >[29245.220216] Call Trace: >[29245.220229] [c000000018f4f380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29245.220246] [c000000018f4f410] [c0000000002fb670] .dump_header+0x70/0x300 >[29245.220260] [c000000018f4f4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29245.220274] [c000000018f4f5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29245.220288] [c000000018f4f660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29245.220304] [c000000018f4f850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29245.220320] [c000000018f4f8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29245.220335] [c000000018f4f980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29245.220546] [c000000018f4fa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29245.220562] [c000000018f4fb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29245.220576] [c000000018f4fb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29245.220591] [c000000018f4fcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29245.220606] [c000000018f4fd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29245.220623] [c000000018f4fe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29245.220635] Mem-Info: >[29245.220650] active_anon:0 inactive_anon:35 isolated_anon:0 > active_file:0 inactive_file:6 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15846 > mapped:37 shmem:0 pagetables:57 bounce:0 > free:1467 free_pcp:3 free_cma:0 >[29245.220690] Node 0 active_anon:0kB inactive_anon:2240kB active_file:0kB inactive_file:384kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2368kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29245.220712] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:2240kB active_file:448kB inactive_file:320kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9168kB pagetables:3648kB bounce:0kB free_pcp:192kB local_pcp:0kB free_cma:0kB >[29245.220744] lowmem_reserve[]: 0 0 0 0 >[29245.220759] Node 0 DMA: 416*64kB (UME) 360*128kB (UME) 103*256kB (UME) 5*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 101632kB >[29245.220801] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29245.220813] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29245.220823] 100 total pagecache pages >[29245.220831] 1 pages in swap cache >[29245.220838] Swap cache stats: add 2985564, delete 2985681, find 2533035/4297133 >[29245.220848] Free swap = 1876224kB >[29245.220855] Total swap = 2052032kB >[29245.220862] 32768 pages RAM >[29245.220868] 0 pages HighMem/MovableOnly >[29245.220875] 2684 pages reserved >[29245.220882] 0 pages cma reserved >[29245.220888] 0 pages hwpoisoned >[29245.220895] Unreclaimable slab info: >[29245.220902] Name Used Total >[29245.221033] xfs_efd_item 1017KB 1017KB >[29245.221043] xfs_buf_item 1020KB 1020KB >[29245.221053] xfs_trans 1022KB 1022KB >[29245.221063] xfs_da_state 1020KB 1020KB >[29245.221071] xfs_log_ticket 1023KB 1023KB >[29245.221085] scsi_sense_cache 1024KB 1024KB >[29245.221106] RAWv6 504KB 504KB >[29245.221116] UDPv6 1012KB 1012KB >[29245.221126] tw_sock_TCPv6 127KB 127KB >[29245.221136] TCPv6 1386KB 1386KB >[29245.221146] cfq_io_cq 1023KB 1023KB >[29245.221155] cfq_queue 1023KB 1023KB >[29245.221165] mqueue_inode_cache 64KB 64KB >[29245.229951] kioctx 63KB 63KB >[29245.229966] posix_timers_cache 1022KB 1022KB >[29245.229977] ip4-frags 1023KB 1023KB >[29245.229987] secpath_cache 768KB 768KB >[29245.229996] RAW 896KB 896KB >[29245.230005] UDP 1008KB 1008KB >[29245.230014] tw_sock_TCP 127KB 127KB >[29245.230024] request_sock_TCP 191KB 191KB >[29245.230033] TCP 2040KB 2040KB >[29245.230043] hugetlbfs_inode_cache 127KB 127KB >[29245.230052] eventpoll_pwq 1023KB 1023KB >[29245.230061] inotify_inode_mark 1023KB 1023KB >[29245.230071] request_queue 755KB 755KB >[29245.230081] blkdev_ioc 1023KB 1023KB >[29245.230090] biovec-128 256KB 256KB >[29245.230099] biovec-64 128KB 128KB >[29245.230108] skbuff_fclone_cache 1024KB 1024KB >[29245.230118] skbuff_head_cache 1600KB 1600KB >[29245.230127] file_lock_cache 1021KB 1021KB >[29245.230136] net_namespace 510KB 510KB >[29245.230145] shmem_inode_cache 1087KB 1087KB >[29245.230155] dtl 10040KB 11520KB >[29245.230164] taskstats 828KB 828KB >[29245.230173] proc_dir_entry 640KB 640KB >[29245.230183] pde_opener 1023KB 1023KB >[29245.230192] seq_file 1024KB 1024KB >[29245.230201] sigqueue 1022KB 1022KB >[29245.230210] kernfs_node_cache 5302KB 5302KB >[29245.230219] mnt_cache 2996KB 2996KB >[29245.230228] names_cache 4096KB 4096KB >[29245.230237] ebitmap_node 1024KB 1024KB >[29245.230246] avtab_node 1023KB 1023KB >[29245.230256] avc_xperms_data 1024KB 1024KB >[29245.230266] selinux_file_security 1024KB 1024KB >[29245.230275] selinux_inode_security 1855KB 1855KB >[29245.230286] nsproxy 1023KB 1023KB >[29245.230295] vm_area_struct 1660KB 1660KB >[29245.230303] mm_struct 8056KB 8056KB >[29245.230313] files_cache 1211KB 1211KB >[29245.230323] signal_cache 1472KB 1472KB >[29245.230334] sighand_cache 2932KB 2932KB >[29245.230344] task_struct 12210KB 12210KB >[29245.230353] cred_jar 4800KB 4800KB >[29245.230362] thread_stack 17408KB 17408KB >[29245.230371] anon_vma 1086KB 1086KB >[29245.230380] pid 1280KB 1280KB >[29245.230389] numa_policy 383KB 383KB >[29245.230399] trace_event_file 255KB 255KB >[29245.230407] ftrace_event_field 447KB 447KB >[29245.230417] pool_workqueue 1024KB 1024KB >[29245.230427] task_group 1020KB 1020KB >[29245.230436] pgtable-2^11 9216KB 9216KB >[29245.230445] pgtable-2^8 4096KB 4096KB >[29245.230458] dma-kmalloc-512 128KB 128KB >[29245.230471] kmalloc-131072 35328KB 35328KB >[29245.230480] kmalloc-65536 9216KB 9216KB >[29245.230489] kmalloc-32768 8192KB 8192KB >[29245.230498] kmalloc-16384 18432KB 20480KB >[29245.230507] kmalloc-8192 8704KB 8704KB >[29245.230516] kmalloc-4096 8192KB 8192KB >[29245.230525] kmalloc-2048 4352KB 4352KB >[29245.230534] kmalloc-1024 19520KB 19520KB >[29245.230544] kmalloc-512 10176KB 11072KB >[29245.230553] kmalloc-256 1408KB 1408KB >[29245.230562] kmalloc-192 1790KB 1790KB >[29245.230572] kmalloc-128 1536KB 1536KB >[29245.230580] kmalloc-96 1470KB 1470KB >[29245.230589] kmalloc-64 1792KB 1792KB >[29245.230598] kmalloc-32 12992KB 12992KB >[29245.230607] kmalloc-16 1024KB 1024KB >[29245.230616] kmalloc-8 1024KB 1024KB >[29245.230625] kmem_cache_node 1024KB 1024KB >[29245.230634] kmem_cache 3315KB 3315KB >[29245.230642] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29245.230671] [ 716] 0 716 2668 0 69632 82 0 systemd-journal >[29245.230686] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29245.230700] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29245.230715] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29245.230729] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29245.230742] [ 2069] 81 2069 208 0 51200 60 -900 dbus-daemon >[29245.230756] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29245.230771] [ 2094] 0 2094 10692 0 122880 81 0 rsyslogd >[29245.230784] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29245.230797] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29245.230810] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29245.230824] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29245.230837] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29245.230850] [ 2139] 0 2139 97 0 51200 37 0 atd >[29245.230864] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29245.230877] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29245.230890] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29245.230906] [ 2337] 0 2337 449 0 53248 87 0 keepalived >[29245.230919] [ 2338] 0 2338 452 2 53248 78 0 keepalived >[29245.230932] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29245.230947] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29245.230960] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29245.230974] [ 2697] 0 2697 87 0 49152 30 0 log >[29245.230987] [ 2699] 0 2699 103 0 51200 32 0 config >[29245.231001] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29245.231016] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29245.231033] [ 8871] 0 8871 3494 1 53248 91 0 crond >[29245.231047] [ 8874] 0 8874 3282 0 51200 14 0 sh >[29245.231061] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29245.231075] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29245.231087] [ 8909] 0 8909 3409 0 51200 57 0 ssh >[29245.231103] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29245.231117] [ 8919] 0 8919 168 1 51200 37 0 auth >[29245.231130] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29245.231144] [ 8932] 0 8932 264 0 40960 56 0 sshd >[29245.239953] [ 8933] 0 8933 3424 0 53248 66 0 crond >[29245.239972] [ 8937] 0 8937 58 0 38912 14 0 sshd >[29245.239986] Out of memory: Kill process 8871 (crond) score 1 or sacrifice child >[29245.240015] Killed process 8874 (sh) total-vm:210048kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29245.910142] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29245.910164] sshd cpuset=/ mems_allowed=0 >[29245.910183] CPU: 13 PID: 8932 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29245.910192] Call Trace: >[29245.910205] [c00000002abd3230] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29245.910221] [c00000002abd32c0] [c0000000002fb670] .dump_header+0x70/0x300 >[29245.910234] [c00000002abd3390] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29245.910247] [c00000002abd3460] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29245.910261] [c00000002abd3510] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29245.910277] [c00000002abd3700] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29245.910292] [c00000002abd37c0] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29245.910306] [c00000002abd3880] [c00000000037a284] .swap_cluster_readahead+0x1e4/0x3e0 >[29245.910320] [c00000002abd3990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29245.910334] [c00000002abd3ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29245.910347] [c00000002abd3b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29245.910361] [c00000002abd3cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29245.910376] [c00000002abd3d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29245.910391] [c00000002abd3e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29245.910402] Mem-Info: >[29245.910417] active_anon:0 inactive_anon:92 isolated_anon:0 > active_file:24 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15846 > mapped:45 shmem:0 pagetables:57 bounce:0 > free:1462 free_pcp:0 free_cma:0 >[29245.910455] Node 0 active_anon:0kB inactive_anon:5888kB active_file:1536kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2880kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29245.910475] Node 0 DMA free:93568kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1472kB active_file:0kB inactive_file:1088kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9184kB pagetables:3648kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29245.910505] lowmem_reserve[]: 0 0 0 0 >[29245.910519] Node 0 DMA: 432*64kB (UME) 365*128kB (UME) 97*256kB (UME) 5*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 101760kB >[29245.910560] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29245.910571] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29245.910581] 113 total pagecache pages >[29245.910588] 6 pages in swap cache >[29245.910596] Swap cache stats: add 2986088, delete 2986200, find 2533453/4297905 >[29245.910606] Free swap = 1868032kB >[29245.910612] Total swap = 2052032kB >[29245.910619] 32768 pages RAM >[29245.910624] 0 pages HighMem/MovableOnly >[29245.910631] 2684 pages reserved >[29245.910637] 0 pages cma reserved >[29245.910643] 0 pages hwpoisoned >[29245.910651] Unreclaimable slab info: >[29245.910657] Name Used Total >[29245.910816] xfs_efd_item 1017KB 1017KB >[29245.910826] xfs_buf_item 1020KB 1020KB >[29245.910835] xfs_trans 1022KB 1022KB >[29245.910844] xfs_da_state 1020KB 1020KB >[29245.910853] xfs_log_ticket 1023KB 1023KB >[29245.910867] scsi_sense_cache 1024KB 1024KB >[29245.910893] RAWv6 504KB 504KB >[29245.910902] UDPv6 1012KB 1012KB >[29245.910912] tw_sock_TCPv6 127KB 127KB >[29245.910921] TCPv6 1386KB 1386KB >[29245.910931] cfq_io_cq 1023KB 1023KB >[29245.910939] cfq_queue 1023KB 1023KB >[29245.910949] mqueue_inode_cache 64KB 64KB >[29245.910959] kioctx 63KB 63KB >[29245.910970] posix_timers_cache 1022KB 1022KB >[29245.910979] ip4-frags 1023KB 1023KB >[29245.910988] secpath_cache 768KB 768KB >[29245.910997] RAW 896KB 896KB >[29245.911006] UDP 1008KB 1008KB >[29245.911015] tw_sock_TCP 127KB 127KB >[29245.911025] request_sock_TCP 191KB 191KB >[29245.911033] TCP 2040KB 2040KB >[29245.911042] hugetlbfs_inode_cache 127KB 127KB >[29245.911051] eventpoll_pwq 1023KB 1023KB >[29245.911060] inotify_inode_mark 1023KB 1023KB >[29245.911070] request_queue 755KB 755KB >[29245.911079] blkdev_ioc 1023KB 1023KB >[29245.911087] biovec-128 256KB 256KB >[29245.911096] biovec-64 128KB 128KB >[29245.911105] skbuff_fclone_cache 1024KB 1024KB >[29245.911115] skbuff_head_cache 1600KB 1600KB >[29245.911123] file_lock_cache 1021KB 1021KB >[29245.911132] net_namespace 510KB 510KB >[29245.911140] shmem_inode_cache 1087KB 1087KB >[29245.911150] dtl 10040KB 11520KB >[29245.911159] taskstats 828KB 828KB >[29245.911168] proc_dir_entry 640KB 640KB >[29245.919959] pde_opener 1023KB 1023KB >[29245.919970] seq_file 1024KB 1024KB >[29245.919980] sigqueue 1022KB 1022KB >[29245.919988] kernfs_node_cache 5302KB 5302KB >[29245.919997] mnt_cache 2996KB 2996KB >[29245.920006] names_cache 4096KB 4096KB >[29245.920015] ebitmap_node 1024KB 1024KB >[29245.920024] avtab_node 1023KB 1023KB >[29245.920033] avc_xperms_data 1024KB 1024KB >[29245.920041] selinux_file_security 1024KB 1024KB >[29245.920051] selinux_inode_security 1855KB 1855KB >[29245.920060] nsproxy 1023KB 1023KB >[29245.920069] vm_area_struct 1660KB 1660KB >[29245.920077] mm_struct 8056KB 8056KB >[29245.920086] files_cache 1211KB 1211KB >[29245.920095] signal_cache 1472KB 1472KB >[29245.920103] sighand_cache 2932KB 2932KB >[29245.920112] task_struct 12210KB 12210KB >[29245.920121] cred_jar 4800KB 4800KB >[29245.920130] thread_stack 17408KB 17408KB >[29245.920138] anon_vma 1086KB 1086KB >[29245.920147] pid 1280KB 1280KB >[29245.920156] numa_policy 383KB 383KB >[29245.920165] trace_event_file 255KB 255KB >[29245.920173] ftrace_event_field 447KB 447KB >[29245.920182] pool_workqueue 1024KB 1024KB >[29245.920191] task_group 1020KB 1020KB >[29245.920200] pgtable-2^11 9216KB 9216KB >[29245.920209] pgtable-2^8 4096KB 4096KB >[29245.920222] dma-kmalloc-512 128KB 128KB >[29245.920235] kmalloc-131072 35328KB 35328KB >[29245.920243] kmalloc-65536 9216KB 9216KB >[29245.920251] kmalloc-32768 8192KB 8192KB >[29245.920260] kmalloc-16384 18432KB 20480KB >[29245.920270] kmalloc-8192 8704KB 8704KB >[29245.920280] kmalloc-4096 8192KB 8192KB >[29245.920288] kmalloc-2048 4352KB 4352KB >[29245.920295] kmalloc-1024 19520KB 19520KB >[29245.920303] kmalloc-512 10176KB 11072KB >[29245.920311] kmalloc-256 1408KB 1408KB >[29245.920317] kmalloc-192 1790KB 1790KB >[29245.920324] kmalloc-128 1536KB 1536KB >[29245.920331] kmalloc-96 1470KB 1470KB >[29245.920337] kmalloc-64 1792KB 1792KB >[29245.920343] kmalloc-32 12992KB 12992KB >[29245.920348] kmalloc-16 1024KB 1024KB >[29245.920353] kmalloc-8 1024KB 1024KB >[29245.920359] kmem_cache_node 1024KB 1024KB >[29245.920365] kmem_cache 3315KB 3315KB >[29245.920370] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29245.920390] [ 716] 0 716 2668 0 69632 89 0 systemd-journal >[29245.920399] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29245.920409] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29245.920417] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29245.920426] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29245.920434] [ 2069] 81 2069 208 1 51200 55 -900 dbus-daemon >[29245.920442] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29245.920451] [ 2094] 0 2094 10692 0 122880 89 0 rsyslogd >[29245.920459] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29245.920467] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29245.920476] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29245.920484] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29245.920492] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29245.920501] [ 2139] 0 2139 97 0 51200 37 0 atd >[29245.920509] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29245.920517] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29245.920525] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29245.920535] [ 2337] 0 2337 449 2 53248 78 0 keepalived >[29245.920543] [ 2338] 0 2338 452 1 53248 83 0 keepalived >[29245.920552] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29245.920562] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29245.920571] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29245.920581] [ 2697] 0 2697 87 0 49152 30 0 log >[29245.920591] [ 2699] 0 2699 103 0 51200 32 0 config >[29245.920603] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29245.920613] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29245.920624] [ 8871] 0 8871 3494 0 53248 93 0 crond >[29245.920633] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29245.920642] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29245.920649] [ 8909] 0 8909 3409 0 51200 57 0 ssh >[29245.920658] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29245.920667] [ 8919] 0 8919 168 0 51200 44 0 auth >[29245.920675] [ 8920] 990 8920 159 2 51200 26 0 pop3-login >[29245.920686] [ 8932] 0 8932 264 0 40960 52 0 sshd >[29245.920695] [ 8933] 0 8933 3428 1 53248 62 0 crond >[29245.920705] [ 8937] 0 8937 110 0 38912 20 0 sshd >[29245.920712] Out of memory: Kill process 8871 (crond) score 1 or sacrifice child >[29245.920724] Killed process 8871 (crond) total-vm:223616kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29245.940229] oom_reaper: reaped process 8871 (crond), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29246.010412] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29246.010426] sshd cpuset=/ mems_allowed=0 >[29246.010440] CPU: 1 PID: 8937 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29246.010448] Call Trace: >[29246.010458] [c00000000e3db1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29246.010474] [c00000000e3db230] [c0000000002fb670] .dump_header+0x70/0x300 >[29246.010487] [c00000000e3db300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29246.010500] [c00000000e3db3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29246.010514] [c00000000e3db480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29246.010528] [c00000000e3db670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29246.010543] [c00000000e3db730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29246.010557] [c00000000e3db7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29246.010571] [c00000000e3db880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29246.010586] [c00000000e3db990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29246.010599] [c00000000e3dbac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29246.010613] [c00000000e3dbb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29246.010626] [c00000000e3dbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29246.010640] [c00000000e3dbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29246.010654] [c00000000e3dbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29246.010665] Mem-Info: >[29246.010679] active_anon:0 inactive_anon:90 isolated_anon:0 > active_file:12 inactive_file:7 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15846 > mapped:45 shmem:0 pagetables:57 bounce:0 > free:1467 free_pcp:0 free_cma:0 >[29246.010716] Node 0 active_anon:0kB inactive_anon:5760kB active_file:768kB inactive_file:448kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2880kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29246.010737] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:2816kB active_file:0kB inactive_file:1920kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9184kB pagetables:3648kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29246.010768] lowmem_reserve[]: 0 0 0 0 >[29246.010781] Node 0 DMA: 425*64kB (UME) 361*128kB (UME) 102*256kB (UME) 5*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102080kB >[29246.010822] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29246.010833] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29246.010843] 109 total pagecache pages >[29246.010850] 1 pages in swap cache >[29246.010858] Swap cache stats: add 2986091, delete 2986208, find 2533454/4297908 >[29246.010867] Free swap = 1872128kB >[29246.010874] Total swap = 2052032kB >[29246.010881] 32768 pages RAM >[29246.010886] 0 pages HighMem/MovableOnly >[29246.010893] 2684 pages reserved >[29246.010899] 0 pages cma reserved >[29246.010906] 0 pages hwpoisoned >[29246.010912] Unreclaimable slab info: >[29246.010918] Name Used Total >[29246.010968] xfs_efd_item 1017KB 1017KB >[29246.010978] xfs_buf_item 1020KB 1020KB >[29246.010987] xfs_trans 1022KB 1022KB >[29246.010996] xfs_da_state 1020KB 1020KB >[29246.011004] xfs_log_ticket 1023KB 1023KB >[29246.011014] scsi_sense_cache 1024KB 1024KB >[29246.011028] RAWv6 504KB 504KB >[29246.011037] UDPv6 1012KB 1012KB >[29246.011046] tw_sock_TCPv6 127KB 127KB >[29246.011055] TCPv6 1386KB 1386KB >[29246.011064] cfq_io_cq 1023KB 1023KB >[29246.011073] cfq_queue 1023KB 1023KB >[29246.011082] mqueue_inode_cache 64KB 64KB >[29246.011091] kioctx 63KB 63KB >[29246.011101] posix_timers_cache 1022KB 1022KB >[29246.011110] ip4-frags 1023KB 1023KB >[29246.011119] secpath_cache 768KB 768KB >[29246.011127] RAW 896KB 896KB >[29246.011136] UDP 1008KB 1008KB >[29246.011144] tw_sock_TCP 127KB 127KB >[29246.011153] request_sock_TCP 191KB 191KB >[29246.011161] TCP 2040KB 2040KB >[29246.011170] hugetlbfs_inode_cache 127KB 127KB >[29246.019954] eventpoll_pwq 1023KB 1023KB >[29246.019965] inotify_inode_mark 1023KB 1023KB >[29246.019976] request_queue 755KB 755KB >[29246.019985] blkdev_ioc 1023KB 1023KB >[29246.019996] biovec-128 256KB 256KB >[29246.020004] biovec-64 128KB 128KB >[29246.020013] skbuff_fclone_cache 1024KB 1024KB >[29246.020022] skbuff_head_cache 1600KB 1600KB >[29246.020031] file_lock_cache 1021KB 1021KB >[29246.020040] net_namespace 510KB 510KB >[29246.020048] shmem_inode_cache 1087KB 1087KB >[29246.020058] dtl 10040KB 11520KB >[29246.020067] taskstats 828KB 828KB >[29246.020075] proc_dir_entry 640KB 640KB >[29246.020083] pde_opener 1023KB 1023KB >[29246.020092] seq_file 1024KB 1024KB >[29246.020100] sigqueue 1022KB 1022KB >[29246.020109] kernfs_node_cache 5302KB 5302KB >[29246.020118] mnt_cache 2996KB 2996KB >[29246.020127] names_cache 4096KB 4096KB >[29246.020135] ebitmap_node 1024KB 1024KB >[29246.020144] avtab_node 1023KB 1023KB >[29246.020153] avc_xperms_data 1024KB 1024KB >[29246.020162] selinux_file_security 1024KB 1024KB >[29246.020171] selinux_inode_security 1855KB 1855KB >[29246.020180] nsproxy 1023KB 1023KB >[29246.020189] vm_area_struct 1660KB 1660KB >[29246.020198] mm_struct 8056KB 8056KB >[29246.020206] files_cache 1211KB 1211KB >[29246.020215] signal_cache 1472KB 1472KB >[29246.020223] sighand_cache 2932KB 2932KB >[29246.020232] task_struct 12210KB 12210KB >[29246.020240] cred_jar 4800KB 4800KB >[29246.020249] thread_stack 17408KB 17408KB >[29246.020257] anon_vma 1086KB 1086KB >[29246.020265] pid 1280KB 1280KB >[29246.020274] numa_policy 383KB 383KB >[29246.020282] trace_event_file 255KB 255KB >[29246.020291] ftrace_event_field 447KB 447KB >[29246.020300] pool_workqueue 1024KB 1024KB >[29246.020309] task_group 1020KB 1020KB >[29246.020318] pgtable-2^11 9216KB 9216KB >[29246.020326] pgtable-2^8 4096KB 4096KB >[29246.020339] dma-kmalloc-512 128KB 128KB >[29246.020351] kmalloc-131072 35328KB 35328KB >[29246.020359] kmalloc-65536 9216KB 9216KB >[29246.020367] kmalloc-32768 8192KB 8192KB >[29246.020376] kmalloc-16384 18432KB 20480KB >[29246.020385] kmalloc-8192 8704KB 8704KB >[29246.020394] kmalloc-4096 8192KB 8192KB >[29246.020402] kmalloc-2048 4352KB 4352KB >[29246.020411] kmalloc-1024 19520KB 19520KB >[29246.020420] kmalloc-512 10176KB 11072KB >[29246.020429] kmalloc-256 1408KB 1408KB >[29246.020438] kmalloc-192 1790KB 1790KB >[29246.020446] kmalloc-128 1536KB 1536KB >[29246.020455] kmalloc-96 1470KB 1470KB >[29246.020463] kmalloc-64 1792KB 1792KB >[29246.020472] kmalloc-32 12992KB 12992KB >[29246.020480] kmalloc-16 1024KB 1024KB >[29246.020489] kmalloc-8 1024KB 1024KB >[29246.020497] kmem_cache_node 1024KB 1024KB >[29246.020506] kmem_cache 3315KB 3315KB >[29246.020514] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29246.020541] [ 716] 0 716 2668 0 69632 89 0 systemd-journal >[29246.020555] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29246.020569] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29246.020583] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29246.020596] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29246.020609] [ 2069] 81 2069 208 1 51200 55 -900 dbus-daemon >[29246.020622] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29246.020636] [ 2094] 0 2094 10692 0 122880 89 0 rsyslogd >[29246.020648] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29246.020660] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29246.020673] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29246.020686] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29246.020699] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29246.020711] [ 2139] 0 2139 97 0 51200 37 0 atd >[29246.020723] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29246.020736] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29246.020748] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29246.020762] [ 2337] 0 2337 449 2 53248 78 0 keepalived >[29246.020775] [ 2338] 0 2338 452 1 53248 83 0 keepalived >[29246.020788] [ 2339] 0 2339 474 0 55296 91 0 keepalived >[29246.020802] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29246.020814] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29246.020826] [ 2697] 0 2697 87 0 49152 30 0 log >[29246.020838] [ 2699] 0 2699 103 0 51200 32 0 config >[29246.020852] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29246.020867] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29246.020883] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29246.020897] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29246.020909] [ 8909] 0 8909 3409 0 51200 57 0 ssh >[29246.020923] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29246.020936] [ 8919] 0 8919 168 0 51200 44 0 auth >[29246.020948] [ 8920] 990 8920 159 1 51200 26 0 pop3-login >[29246.020962] [ 8932] 0 8932 264 0 40960 52 0 sshd >[29246.020974] [ 8933] 0 8933 3428 1 53248 62 0 crond >[29246.020987] [ 8937] 0 8937 110 0 38912 20 0 sshd >[29246.020998] Out of memory: Kill process 2339 (keepalived) score 1 or sacrifice child >[29246.021018] Killed process 2339 (keepalived) total-vm:30336kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29246.040116] oom_reaper: reaped process 2339 (keepalived), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29246.120514] systemd-journal invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29246.120523] systemd-journal cpuset=/ mems_allowed=0 >[29246.120535] CPU: 4 PID: 716 Comm: systemd-journal Not tainted 4.18.7-200.fc28.ppc64 #1 >[29246.120541] Call Trace: >[29246.120549] [c000000060273380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29246.120561] [c000000060273410] [c0000000002fb670] .dump_header+0x70/0x300 >[29246.120570] [c0000000602734e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29246.120579] [c0000000602735b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29246.120589] [c000000060273660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29246.120600] [c000000060273850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29246.120611] [c0000000602738f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29246.120621] [c000000060273980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29246.120749] [c000000060273a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29246.120760] [c000000060273b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29246.120769] [c000000060273b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29246.120779] [c000000060273cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29246.120790] [c000000060273d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29246.120800] [c000000060273e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29246.120807] Mem-Info: >[29246.120818] active_anon:0 inactive_anon:81 isolated_anon:0 > active_file:22 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2800 slab_unreclaimable:15846 > mapped:38 shmem:0 pagetables:57 bounce:0 > free:1481 free_pcp:1 free_cma:0 >[29246.120845] Node 0 active_anon:0kB inactive_anon:5184kB active_file:1408kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2432kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29246.120860] Node 0 DMA free:94784kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:2240kB active_file:1216kB inactive_file:832kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9184kB pagetables:3648kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29246.120882] lowmem_reserve[]: 0 0 0 0 >[29246.120892] Node 0 DMA: 426*64kB (UME) 361*128kB (UME) 102*256kB (UME) 5*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102144kB >[29246.120920] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29246.120928] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29246.120935] 97 total pagecache pages >[29246.120940] 1 pages in swap cache >[29246.120946] Swap cache stats: add 2986092, delete 2986209, find 2533455/4297910 >[29246.120952] Free swap = 1872128kB >[29246.120957] Total swap = 2052032kB >[29246.120961] 32768 pages RAM >[29246.120965] 0 pages HighMem/MovableOnly >[29246.120970] 2684 pages reserved >[29246.120974] 0 pages cma reserved >[29246.120978] 0 pages hwpoisoned >[29246.120983] Unreclaimable slab info: >[29246.120987] Name Used Total >[29246.121076] xfs_efd_item 1017KB 1017KB >[29246.121083] xfs_buf_item 1020KB 1020KB >[29246.121089] xfs_trans 1022KB 1022KB >[29246.121095] xfs_da_state 1020KB 1020KB >[29246.121101] xfs_log_ticket 1023KB 1023KB >[29246.121110] scsi_sense_cache 1024KB 1024KB >[29246.121125] RAWv6 504KB 504KB >[29246.121131] UDPv6 1012KB 1012KB >[29246.121138] tw_sock_TCPv6 127KB 127KB >[29246.121144] TCPv6 1386KB 1386KB >[29246.121151] cfq_io_cq 1023KB 1023KB >[29246.129954] cfq_queue 1023KB 1023KB >[29246.129961] mqueue_inode_cache 64KB 64KB >[29246.129968] kioctx 63KB 63KB >[29246.129975] posix_timers_cache 1022KB 1022KB >[29246.129980] ip4-frags 1023KB 1023KB >[29246.129987] secpath_cache 768KB 768KB >[29246.129992] RAW 896KB 896KB >[29246.129998] UDP 1008KB 1008KB >[29246.130004] tw_sock_TCP 127KB 127KB >[29246.130010] request_sock_TCP 191KB 191KB >[29246.130015] TCP 2040KB 2040KB >[29246.130021] hugetlbfs_inode_cache 127KB 127KB >[29246.130027] eventpoll_pwq 1023KB 1023KB >[29246.130034] inotify_inode_mark 1023KB 1023KB >[29246.130040] request_queue 755KB 755KB >[29246.130046] blkdev_ioc 1023KB 1023KB >[29246.130052] biovec-128 256KB 256KB >[29246.130058] biovec-64 128KB 128KB >[29246.130064] skbuff_fclone_cache 1024KB 1024KB >[29246.130070] skbuff_head_cache 1600KB 1600KB >[29246.130076] file_lock_cache 1021KB 1021KB >[29246.130082] net_namespace 510KB 510KB >[29246.130088] shmem_inode_cache 1087KB 1087KB >[29246.130094] dtl 10040KB 11520KB >[29246.130100] taskstats 828KB 828KB >[29246.130107] proc_dir_entry 640KB 640KB >[29246.130113] pde_opener 1023KB 1023KB >[29246.130119] seq_file 1024KB 1024KB >[29246.130124] sigqueue 1022KB 1022KB >[29246.130130] kernfs_node_cache 5302KB 5302KB >[29246.130136] mnt_cache 2996KB 2996KB >[29246.130143] names_cache 4096KB 4096KB >[29246.130149] ebitmap_node 1024KB 1024KB >[29246.130155] avtab_node 1023KB 1023KB >[29246.130161] avc_xperms_data 1024KB 1024KB >[29246.130167] selinux_file_security 1024KB 1024KB >[29246.130172] selinux_inode_security 1855KB 1855KB >[29246.130179] nsproxy 1023KB 1023KB >[29246.130186] vm_area_struct 1660KB 1660KB >[29246.130192] mm_struct 8056KB 8056KB >[29246.130198] files_cache 1211KB 1211KB >[29246.130204] signal_cache 1472KB 1472KB >[29246.130210] sighand_cache 2932KB 2932KB >[29246.130216] task_struct 12210KB 12210KB >[29246.130222] cred_jar 4800KB 4800KB >[29246.130228] thread_stack 17408KB 17408KB >[29246.130234] anon_vma 1086KB 1086KB >[29246.130240] pid 1280KB 1280KB >[29246.130246] numa_policy 383KB 383KB >[29246.130252] trace_event_file 255KB 255KB >[29246.130258] ftrace_event_field 447KB 447KB >[29246.130264] pool_workqueue 1024KB 1024KB >[29246.130270] task_group 1020KB 1020KB >[29246.130276] pgtable-2^11 9216KB 9216KB >[29246.130282] pgtable-2^8 4096KB 4096KB >[29246.130290] dma-kmalloc-512 128KB 128KB >[29246.130298] kmalloc-131072 35328KB 35328KB >[29246.130304] kmalloc-65536 9216KB 9216KB >[29246.130311] kmalloc-32768 8192KB 8192KB >[29246.130317] kmalloc-16384 18432KB 20480KB >[29246.130323] kmalloc-8192 8704KB 8704KB >[29246.130329] kmalloc-4096 8192KB 8192KB >[29246.130335] kmalloc-2048 4352KB 4352KB >[29246.130341] kmalloc-1024 19520KB 19520KB >[29246.130348] kmalloc-512 10176KB 11072KB >[29246.130354] kmalloc-256 1408KB 1408KB >[29246.130360] kmalloc-192 1790KB 1790KB >[29246.130366] kmalloc-128 1536KB 1536KB >[29246.130372] kmalloc-96 1470KB 1470KB >[29246.130378] kmalloc-64 1792KB 1792KB >[29246.130384] kmalloc-32 12992KB 12992KB >[29246.130390] kmalloc-16 1024KB 1024KB >[29246.130396] kmalloc-8 1024KB 1024KB >[29246.130402] kmem_cache_node 1024KB 1024KB >[29246.130407] kmem_cache 3315KB 3315KB >[29246.130413] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29246.130435] [ 716] 0 716 2668 0 69632 89 0 systemd-journal >[29246.130445] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29246.130455] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29246.130464] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29246.130474] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29246.130483] [ 2069] 81 2069 208 1 51200 55 -900 dbus-daemon >[29246.130493] [ 2089] 0 2089 1288 0 51200 40 0 irqbalance >[29246.130503] [ 2094] 0 2094 10692 0 122880 89 0 rsyslogd >[29246.130512] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29246.130521] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29246.130530] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29246.130540] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29246.130549] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29246.130558] [ 2139] 0 2139 97 0 51200 37 0 atd >[29246.130567] [ 2140] 0 2140 3304 0 51200 50 0 crond >[29246.130576] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29246.130585] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29246.130595] [ 2337] 0 2337 449 2 53248 78 0 keepalived >[29246.130604] [ 2338] 0 2338 452 1 53248 83 0 keepalived >[29246.130615] [ 2695] 0 2695 91 0 49152 33 0 dovecot >[29246.130624] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29246.130633] [ 2697] 0 2697 87 0 49152 30 0 log >[29246.130642] [ 2699] 0 2699 103 0 51200 32 0 config >[29246.130652] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29246.130662] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29246.130673] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29246.130680] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29246.130688] [ 8909] 0 8909 3409 0 51200 57 0 ssh >[29246.130696] [ 8918] 990 8918 159 0 51200 36 0 pop3-login >[29246.130704] [ 8919] 0 8919 168 0 51200 44 0 auth >[29246.130709] [ 8920] 990 8920 159 1 51200 26 0 pop3-login >[29246.130716] [ 8932] 0 8932 264 0 40960 52 0 sshd >[29246.130721] [ 8933] 0 8933 3428 1 53248 62 0 crond >[29246.130727] [ 8937] 0 8937 110 0 38912 20 0 sshd >[29246.130732] Out of memory: Kill process 716 (systemd-journal) score 1 or sacrifice child >[29246.130741] Killed process 716 (systemd-journal) total-vm:170752kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29246.131111] oom_reaper: reaped process 716 (systemd-journal), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29247.041124] keepalived invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29247.041148] keepalived cpuset=/ mems_allowed=0 >[29247.049972] CPU: 7 PID: 2337 Comm: keepalived Not tainted 4.18.7-200.fc28.ppc64 #1 >[29247.049982] Call Trace: >[29247.049995] [c00000000c1e3380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29247.050011] [c00000000c1e3410] [c0000000002fb670] .dump_header+0x70/0x300 >[29247.050023] [c00000000c1e34e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29247.050036] [c00000000c1e35b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29247.050049] [c00000000c1e3660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29247.050063] [c00000000c1e3850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29247.050077] [c00000000c1e38f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29247.050090] [c00000000c1e3980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29247.050288] [c00000000c1e3a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29247.050302] [c00000000c1e3b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29247.050315] [c00000000c1e3b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29247.050328] [c00000000c1e3cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29247.050342] [c00000000c1e3d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29247.050356] [c00000000c1e3e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29247.050366] Mem-Info: >[29247.050381] active_anon:0 inactive_anon:30 isolated_anon:0 > active_file:0 inactive_file:8 isolated_file:0 > unevictable:0 dirty:0 writeback:2 unstable:0 > slab_reclaimable:2799 slab_unreclaimable:15845 > mapped:0 shmem:0 pagetables:54 bounce:0 > free:1467 free_pcp:0 free_cma:0 >[29247.050417] Node 0 active_anon:0kB inactive_anon:1920kB active_file:0kB inactive_file:512kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:128kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29247.050437] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:2496kB inactive_anon:128kB active_file:832kB inactive_file:0kB unevictable:0kB writepending:128kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9152kB pagetables:3456kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29247.050466] lowmem_reserve[]: 0 0 0 0 >[29247.050479] Node 0 DMA: 434*64kB (UME) 360*128kB (UME) 102*256kB (UME) 4*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 103040kB >[29247.050519] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29247.050530] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29247.050540] 14 total pagecache pages >[29247.050547] 0 pages in swap cache >[29247.050554] Swap cache stats: add 2986945, delete 2987063, find 2534134/4299173 >[29247.050562] Free swap = 1880896kB >[29247.050568] Total swap = 2052032kB >[29247.050575] 32768 pages RAM >[29247.050580] 0 pages HighMem/MovableOnly >[29247.050586] 2684 pages reserved >[29247.050592] 0 pages cma reserved >[29247.050598] 0 pages hwpoisoned >[29247.050605] Unreclaimable slab info: >[29247.050611] Name Used Total >[29247.050778] xfs_efd_item 1017KB 1017KB >[29247.050787] xfs_buf_item 1020KB 1020KB >[29247.050795] xfs_trans 1022KB 1022KB >[29247.050803] xfs_da_state 1020KB 1020KB >[29247.050812] xfs_log_ticket 1023KB 1023KB >[29247.050825] scsi_sense_cache 1024KB 1024KB >[29247.050848] RAWv6 504KB 504KB >[29247.050857] UDPv6 1012KB 1012KB >[29247.050866] tw_sock_TCPv6 127KB 127KB >[29247.050875] TCPv6 1386KB 1386KB >[29247.050886] cfq_io_cq 1023KB 1023KB >[29247.050894] cfq_queue 1023KB 1023KB >[29247.050903] mqueue_inode_cache 64KB 64KB >[29247.050913] kioctx 63KB 63KB >[29247.050924] posix_timers_cache 1022KB 1022KB >[29247.050933] ip4-frags 1023KB 1023KB >[29247.050942] secpath_cache 768KB 768KB >[29247.050950] RAW 896KB 896KB >[29247.050958] UDP 1008KB 1008KB >[29247.050966] tw_sock_TCP 127KB 127KB >[29247.050975] request_sock_TCP 191KB 191KB >[29247.050983] TCP 2040KB 2040KB >[29247.050992] hugetlbfs_inode_cache 127KB 127KB >[29247.051000] eventpoll_pwq 1023KB 1023KB >[29247.051009] inotify_inode_mark 1023KB 1023KB >[29247.051018] request_queue 755KB 755KB >[29247.051027] blkdev_ioc 1023KB 1023KB >[29247.051036] biovec-128 256KB 256KB >[29247.051044] biovec-64 128KB 128KB >[29247.051052] skbuff_fclone_cache 1024KB 1024KB >[29247.051060] skbuff_head_cache 1600KB 1600KB >[29247.051068] file_lock_cache 1021KB 1021KB >[29247.051077] net_namespace 510KB 510KB >[29247.051086] shmem_inode_cache 1087KB 1087KB >[29247.051095] dtl 10040KB 11520KB >[29247.051103] taskstats 828KB 828KB >[29247.051112] proc_dir_entry 640KB 640KB >[29247.051120] pde_opener 1023KB 1023KB >[29247.051129] seq_file 1024KB 1024KB >[29247.051137] sigqueue 1022KB 1022KB >[29247.051146] kernfs_node_cache 5302KB 5302KB >[29247.051155] mnt_cache 2996KB 2996KB >[29247.051164] names_cache 4096KB 4096KB >[29247.051173] ebitmap_node 1024KB 1024KB >[29247.059962] avtab_node 1023KB 1023KB >[29247.059975] avc_xperms_data 1024KB 1024KB >[29247.059987] selinux_file_security 1024KB 1024KB >[29247.059999] selinux_inode_security 1855KB 1855KB >[29247.060013] nsproxy 1023KB 1023KB >[29247.060025] vm_area_struct 1660KB 1660KB >[29247.060037] mm_struct 8056KB 8056KB >[29247.060049] files_cache 1211KB 1211KB >[29247.060061] signal_cache 1472KB 1472KB >[29247.060073] sighand_cache 2932KB 2932KB >[29247.060085] task_struct 12210KB 12210KB >[29247.060097] cred_jar 4800KB 4800KB >[29247.060109] thread_stack 17408KB 17408KB >[29247.060121] anon_vma 1086KB 1086KB >[29247.060133] pid 1280KB 1280KB >[29247.060145] numa_policy 383KB 383KB >[29247.060157] trace_event_file 255KB 255KB >[29247.060168] ftrace_event_field 447KB 447KB >[29247.060180] pool_workqueue 1024KB 1024KB >[29247.060191] task_group 1020KB 1020KB >[29247.060202] pgtable-2^11 9216KB 9216KB >[29247.060213] pgtable-2^8 4096KB 4096KB >[29247.060226] dma-kmalloc-512 128KB 128KB >[29247.060241] kmalloc-131072 35328KB 35328KB >[29247.060252] kmalloc-65536 9216KB 9216KB >[29247.060263] kmalloc-32768 8192KB 8192KB >[29247.060274] kmalloc-16384 18432KB 20480KB >[29247.060285] kmalloc-8192 8704KB 8704KB >[29247.060296] kmalloc-4096 8192KB 8192KB >[29247.060306] kmalloc-2048 4352KB 4352KB >[29247.060318] kmalloc-1024 19512KB 19520KB >[29247.060330] kmalloc-512 10166KB 11008KB >[29247.060341] kmalloc-256 1408KB 1408KB >[29247.060352] kmalloc-192 1790KB 1790KB >[29247.060363] kmalloc-128 1536KB 1536KB >[29247.060374] kmalloc-96 1470KB 1470KB >[29247.060385] kmalloc-64 1792KB 1792KB >[29247.060395] kmalloc-32 12992KB 12992KB >[29247.060406] kmalloc-16 1024KB 1024KB >[29247.060417] kmalloc-8 1024KB 1024KB >[29247.060428] kmem_cache_node 1024KB 1024KB >[29247.060439] kmem_cache 3315KB 3315KB >[29247.060447] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29247.060471] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29247.060489] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29247.060505] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29247.060521] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29247.060536] [ 2069] 81 2069 208 0 51200 73 -900 dbus-daemon >[29247.060551] [ 2089] 0 2089 1288 0 51200 42 0 irqbalance >[29247.060565] [ 2094] 0 2094 10692 0 122880 86 0 rsyslogd >[29247.060578] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29247.060592] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29247.060605] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29247.060618] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29247.060631] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29247.060643] [ 2139] 0 2139 97 0 51200 37 0 atd >[29247.060656] [ 2140] 0 2140 3304 0 51200 55 0 crond >[29247.060668] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29247.060681] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29247.060695] [ 2337] 0 2337 449 0 53248 80 0 keepalived >[29247.060708] [ 2338] 0 2338 452 1 53248 77 0 keepalived >[29247.060722] [ 2695] 0 2695 91 0 49152 31 0 dovecot >[29247.060735] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29247.060748] [ 2697] 0 2697 87 0 49152 30 0 log >[29247.060760] [ 2699] 0 2699 103 0 51200 32 0 config >[29247.060774] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29247.060789] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29247.060803] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29247.060817] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29247.060829] [ 8909] 0 8909 3409 0 51200 65 0 ssh >[29247.060843] [ 8918] 990 8918 159 0 51200 30 0 pop3-login >[29247.060857] [ 8919] 0 8919 168 0 51200 45 0 auth >[29247.060869] [ 8920] 990 8920 159 0 51200 37 0 pop3-login >[29247.060884] [ 8932] 0 8932 264 0 40960 63 0 sshd >[29247.060896] [ 8933] 0 8933 3428 0 53248 63 0 crond >[29247.060910] [ 8937] 0 8937 127 0 38912 27 0 sshd >[29247.060921] Out of memory: Kill process 2094 (rsyslogd) score 1 or sacrifice child >[29247.060954] Killed process 2094 (rsyslogd) total-vm:684288kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29247.090390] oom_reaper: reaped process 2094 (rsyslogd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29247.090940] ssh invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29247.090955] ssh cpuset=/ mems_allowed=0 >[29247.090974] CPU: 3 PID: 8909 Comm: ssh Not tainted 4.18.7-200.fc28.ppc64 #1 >[29247.090986] Call Trace: >[29247.090998] [c000000009f37380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29247.091016] [c000000009f37410] [c0000000002fb670] .dump_header+0x70/0x300 >[29247.091031] [c000000009f374e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29247.091049] [c000000009f375b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29247.091070] [c000000009f37660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29247.091089] [c000000009f37850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29247.091108] [c000000009f378f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29247.091127] [c000000009f37980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29247.100180] [c000000009f37a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29247.100205] [c000000009f37b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29247.100225] [c000000009f37b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29247.100245] [c000000009f37cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29247.100265] [c000000009f37d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29247.100285] [c000000009f37e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29247.100310] Mem-Info: >[29247.100329] active_anon:0 inactive_anon:30 isolated_anon:0 > active_file:0 inactive_file:8 isolated_file:0 > unevictable:0 dirty:0 writeback:2 unstable:0 > slab_reclaimable:2799 slab_unreclaimable:15845 > mapped:0 shmem:0 pagetables:54 bounce:0 > free:1490 free_pcp:0 free_cma:0 >[29247.100381] Node 0 active_anon:0kB inactive_anon:1920kB active_file:0kB inactive_file:512kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:128kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29247.100410] Node 0 DMA free:95360kB min:94144kB low:117632kB high:141120kB active_anon:2496kB inactive_anon:128kB active_file:832kB inactive_file:0kB unevictable:0kB writepending:128kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9152kB pagetables:3456kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29247.100451] lowmem_reserve[]: 0 0 0 0 >[29247.100470] Node 0 DMA: 425*64kB (UE) 357*128kB (UME) 102*256kB (UME) 4*512kB (M) 1*1024kB (M) 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 102080kB >[29247.100528] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29247.100544] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29247.100557] 14 total pagecache pages >[29247.100566] 3 pages in swap cache >[29247.100575] Swap cache stats: add 2986952, delete 2987067, find 2534139/4299184 >[29247.100585] Free swap = 1884992kB >[29247.100592] Total swap = 2052032kB >[29247.100600] 32768 pages RAM >[29247.100606] 0 pages HighMem/MovableOnly >[29247.100613] 2684 pages reserved >[29247.100618] 0 pages cma reserved >[29247.100623] 0 pages hwpoisoned >[29247.100629] Unreclaimable slab info: >[29247.100634] Name Used Total >[29247.100684] xfs_efd_item 1017KB 1017KB >[29247.100693] xfs_buf_item 1020KB 1020KB >[29247.100701] xfs_trans 1022KB 1022KB >[29247.100708] xfs_da_state 1020KB 1020KB >[29247.100716] xfs_log_ticket 1023KB 1023KB >[29247.100726] scsi_sense_cache 1024KB 1024KB >[29247.100736] RAWv6 504KB 504KB >[29247.100743] UDPv6 1012KB 1012KB >[29247.100750] tw_sock_TCPv6 127KB 127KB >[29247.100757] TCPv6 1386KB 1386KB >[29247.100763] cfq_io_cq 1023KB 1023KB >[29247.100769] cfq_queue 1023KB 1023KB >[29247.100776] mqueue_inode_cache 64KB 64KB >[29247.100782] kioctx 63KB 63KB >[29247.100789] posix_timers_cache 1022KB 1022KB >[29247.100796] ip4-frags 1023KB 1023KB >[29247.100801] secpath_cache 768KB 768KB >[29247.100807] RAW 896KB 896KB >[29247.100813] UDP 1008KB 1008KB >[29247.100819] tw_sock_TCP 127KB 127KB >[29247.100825] request_sock_TCP 191KB 191KB >[29247.100830] TCP 2040KB 2040KB >[29247.100836] hugetlbfs_inode_cache 127KB 127KB >[29247.100842] eventpoll_pwq 1023KB 1023KB >[29247.100848] inotify_inode_mark 1023KB 1023KB >[29247.100854] request_queue 755KB 755KB >[29247.100860] blkdev_ioc 1023KB 1023KB >[29247.100866] biovec-128 256KB 256KB >[29247.100872] biovec-64 128KB 128KB >[29247.100878] skbuff_fclone_cache 1024KB 1024KB >[29247.100884] skbuff_head_cache 1600KB 1600KB >[29247.100890] file_lock_cache 1021KB 1021KB >[29247.100896] net_namespace 510KB 510KB >[29247.100902] shmem_inode_cache 1087KB 1087KB >[29247.100909] dtl 10040KB 11520KB >[29247.100916] taskstats 828KB 828KB >[29247.100923] proc_dir_entry 640KB 640KB >[29247.100929] pde_opener 1023KB 1023KB >[29247.100935] seq_file 1024KB 1024KB >[29247.100941] sigqueue 1022KB 1022KB >[29247.100947] kernfs_node_cache 5302KB 5302KB >[29247.100953] mnt_cache 2996KB 2996KB >[29247.100959] names_cache 4096KB 4096KB >[29247.100965] ebitmap_node 1024KB 1024KB >[29247.100971] avtab_node 1023KB 1023KB >[29247.100977] avc_xperms_data 1024KB 1024KB >[29247.100983] selinux_file_security 1024KB 1024KB >[29247.100989] selinux_inode_security 1855KB 1855KB >[29247.100996] nsproxy 1023KB 1023KB >[29247.101002] vm_area_struct 1660KB 1660KB >[29247.101007] mm_struct 8056KB 8056KB >[29247.101013] files_cache 1211KB 1211KB >[29247.101020] signal_cache 1472KB 1472KB >[29247.101026] sighand_cache 2932KB 2932KB >[29247.101032] task_struct 12210KB 12210KB >[29247.101038] cred_jar 4800KB 4800KB >[29247.101045] thread_stack 17408KB 17408KB >[29247.101050] anon_vma 1086KB 1086KB >[29247.101056] pid 1280KB 1280KB >[29247.101063] numa_policy 383KB 383KB >[29247.101069] trace_event_file 255KB 255KB >[29247.101074] ftrace_event_field 447KB 447KB >[29247.101079] pool_workqueue 1024KB 1024KB >[29247.101084] task_group 1020KB 1020KB >[29247.101089] pgtable-2^11 9216KB 9216KB >[29247.101093] pgtable-2^8 4096KB 4096KB >[29247.101100] dma-kmalloc-512 128KB 128KB >[29247.101106] kmalloc-131072 35328KB 35328KB >[29247.101111] kmalloc-65536 9216KB 9216KB >[29247.101116] kmalloc-32768 8192KB 8192KB >[29247.101121] kmalloc-16384 18432KB 20480KB >[29247.101126] kmalloc-8192 8704KB 8704KB >[29247.101131] kmalloc-4096 8192KB 8192KB >[29247.101135] kmalloc-2048 4352KB 4352KB >[29247.101141] kmalloc-1024 19512KB 19520KB >[29247.101147] kmalloc-512 10166KB 11008KB >[29247.101152] kmalloc-256 1408KB 1408KB >[29247.101157] kmalloc-192 1790KB 1790KB >[29247.101162] kmalloc-128 1536KB 1536KB >[29247.101166] kmalloc-96 1470KB 1470KB >[29247.101171] kmalloc-64 1792KB 1792KB >[29247.109840] kmalloc-32 12992KB 12992KB >[29247.109855] kmalloc-16 1024KB 1024KB >[29247.109864] kmalloc-8 1024KB 1024KB >[29247.109872] kmem_cache_node 1024KB 1024KB >[29247.109881] kmem_cache 3315KB 3315KB >[29247.109890] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29247.109914] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29247.109929] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29247.109940] [ 2067] 0 2067 328 0 51200 85 0 systemd-logind >[29247.109955] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29247.109965] [ 2069] 81 2069 208 0 51200 73 -900 dbus-daemon >[29247.109975] [ 2089] 0 2089 1288 0 51200 42 0 irqbalance >[29247.109988] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29247.110000] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29247.110011] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29247.110022] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29247.110034] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29247.110044] [ 2139] 0 2139 97 0 51200 37 0 atd >[29247.110054] [ 2140] 0 2140 3304 0 51200 55 0 crond >[29247.110066] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29247.110076] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29247.110088] [ 2337] 0 2337 449 0 53248 80 0 keepalived >[29247.110099] [ 2338] 0 2338 452 1 53248 77 0 keepalived >[29247.110111] [ 2695] 0 2695 91 0 49152 31 0 dovecot >[29247.110123] [ 2696] 97 2696 84 0 51200 16 0 anvil >[29247.110133] [ 2697] 0 2697 87 0 49152 30 0 log >[29247.110145] [ 2699] 0 2699 103 0 51200 32 0 config >[29247.110158] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29247.110170] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29247.110183] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29247.110194] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29247.110205] [ 8909] 0 8909 3409 0 51200 65 0 ssh >[29247.110217] [ 8918] 990 8918 159 0 51200 30 0 pop3-login >[29247.110230] [ 8919] 0 8919 168 0 51200 45 0 auth >[29247.110242] [ 8920] 990 8920 159 0 51200 37 0 pop3-login >[29247.110254] [ 8932] 0 8932 264 0 40960 63 0 sshd >[29247.110265] [ 8933] 0 8933 3428 0 53248 63 0 crond >[29247.110278] [ 8937] 0 8937 127 0 38912 27 0 sshd >[29247.110290] Out of memory: Kill process 2067 (systemd-logind) score 1 or sacrifice child >[29247.110315] Killed process 2067 (systemd-logind) total-vm:20992kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29247.140201] oom_reaper: reaped process 2067 (systemd-logind), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29250.619977] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29250.619999] sshd cpuset=/ mems_allowed=0 >[29250.620014] CPU: 14 PID: 8932 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29250.620021] Call Trace: >[29250.620031] [c00000002abd3230] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29250.620042] [c00000002abd32c0] [c0000000002fb670] .dump_header+0x70/0x300 >[29250.620051] [c00000002abd3390] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29250.620061] [c00000002abd3460] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29250.620071] [c00000002abd3510] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29250.620082] [c00000002abd3700] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29250.620092] [c00000002abd37c0] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29250.620102] [c00000002abd3880] [c00000000037a284] .swap_cluster_readahead+0x1e4/0x3e0 >[29250.620112] [c00000002abd3990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29250.620123] [c00000002abd3ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29250.620132] [c00000002abd3b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29250.620141] [c00000002abd3cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29250.620153] [c00000002abd3d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29250.620163] [c00000002abd3e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29250.620171] Mem-Info: >[29250.620182] active_anon:46 inactive_anon:6 isolated_anon:0 > active_file:0 inactive_file:44 isolated_file:0 > unevictable:0 dirty:0 writeback:19 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15848 > mapped:36 shmem:0 pagetables:50 bounce:0 > free:1465 free_pcp:0 free_cma:0 >[29250.620210] Node 0 active_anon:2944kB inactive_anon:384kB active_file:0kB inactive_file:2816kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2304kB dirty:0kB writeback:1216kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29250.620226] Node 0 DMA free:93760kB min:94144kB low:117632kB high:141120kB active_anon:704kB inactive_anon:256kB active_file:576kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9120kB pagetables:3200kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29250.620248] lowmem_reserve[]: 0 0 0 0 >[29250.620257] Node 0 DMA: 457*64kB (UME) 363*128kB (UME) 96*256kB (UME) 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100288kB >[29250.620285] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29250.620293] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29250.620300] 67 total pagecache pages >[29250.620305] 29 pages in swap cache >[29250.620311] Swap cache stats: add 2990414, delete 2990499, find 2537130/4304693 >[29250.620317] Free swap = 1892544kB >[29250.620322] Total swap = 2052032kB >[29250.620326] 32768 pages RAM >[29250.620330] 0 pages HighMem/MovableOnly >[29250.620334] 2684 pages reserved >[29250.620338] 0 pages cma reserved >[29250.620342] 0 pages hwpoisoned >[29250.620347] Unreclaimable slab info: >[29250.620352] Name Used Total >[29250.620523] xfs_efd_item 1017KB 1017KB >[29250.620530] xfs_buf_item 1020KB 1020KB >[29250.620537] xfs_trans 1022KB 1022KB >[29250.620543] xfs_da_state 1020KB 1020KB >[29250.620549] xfs_log_ticket 1023KB 1023KB >[29250.620560] scsi_sense_cache 1024KB 1024KB >[29250.620581] RAWv6 504KB 504KB >[29250.620588] UDPv6 1012KB 1012KB >[29250.620594] tw_sock_TCPv6 127KB 127KB >[29250.620601] TCPv6 1386KB 1386KB >[29250.620608] cfq_io_cq 1023KB 1023KB >[29250.620614] cfq_queue 1023KB 1023KB >[29250.620621] mqueue_inode_cache 64KB 64KB >[29250.620628] kioctx 63KB 63KB >[29250.620635] posix_timers_cache 1022KB 1022KB >[29250.620642] ip4-frags 1023KB 1023KB >[29250.620648] secpath_cache 768KB 768KB >[29250.620654] RAW 896KB 896KB >[29250.620660] UDP 1008KB 1008KB >[29250.620666] tw_sock_TCP 127KB 127KB >[29250.620672] request_sock_TCP 191KB 191KB >[29250.620678] TCP 2040KB 2040KB >[29250.620684] hugetlbfs_inode_cache 127KB 127KB >[29250.620691] eventpoll_pwq 1023KB 1023KB >[29250.620697] inotify_inode_mark 1023KB 1023KB >[29250.620703] request_queue 755KB 755KB >[29250.620710] blkdev_ioc 1023KB 1023KB >[29250.620716] biovec-128 256KB 256KB >[29250.620722] biovec-64 128KB 128KB >[29250.620729] skbuff_fclone_cache 1024KB 1024KB >[29250.620735] skbuff_head_cache 1600KB 1600KB >[29250.620741] file_lock_cache 1021KB 1021KB >[29250.620747] net_namespace 510KB 510KB >[29250.620753] shmem_inode_cache 1087KB 1087KB >[29250.620761] dtl 10040KB 11520KB >[29250.620767] taskstats 828KB 828KB >[29250.620773] proc_dir_entry 640KB 640KB >[29250.620779] pde_opener 1023KB 1023KB >[29250.620785] seq_file 1024KB 1024KB >[29250.620792] sigqueue 1022KB 1022KB >[29250.620798] kernfs_node_cache 5302KB 5302KB >[29250.620804] mnt_cache 2996KB 2996KB >[29250.620810] names_cache 4096KB 4096KB >[29250.620816] ebitmap_node 1024KB 1024KB >[29250.620822] avtab_node 1023KB 1023KB >[29250.620829] avc_xperms_data 1024KB 1024KB >[29250.620835] selinux_file_security 1024KB 1024KB >[29250.620841] selinux_inode_security 1855KB 1855KB >[29250.620848] nsproxy 1023KB 1023KB >[29250.620854] vm_area_struct 1660KB 1660KB >[29250.620860] mm_struct 8056KB 8056KB >[29250.620866] files_cache 1211KB 1211KB >[29250.620872] signal_cache 1472KB 1472KB >[29250.620878] sighand_cache 2932KB 2932KB >[29250.620884] task_struct 12210KB 12210KB >[29250.620890] cred_jar 4800KB 4800KB >[29250.620896] thread_stack 17408KB 17408KB >[29250.620902] anon_vma 1086KB 1086KB >[29250.620908] pid 1280KB 1280KB >[29250.620914] numa_policy 383KB 383KB >[29250.620920] trace_event_file 255KB 255KB >[29250.620926] ftrace_event_field 447KB 447KB >[29250.620932] pool_workqueue 1024KB 1024KB >[29250.620939] task_group 1020KB 1020KB >[29250.620944] pgtable-2^11 9216KB 9216KB >[29250.620950] pgtable-2^8 4096KB 4096KB >[29250.620959] dma-kmalloc-512 128KB 128KB >[29250.620967] kmalloc-131072 35328KB 35328KB >[29250.620973] kmalloc-65536 9216KB 9216KB >[29250.620979] kmalloc-32768 8192KB 8192KB >[29250.620986] kmalloc-16384 18432KB 20480KB >[29250.620992] kmalloc-8192 8704KB 8704KB >[29250.620998] kmalloc-4096 8192KB 8192KB >[29250.621003] kmalloc-2048 4352KB 4352KB >[29250.621010] kmalloc-1024 19512KB 19520KB >[29250.621018] kmalloc-512 10165KB 11008KB >[29250.621024] kmalloc-256 1408KB 1408KB >[29250.621030] kmalloc-192 1790KB 1790KB >[29250.621036] kmalloc-128 1536KB 1536KB >[29250.621042] kmalloc-96 1470KB 1470KB >[29250.621048] kmalloc-64 1792KB 1792KB >[29250.621054] kmalloc-32 12992KB 12992KB >[29250.621060] kmalloc-16 1024KB 1024KB >[29250.621066] kmalloc-8 1024KB 1024KB >[29250.621072] kmem_cache_node 1024KB 1024KB >[29250.621078] kmem_cache 3315KB 3315KB >[29250.621084] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29250.621106] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29250.621117] [ 2033] 0 2033 346 0 49152 72 -1000 auditd >[29250.621127] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29250.621136] [ 2069] 81 2069 208 2 51200 58 -900 dbus-daemon >[29250.621146] [ 2089] 0 2089 1288 1 51200 33 0 irqbalance >[29250.621156] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29250.621165] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29250.629938] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29250.629960] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29250.629970] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29250.629980] [ 2139] 0 2139 97 0 51200 37 0 atd >[29250.629989] [ 2140] 0 2140 3304 0 51200 55 0 crond >[29250.629998] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29250.630007] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29250.630017] [ 2337] 0 2337 449 0 53248 89 0 keepalived >[29250.630026] [ 2338] 0 2338 452 0 53248 80 0 keepalived >[29250.630036] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29250.630044] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29250.630053] [ 2697] 0 2697 87 0 49152 30 0 log >[29250.630061] [ 2699] 0 2699 103 0 51200 39 0 config >[29250.630072] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29250.630083] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29250.630095] [ 8876] 0 8876 3282 0 51200 30 0 copyToOther >[29250.630105] [ 8908] 0 8908 3270 0 51200 17 0 rsync >[29250.630113] [ 8909] 0 8909 3423 19 51200 45 0 ssh >[29250.630124] [ 8919] 97 8919 167 2 51200 33 0 auth >[29250.630133] [ 8920] 990 8920 159 0 51200 32 0 pop3-login >[29250.630143] [ 8932] 0 8932 292 2 40960 54 0 sshd >[29250.630152] [ 8933] 0 8933 3433 0 53248 84 0 crond >[29250.630161] [ 8937] 0 8937 275 0 40960 67 0 sshd >[29250.630169] Out of memory: Kill process 2337 (keepalived) score 1 or sacrifice child >[29250.630188] Killed process 2338 (keepalived) total-vm:28928kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29250.642292] oom_reaper: reaped process 2338 (keepalived), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29265.360753] copyToOther invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29265.360773] copyToOther cpuset=/ mems_allowed=0 >[29265.360786] CPU: 8 PID: 8876 Comm: copyToOther Not tainted 4.18.7-200.fc28.ppc64 #1 >[29265.360793] Call Trace: >[29265.360802] [c00000005792f380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29265.360813] [c00000005792f410] [c0000000002fb670] .dump_header+0x70/0x300 >[29265.360822] [c00000005792f4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29265.360830] [c00000005792f5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29265.360840] [c00000005792f660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29265.360849] [c00000005792f850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29265.360859] [c00000005792f8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29265.360868] [c00000005792f980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29265.360986] [c00000005792fa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29265.360996] [c00000005792fb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29265.361005] [c00000005792fb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29265.361013] [c00000005792fcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29265.361023] [c00000005792fd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29265.361033] [c00000005792fe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29265.361040] Mem-Info: >[29265.361050] active_anon:51 inactive_anon:91 isolated_anon:0 > active_file:0 inactive_file:66 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15854 > mapped:60 shmem:0 pagetables:49 bounce:0 > free:1471 free_pcp:1 free_cma:0 >[29265.361075] Node 0 active_anon:3264kB inactive_anon:5824kB active_file:0kB inactive_file:4224kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3840kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29265.361090] Node 0 DMA free:94144kB min:94144kB low:117632kB high:141120kB active_anon:1600kB inactive_anon:1856kB active_file:768kB inactive_file:1536kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9088kB pagetables:3136kB bounce:0kB free_pcp:64kB local_pcp:0kB free_cma:0kB >[29265.361110] lowmem_reserve[]: 0 0 0 0 >[29265.361118] Node 0 DMA: 456*64kB (UME) 365*128kB (UME) 96*256kB (UME) 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100480kB >[29265.361143] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29265.361151] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29265.361157] 224 total pagecache pages >[29265.369953] 28 pages in swap cache >[29265.369959] Swap cache stats: add 3003873, delete 3003962, find 2548982/4326357 >[29265.369965] Free swap = 1892608kB >[29265.369969] Total swap = 2052032kB >[29265.369974] 32768 pages RAM >[29265.369977] 0 pages HighMem/MovableOnly >[29265.369981] 2684 pages reserved >[29265.369986] 0 pages cma reserved >[29265.369990] 0 pages hwpoisoned >[29265.369994] Unreclaimable slab info: >[29265.369998] Name Used Total >[29265.370171] xfs_efd_item 1017KB 1017KB >[29265.370178] xfs_buf_item 1020KB 1020KB >[29265.370183] xfs_trans 1022KB 1022KB >[29265.370189] xfs_da_state 1020KB 1020KB >[29265.370194] xfs_log_ticket 1023KB 1023KB >[29265.370205] scsi_sense_cache 1024KB 1024KB >[29265.370226] RAWv6 504KB 504KB >[29265.370232] UDPv6 1012KB 1012KB >[29265.370238] tw_sock_TCPv6 127KB 127KB >[29265.370244] TCPv6 1386KB 1386KB >[29265.370251] cfq_io_cq 1023KB 1023KB >[29265.370257] cfq_queue 1023KB 1023KB >[29265.370263] mqueue_inode_cache 64KB 64KB >[29265.370270] kioctx 63KB 63KB >[29265.370276] posix_timers_cache 1022KB 1022KB >[29265.370282] ip4-frags 1023KB 1023KB >[29265.370288] secpath_cache 768KB 768KB >[29265.370293] RAW 896KB 896KB >[29265.370299] UDP 1008KB 1008KB >[29265.370304] tw_sock_TCP 127KB 127KB >[29265.370310] request_sock_TCP 191KB 191KB >[29265.370315] TCP 2040KB 2040KB >[29265.370321] hugetlbfs_inode_cache 127KB 127KB >[29265.370326] eventpoll_pwq 1023KB 1023KB >[29265.370332] inotify_inode_mark 1023KB 1023KB >[29265.370338] request_queue 755KB 755KB >[29265.370344] blkdev_ioc 1023KB 1023KB >[29265.370349] biovec-128 256KB 256KB >[29265.370355] biovec-64 128KB 128KB >[29265.370361] skbuff_fclone_cache 1024KB 1024KB >[29265.370366] skbuff_head_cache 1600KB 1600KB >[29265.370371] file_lock_cache 1021KB 1021KB >[29265.370377] net_namespace 510KB 510KB >[29265.370383] shmem_inode_cache 1087KB 1087KB >[29265.370389] dtl 10040KB 11520KB >[29265.370395] taskstats 828KB 828KB >[29265.370400] proc_dir_entry 640KB 640KB >[29265.370405] pde_opener 1023KB 1023KB >[29265.370411] seq_file 1024KB 1024KB >[29265.370416] sigqueue 1022KB 1022KB >[29265.370422] kernfs_node_cache 5302KB 5302KB >[29265.370427] mnt_cache 2996KB 2996KB >[29265.370433] names_cache 4096KB 4096KB >[29265.370439] ebitmap_node 1024KB 1024KB >[29265.370444] avtab_node 1023KB 1023KB >[29265.370450] avc_xperms_data 1024KB 1024KB >[29265.370456] selinux_file_security 1024KB 1024KB >[29265.370461] selinux_inode_security 1855KB 1855KB >[29265.370468] nsproxy 1023KB 1023KB >[29265.370473] vm_area_struct 1660KB 1660KB >[29265.370478] mm_struct 8056KB 8056KB >[29265.370484] files_cache 1211KB 1211KB >[29265.370489] signal_cache 1472KB 1472KB >[29265.370494] sighand_cache 2932KB 2932KB >[29265.370500] task_struct 12210KB 12210KB >[29265.370505] cred_jar 4800KB 4800KB >[29265.370510] thread_stack 17408KB 17408KB >[29265.370516] anon_vma 1086KB 1086KB >[29265.370521] pid 1280KB 1280KB >[29265.370527] numa_policy 383KB 383KB >[29265.370532] trace_event_file 255KB 255KB >[29265.370537] ftrace_event_field 447KB 447KB >[29265.370543] pool_workqueue 1024KB 1024KB >[29265.370549] task_group 1020KB 1020KB >[29265.370554] pgtable-2^11 9216KB 9216KB >[29265.370559] pgtable-2^8 4096KB 4096KB >[29265.370566] dma-kmalloc-512 128KB 128KB >[29265.370573] kmalloc-131072 35328KB 35328KB >[29265.370579] kmalloc-65536 9216KB 9216KB >[29265.370584] kmalloc-32768 8192KB 8192KB >[29265.370590] kmalloc-16384 18432KB 20480KB >[29265.370596] kmalloc-8192 8704KB 8704KB >[29265.370601] kmalloc-4096 8192KB 8192KB >[29265.370607] kmalloc-2048 4352KB 4352KB >[29265.370613] kmalloc-1024 19512KB 19520KB >[29265.370621] kmalloc-512 10165KB 11008KB >[29265.370626] kmalloc-256 1408KB 1408KB >[29265.370632] kmalloc-192 1790KB 1790KB >[29265.370638] kmalloc-128 1536KB 1536KB >[29265.370644] kmalloc-96 1470KB 1470KB >[29265.370649] kmalloc-64 1792KB 1792KB >[29265.370655] kmalloc-32 12992KB 12992KB >[29265.370660] kmalloc-16 1024KB 1024KB >[29265.370665] kmalloc-8 1024KB 1024KB >[29265.370671] kmem_cache_node 1024KB 1024KB >[29265.370677] kmem_cache 3315KB 3315KB >[29265.370682] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29265.370703] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29265.370713] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29265.370722] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29265.370731] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29265.370739] [ 2089] 0 2089 1288 0 51200 47 0 irqbalance >[29265.370749] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29265.370757] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29265.370765] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29265.370774] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29265.370783] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29265.370791] [ 2139] 0 2139 97 0 51200 37 0 atd >[29265.370799] [ 2140] 0 2140 3304 0 51200 55 0 crond >[29265.370808] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29265.370816] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29265.370825] [ 2337] 0 2337 449 1 53248 75 0 keepalived >[29265.370835] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29265.370843] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29265.370851] [ 2697] 0 2697 87 0 49152 30 0 log >[29265.370860] [ 2699] 0 2699 103 0 51200 39 0 config >[29265.370870] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29265.370880] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29265.370891] [ 8876] 0 8876 3282 0 51200 17 0 copyToOther >[29265.370901] [ 8919] 97 8919 167 0 51200 36 0 auth >[29265.370910] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29265.370919] [ 8932] 0 8932 302 12 40960 58 0 sshd >[29265.370927] [ 8933] 0 8933 3482 0 53248 97 0 crond >[29265.370935] [ 8937] 0 8937 302 8 40960 56 0 sshd >[29265.370942] Out of memory: Kill process 8933 (crond) score 1 or sacrifice child >[29265.370961] Killed process 8933 (crond) total-vm:222848kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29265.390102] oom_reaper: reaped process 8933 (crond), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29265.400282] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29265.400289] sshd cpuset=/ mems_allowed=0 >[29265.400297] CPU: 12 PID: 8937 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29265.400302] Call Trace: >[29265.400308] [c00000000e3db380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29265.400315] [c00000000e3db410] [c0000000002fb670] .dump_header+0x70/0x300 >[29265.400322] [c00000000e3db4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29265.400329] [c00000000e3db5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29265.400336] [c00000000e3db660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29265.400343] [c00000000e3db850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29265.400351] [c00000000e3db8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29265.400358] [c00000000e3db980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29265.400452] [c00000000e3dba50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29265.400460] [c00000000e3dbb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29265.400467] [c00000000e3dbb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29265.400474] [c00000000e3dbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29265.400481] [c00000000e3dbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29265.400490] [c00000000e3dbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29265.400495] Mem-Info: >[29265.400503] active_anon:46 inactive_anon:77 isolated_anon:0 > active_file:0 inactive_file:26 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15854 > mapped:18 shmem:0 pagetables:49 bounce:0 > free:1485 free_pcp:7 free_cma:0 >[29265.400523] Node 0 active_anon:2944kB inactive_anon:4928kB active_file:0kB inactive_file:1664kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1152kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29265.400536] Node 0 DMA free:95040kB min:94144kB low:117632kB high:141120kB active_anon:1280kB inactive_anon:2048kB active_file:1664kB inactive_file:640kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9088kB pagetables:3136kB bounce:0kB free_pcp:448kB local_pcp:0kB free_cma:0kB >[29265.400554] lowmem_reserve[]: 0 0 0 0 >[29265.400562] Node 0 DMA: 449*64kB (UME) 365*128kB (UME) 96*256kB (UME) 0*512kB 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100032kB >[29265.400582] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29265.400588] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29265.400594] 199 total pagecache pages >[29265.400598] 26 pages in swap cache >[29265.400602] Swap cache stats: add 3003877, delete 3003968, find 2548988/4326367 >[29265.400608] Free swap = 1896704kB >[29265.400612] Total swap = 2052032kB >[29265.400617] 32768 pages RAM >[29265.400620] 0 pages HighMem/MovableOnly >[29265.400622] 2684 pages reserved >[29265.400625] 0 pages cma reserved >[29265.400628] 0 pages hwpoisoned >[29265.400631] Unreclaimable slab info: >[29265.400633] Name Used Total >[29265.400716] xfs_efd_item 1017KB 1017KB >[29265.400722] xfs_buf_item 1020KB 1020KB >[29265.400726] xfs_trans 1022KB 1022KB >[29265.400731] xfs_da_state 1020KB 1020KB >[29265.400735] xfs_log_ticket 1023KB 1023KB >[29265.400743] scsi_sense_cache 1024KB 1024KB >[29265.400756] RAWv6 504KB 504KB >[29265.400762] UDPv6 1012KB 1012KB >[29265.400768] tw_sock_TCPv6 127KB 127KB >[29265.400774] TCPv6 1386KB 1386KB >[29265.400778] cfq_io_cq 1023KB 1023KB >[29265.400782] cfq_queue 1023KB 1023KB >[29265.400786] mqueue_inode_cache 64KB 64KB >[29265.400790] kioctx 63KB 63KB >[29265.400794] posix_timers_cache 1022KB 1022KB >[29265.400798] ip4-frags 1023KB 1023KB >[29265.400802] secpath_cache 768KB 768KB >[29265.400806] RAW 896KB 896KB >[29265.400810] UDP 1008KB 1008KB >[29265.400813] tw_sock_TCP 127KB 127KB >[29265.400817] request_sock_TCP 191KB 191KB >[29265.400821] TCP 2040KB 2040KB >[29265.400824] hugetlbfs_inode_cache 127KB 127KB >[29265.400828] eventpoll_pwq 1023KB 1023KB >[29265.400832] inotify_inode_mark 1023KB 1023KB >[29265.400836] request_queue 755KB 755KB >[29265.400840] blkdev_ioc 1023KB 1023KB >[29265.400844] biovec-128 256KB 256KB >[29265.400850] biovec-64 128KB 128KB >[29265.400855] skbuff_fclone_cache 1024KB 1024KB >[29265.400860] skbuff_head_cache 1600KB 1600KB >[29265.400865] file_lock_cache 1021KB 1021KB >[29265.400871] net_namespace 510KB 510KB >[29265.400876] shmem_inode_cache 1087KB 1087KB >[29265.400881] dtl 10040KB 11520KB >[29265.400886] taskstats 828KB 828KB >[29265.400892] proc_dir_entry 640KB 640KB >[29265.400898] pde_opener 1023KB 1023KB >[29265.400903] seq_file 1024KB 1024KB >[29265.400907] sigqueue 1022KB 1022KB >[29265.400911] kernfs_node_cache 5302KB 5302KB >[29265.400915] mnt_cache 2996KB 2996KB >[29265.400919] names_cache 4096KB 4096KB >[29265.400923] ebitmap_node 1024KB 1024KB >[29265.400926] avtab_node 1023KB 1023KB >[29265.400930] avc_xperms_data 1024KB 1024KB >[29265.400934] selinux_file_security 1024KB 1024KB >[29265.400938] selinux_inode_security 1855KB 1855KB >[29265.400942] nsproxy 1023KB 1023KB >[29265.400946] vm_area_struct 1660KB 1660KB >[29265.400949] mm_struct 8056KB 8056KB >[29265.400953] files_cache 1211KB 1211KB >[29265.400958] signal_cache 1472KB 1472KB >[29265.400964] sighand_cache 2932KB 2932KB >[29265.400969] task_struct 12210KB 12210KB >[29265.400974] cred_jar 4800KB 4800KB >[29265.400979] thread_stack 17408KB 17408KB >[29265.400984] anon_vma 1086KB 1086KB >[29265.400989] pid 1280KB 1280KB >[29265.400994] numa_policy 383KB 383KB >[29265.401000] trace_event_file 255KB 255KB >[29265.401005] ftrace_event_field 447KB 447KB >[29265.401010] pool_workqueue 1024KB 1024KB >[29265.401016] task_group 1020KB 1020KB >[29265.401020] pgtable-2^11 9216KB 9216KB >[29265.401024] pgtable-2^8 4096KB 4096KB >[29265.401029] dma-kmalloc-512 128KB 128KB >[29265.401034] kmalloc-131072 35328KB 35328KB >[29265.401037] kmalloc-65536 9216KB 9216KB >[29265.401041] kmalloc-32768 8192KB 8192KB >[29265.401045] kmalloc-16384 18432KB 20480KB >[29265.401049] kmalloc-8192 8704KB 8704KB >[29265.401053] kmalloc-4096 8192KB 8192KB >[29265.401056] kmalloc-2048 4352KB 4352KB >[29265.401061] kmalloc-1024 19512KB 19520KB >[29265.401065] kmalloc-512 10165KB 11008KB >[29265.401069] kmalloc-256 1408KB 1408KB >[29265.401074] kmalloc-192 1790KB 1790KB >[29265.401079] kmalloc-128 1536KB 1536KB >[29265.401084] kmalloc-96 1470KB 1470KB >[29265.401090] kmalloc-64 1792KB 1792KB >[29265.401095] kmalloc-32 12992KB 12992KB >[29265.401099] kmalloc-16 1024KB 1024KB >[29265.401104] kmalloc-8 1024KB 1024KB >[29265.401108] kmem_cache_node 1024KB 1024KB >[29265.401113] kmem_cache 3315KB 3315KB >[29265.401117] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29265.401130] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29265.401139] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29265.401147] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29265.401155] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29265.409957] [ 2089] 0 2089 1288 0 51200 47 0 irqbalance >[29265.409968] [ 2095] 0 2095 434 0 53248 83 0 rngd >[29265.409976] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29265.409983] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29265.409991] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29265.409997] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29265.410004] [ 2139] 0 2139 97 0 51200 37 0 atd >[29265.410011] [ 2140] 0 2140 3304 0 51200 55 0 crond >[29265.410019] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29265.410026] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29265.410035] [ 2337] 0 2337 449 1 53248 75 0 keepalived >[29265.410045] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29265.410053] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29265.410059] [ 2697] 0 2697 87 0 49152 30 0 log >[29265.410066] [ 2699] 0 2699 103 0 51200 39 0 config >[29265.410074] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29265.410082] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29265.410091] [ 8876] 0 8876 3282 0 51200 17 0 copyToOther >[29265.410100] [ 8919] 97 8919 167 0 51200 36 0 auth >[29265.410107] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29265.410114] [ 8932] 0 8932 302 12 40960 58 0 sshd >[29265.410121] [ 8937] 0 8937 302 4 40960 60 0 sshd >[29265.410127] Out of memory: Kill process 2095 (rngd) score 1 or sacrifice child >[29265.410137] Killed process 2095 (rngd) total-vm:27776kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29265.440053] oom_reaper: reaped process 2095 (rngd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29266.640831] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29266.640848] sshd cpuset=/ mems_allowed=0 >[29266.640859] CPU: 11 PID: 8937 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29266.640864] Call Trace: >[29266.640873] [c00000000e3db230] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29266.640881] [c00000000e3db2c0] [c0000000002fb670] .dump_header+0x70/0x300 >[29266.640888] [c00000000e3db390] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29266.640895] [c00000000e3db460] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29266.640902] [c00000000e3db510] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29266.640910] [c00000000e3db700] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29266.640918] [c00000000e3db7c0] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29266.640925] [c00000000e3db880] [c00000000037a284] .swap_cluster_readahead+0x1e4/0x3e0 >[29266.640933] [c00000000e3db990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29266.640940] [c00000000e3dbac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29266.640947] [c00000000e3dbb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29266.640954] [c00000000e3dbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29266.640962] [c00000000e3dbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29266.640970] [c00000000e3dbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29266.640976] Mem-Info: >[29266.640985] active_anon:0 inactive_anon:14 isolated_anon:0 > active_file:26 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15865 > mapped:50 shmem:0 pagetables:50 bounce:0 > free:1466 free_pcp:0 free_cma:0 >[29266.641005] Node 0 active_anon:0kB inactive_anon:896kB active_file:1664kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3200kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29266.641017] Node 0 DMA free:93824kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1088kB active_file:576kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9088kB pagetables:3200kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29266.641033] lowmem_reserve[]: 0 0 0 0 >[29266.641040] Node 0 DMA: 450*64kB (UME) 360*128kB (UME) 97*256kB (UME) 1*512kB (M) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 100224kB >[29266.641061] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29266.641067] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29266.641072] 174 total pagecache pages >[29266.641076] 18 pages in swap cache >[29266.641080] Swap cache stats: add 3004548, delete 3004647, find 2549554/4327409 >[29266.641084] Free swap = 1896704kB >[29266.641088] Total swap = 2052032kB >[29266.641091] 32768 pages RAM >[29266.641094] 0 pages HighMem/MovableOnly >[29266.641097] 2684 pages reserved >[29266.641101] 0 pages cma reserved >[29266.641104] 0 pages hwpoisoned >[29266.641108] Unreclaimable slab info: >[29266.641111] Name Used Total >[29266.650048] xfs_efd_item 1017KB 1017KB >[29266.650056] xfs_buf_item 1020KB 1020KB >[29266.650061] xfs_trans 1022KB 1022KB >[29266.650066] xfs_da_state 1020KB 1020KB >[29266.650070] xfs_log_ticket 1023KB 1023KB >[29266.650079] scsi_sense_cache 1024KB 1024KB >[29266.650095] RAWv6 504KB 504KB >[29266.650100] UDPv6 1012KB 1012KB >[29266.650105] tw_sock_TCPv6 127KB 127KB >[29266.650110] TCPv6 1386KB 1386KB >[29266.650115] cfq_io_cq 1023KB 1023KB >[29266.650119] cfq_queue 1023KB 1023KB >[29266.650124] mqueue_inode_cache 64KB 64KB >[29266.650129] kioctx 63KB 63KB >[29266.650135] posix_timers_cache 1022KB 1022KB >[29266.650139] ip4-frags 1023KB 1023KB >[29266.650144] secpath_cache 768KB 768KB >[29266.650148] RAW 896KB 896KB >[29266.650152] UDP 1008KB 1008KB >[29266.650157] tw_sock_TCP 127KB 127KB >[29266.650161] request_sock_TCP 191KB 191KB >[29266.650166] TCP 2040KB 2040KB >[29266.650170] hugetlbfs_inode_cache 127KB 127KB >[29266.650175] eventpoll_pwq 1023KB 1023KB >[29266.650179] inotify_inode_mark 1023KB 1023KB >[29266.650184] request_queue 755KB 755KB >[29266.650188] blkdev_ioc 1023KB 1023KB >[29266.650193] biovec-128 256KB 256KB >[29266.650197] biovec-64 128KB 128KB >[29266.650202] skbuff_fclone_cache 1024KB 1024KB >[29266.650206] skbuff_head_cache 1600KB 1600KB >[29266.650210] file_lock_cache 1021KB 1021KB >[29266.650215] net_namespace 510KB 510KB >[29266.650219] shmem_inode_cache 1087KB 1087KB >[29266.650225] dtl 10040KB 11520KB >[29266.650229] taskstats 828KB 828KB >[29266.650234] proc_dir_entry 640KB 640KB >[29266.650238] pde_opener 1023KB 1023KB >[29266.650242] seq_file 1024KB 1024KB >[29266.650247] sigqueue 1022KB 1022KB >[29266.650251] kernfs_node_cache 5302KB 5302KB >[29266.650256] mnt_cache 2996KB 2996KB >[29266.650260] names_cache 4096KB 4096KB >[29266.650265] ebitmap_node 1024KB 1024KB >[29266.650269] avtab_node 1023KB 1023KB >[29266.650274] avc_xperms_data 1024KB 1024KB >[29266.650278] selinux_file_security 1024KB 1024KB >[29266.650282] selinux_inode_security 1855KB 1855KB >[29266.650288] nsproxy 1023KB 1023KB >[29266.650292] vm_area_struct 1660KB 1660KB >[29266.650296] mm_struct 8056KB 8056KB >[29266.650301] files_cache 1211KB 1211KB >[29266.650305] signal_cache 1472KB 1472KB >[29266.650309] sighand_cache 2932KB 2932KB >[29266.650314] task_struct 12210KB 12210KB >[29266.650318] cred_jar 4800KB 4800KB >[29266.650322] thread_stack 17408KB 17408KB >[29266.650327] anon_vma 1086KB 1086KB >[29266.650331] pid 1280KB 1280KB >[29266.650336] numa_policy 383KB 383KB >[29266.650340] trace_event_file 255KB 255KB >[29266.650344] ftrace_event_field 447KB 447KB >[29266.650349] pool_workqueue 1024KB 1024KB >[29266.650354] task_group 1020KB 1020KB >[29266.650358] pgtable-2^11 9216KB 9216KB >[29266.650363] pgtable-2^8 4096KB 4096KB >[29266.650368] dma-kmalloc-512 128KB 128KB >[29266.650374] kmalloc-131072 35328KB 35328KB >[29266.650379] kmalloc-65536 9216KB 9216KB >[29266.650383] kmalloc-32768 8192KB 8192KB >[29266.650388] kmalloc-16384 18432KB 20480KB >[29266.650392] kmalloc-8192 8704KB 8704KB >[29266.650397] kmalloc-4096 8192KB 8192KB >[29266.650401] kmalloc-2048 4352KB 4352KB >[29266.650406] kmalloc-1024 19512KB 19520KB >[29266.650412] kmalloc-512 10165KB 11008KB >[29266.650416] kmalloc-256 1408KB 1408KB >[29266.650420] kmalloc-192 1790KB 1790KB >[29266.650425] kmalloc-128 1536KB 1536KB >[29266.650429] kmalloc-96 1470KB 1470KB >[29266.650433] kmalloc-64 1792KB 1792KB >[29266.650437] kmalloc-32 12992KB 12992KB >[29266.650442] kmalloc-16 1024KB 1024KB >[29266.650446] kmalloc-8 1024KB 1024KB >[29266.650451] kmem_cache_node 1024KB 1024KB >[29266.650455] kmem_cache 3315KB 3315KB >[29266.650459] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29266.650477] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29266.650485] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29266.650492] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29266.650499] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29266.650506] [ 2089] 0 2089 1288 0 51200 47 0 irqbalance >[29266.650514] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29266.650521] [ 2102] 0 2102 4079 0 57344 76 0 gssproxy >[29266.650528] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29266.650534] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29266.650541] [ 2139] 0 2139 97 0 51200 37 0 atd >[29266.650548] [ 2140] 0 2140 3304 0 51200 57 0 crond >[29266.650555] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29266.650561] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29266.650569] [ 2337] 0 2337 449 1 53248 75 0 keepalived >[29266.650577] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29266.650584] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29266.650590] [ 2697] 0 2697 87 0 49152 30 0 log >[29266.650597] [ 2699] 0 2699 103 0 51200 39 0 config >[29266.650605] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29266.650613] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29266.650623] [ 8876] 0 8876 3282 1 51200 12 0 copyToOther >[29266.650632] [ 8919] 97 8919 167 0 51200 36 0 auth >[29266.650638] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29266.650646] [ 8932] 0 8932 302 0 40960 72 0 sshd >[29266.650653] [ 8937] 0 8937 302 0 40960 71 0 sshd >[29266.650659] [ 8940] 0 8940 3269 0 51200 21 0 rsync >[29266.650665] Out of memory: Kill process 2102 (gssproxy) score 1 or sacrifice child >[29266.650698] Killed process 2102 (gssproxy) total-vm:261056kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29266.670007] oom_reaper: reaped process 2102 (gssproxy), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29269.940860] systemd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29269.940878] systemd cpuset=/ mems_allowed=0 >[29269.940892] CPU: 12 PID: 1 Comm: systemd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29269.940899] Call Trace: >[29269.940908] [c00000007e9ff380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29269.940918] [c00000007e9ff410] [c0000000002fb670] .dump_header+0x70/0x300 >[29269.940927] [c00000007e9ff4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29269.940935] [c00000007e9ff5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29269.940944] [c00000007e9ff660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29269.940953] [c00000007e9ff850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29269.940963] [c00000007e9ff8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29269.940972] [c00000007e9ff980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29269.941075] [c00000007e9ffa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29269.941085] [c00000007e9ffb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29269.941093] [c00000007e9ffb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29269.941102] [c00000007e9ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29269.941112] [c00000007e9ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29269.941121] [c00000007e9ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29269.941128] Mem-Info: >[29269.941138] active_anon:0 inactive_anon:36 isolated_anon:0 > active_file:50 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15886 > mapped:41 shmem:0 pagetables:49 bounce:0 > free:1453 free_pcp:0 free_cma:0 >[29269.949939] Node 0 active_anon:0kB inactive_anon:2304kB active_file:3200kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2624kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29269.949964] Node 0 DMA free:92992kB min:94144kB low:117632kB high:141120kB active_anon:1280kB inactive_anon:0kB active_file:0kB inactive_file:1344kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9024kB pagetables:3136kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29269.949985] lowmem_reserve[]: 0 0 0 0 >[29269.949993] Node 0 DMA: 380*64kB (UE) 363*128kB (UE) 102*256kB (UE) 3*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 98432kB >[29269.950018] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29269.950025] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29269.950031] 128 total pagecache pages >[29269.950036] 25 pages in swap cache >[29269.950041] Swap cache stats: add 3006724, delete 3006817, find 2551523/4331031 >[29269.950046] Free swap = 1904896kB >[29269.950050] Total swap = 2052032kB >[29269.950055] 32768 pages RAM >[29269.950058] 0 pages HighMem/MovableOnly >[29269.950062] 2684 pages reserved >[29269.950066] 0 pages cma reserved >[29269.950070] 0 pages hwpoisoned >[29269.950074] Unreclaimable slab info: >[29269.950078] Name Used Total >[29269.950246] xfs_efd_item 1017KB 1017KB >[29269.950253] xfs_buf_item 1020KB 1020KB >[29269.950258] xfs_trans 1022KB 1022KB >[29269.950264] xfs_da_state 1020KB 1020KB >[29269.950270] xfs_log_ticket 1023KB 1023KB >[29269.950282] scsi_sense_cache 1024KB 1024KB >[29269.950301] RAWv6 504KB 504KB >[29269.950307] UDPv6 1012KB 1012KB >[29269.950313] tw_sock_TCPv6 127KB 127KB >[29269.950319] TCPv6 1386KB 1386KB >[29269.950325] cfq_io_cq 1023KB 1023KB >[29269.950329] cfq_queue 1023KB 1023KB >[29269.950335] mqueue_inode_cache 64KB 64KB >[29269.950340] kioctx 63KB 63KB >[29269.950345] posix_timers_cache 1022KB 1022KB >[29269.950350] ip4-frags 1023KB 1023KB >[29269.950355] secpath_cache 768KB 768KB >[29269.950359] RAW 896KB 896KB >[29269.950363] UDP 1008KB 1008KB >[29269.950368] tw_sock_TCP 127KB 127KB >[29269.950372] request_sock_TCP 191KB 191KB >[29269.950377] TCP 2040KB 2040KB >[29269.950381] hugetlbfs_inode_cache 127KB 127KB >[29269.950386] eventpoll_pwq 1023KB 1023KB >[29269.950390] inotify_inode_mark 1023KB 1023KB >[29269.950395] request_queue 755KB 755KB >[29269.950399] blkdev_ioc 1023KB 1023KB >[29269.950404] biovec-128 256KB 256KB >[29269.950408] biovec-64 128KB 128KB >[29269.950413] skbuff_fclone_cache 1024KB 1024KB >[29269.950417] skbuff_head_cache 1600KB 1600KB >[29269.950421] file_lock_cache 1021KB 1021KB >[29269.950426] net_namespace 510KB 510KB >[29269.950430] shmem_inode_cache 1087KB 1087KB >[29269.950436] dtl 10040KB 11520KB >[29269.950440] taskstats 828KB 828KB >[29269.950445] proc_dir_entry 640KB 640KB >[29269.950449] pde_opener 1023KB 1023KB >[29269.950453] seq_file 1024KB 1024KB >[29269.950458] sigqueue 1022KB 1022KB >[29269.950462] kernfs_node_cache 5302KB 5302KB >[29269.950467] mnt_cache 2996KB 2996KB >[29269.950472] names_cache 4096KB 4096KB >[29269.950476] ebitmap_node 1024KB 1024KB >[29269.950481] avtab_node 1023KB 1023KB >[29269.950485] avc_xperms_data 1024KB 1024KB >[29269.950490] selinux_file_security 1024KB 1024KB >[29269.950494] selinux_inode_security 1855KB 1855KB >[29269.950499] nsproxy 1023KB 1023KB >[29269.950503] vm_area_struct 1660KB 1660KB >[29269.950508] mm_struct 8056KB 8056KB >[29269.950512] files_cache 1211KB 1211KB >[29269.950516] signal_cache 1472KB 1472KB >[29269.950520] sighand_cache 2932KB 2932KB >[29269.950525] task_struct 12210KB 12210KB >[29269.950529] cred_jar 4800KB 4800KB >[29269.950534] thread_stack 17408KB 17408KB >[29269.950538] anon_vma 1086KB 1086KB >[29269.950542] pid 1280KB 1280KB >[29269.950547] numa_policy 383KB 383KB >[29269.950551] trace_event_file 255KB 255KB >[29269.950555] ftrace_event_field 447KB 447KB >[29269.950560] pool_workqueue 1024KB 1024KB >[29269.950564] task_group 1020KB 1020KB >[29269.950569] pgtable-2^11 9216KB 9216KB >[29269.950573] pgtable-2^8 4096KB 4096KB >[29269.950579] dma-kmalloc-512 128KB 128KB >[29269.950585] kmalloc-131072 35328KB 35328KB >[29269.950589] kmalloc-65536 9216KB 9216KB >[29269.950593] kmalloc-32768 8192KB 8192KB >[29269.950598] kmalloc-16384 18432KB 20480KB >[29269.950603] kmalloc-8192 8704KB 8704KB >[29269.950607] kmalloc-4096 8192KB 8192KB >[29269.950612] kmalloc-2048 4352KB 4352KB >[29269.950617] kmalloc-1024 19512KB 19520KB >[29269.950623] kmalloc-512 10165KB 11008KB >[29269.950627] kmalloc-256 1408KB 1408KB >[29269.950632] kmalloc-192 1790KB 1790KB >[29269.950637] kmalloc-128 1536KB 1536KB >[29269.950641] kmalloc-96 1470KB 1470KB >[29269.950645] kmalloc-64 1792KB 1792KB >[29269.950649] kmalloc-32 12992KB 12992KB >[29269.950654] kmalloc-16 1024KB 1024KB >[29269.950658] kmalloc-8 1024KB 1024KB >[29269.950663] kmem_cache_node 1024KB 1024KB >[29269.950667] kmem_cache 3315KB 3315KB >[29269.950671] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29269.950689] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29269.950697] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29269.950705] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29269.950711] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29269.950718] [ 2089] 0 2089 1288 0 51200 41 0 irqbalance >[29269.950726] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29269.950734] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29269.950741] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29269.950748] [ 2139] 0 2139 97 0 51200 37 0 atd >[29269.950755] [ 2140] 0 2140 3304 0 51200 60 0 crond >[29269.950762] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29269.950769] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29269.950777] [ 2337] 0 2337 449 1 53248 75 0 keepalived >[29269.950785] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29269.950791] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29269.950798] [ 2697] 0 2697 87 0 49152 30 0 log >[29269.950804] [ 2699] 0 2699 103 0 51200 39 0 config >[29269.950812] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29269.950821] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29269.950830] [ 8876] 0 8876 3282 0 51200 14 0 copyToOther >[29269.950839] [ 8919] 97 8919 167 0 51200 36 0 auth >[29269.950846] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29269.950854] [ 8932] 0 8932 302 2 40960 62 0 sshd >[29269.950861] [ 8937] 0 8937 302 0 40960 76 0 sshd >[29269.950868] [ 8940] 0 8940 3270 0 51200 20 0 rsync >[29269.950874] [ 8941] 0 8941 203 0 40960 50 0 ssh >[29269.950880] Out of memory: Kill process 2337 (keepalived) score 1 or sacrifice child >[29269.950898] Killed process 2337 (keepalived) total-vm:28736kB, anon-rss:0kB, file-rss:64kB, shmem-rss:0kB >[29269.970076] oom_reaper: reaped process 2337 (keepalived), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29269.970202] ssh invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29269.970212] ssh cpuset=/ mems_allowed=0 >[29269.970223] CPU: 8 PID: 8941 Comm: ssh Not tainted 4.18.7-200.fc28.ppc64 #1 >[29269.970229] Call Trace: >[29269.970238] [c0000000277971a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29269.970249] [c000000027797230] [c0000000002fb670] .dump_header+0x70/0x300 >[29269.970259] [c000000027797300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29269.970269] [c0000000277973d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29269.970280] [c000000027797480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29269.970291] [c000000027797670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29269.970302] [c000000027797730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29269.970313] [c0000000277977f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29269.970323] [c000000027797880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29269.970334] [c000000027797990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29269.970344] [c000000027797ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29269.970354] [c000000027797b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29269.970365] [c000000027797cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29269.970376] [c000000027797d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29269.970387] [c000000027797e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29269.970394] Mem-Info: >[29269.970404] active_anon:0 inactive_anon:36 isolated_anon:0 > active_file:50 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2801 slab_unreclaimable:15886 > mapped:41 shmem:0 pagetables:49 bounce:0 > free:1452 free_pcp:2 free_cma:0 >[29269.970431] Node 0 active_anon:0kB inactive_anon:2304kB active_file:3200kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2624kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29269.970446] Node 0 DMA free:92928kB min:94144kB low:117632kB high:141120kB active_anon:1280kB inactive_anon:0kB active_file:0kB inactive_file:1344kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9024kB pagetables:3136kB bounce:0kB free_pcp:128kB local_pcp:0kB free_cma:0kB >[29269.970469] lowmem_reserve[]: 0 0 0 0 >[29269.970478] Node 0 DMA: 378*64kB (UE) 363*128kB (UE) 102*256kB (UE) 3*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 98304kB >[29269.970505] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29269.970514] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29269.970520] 129 total pagecache pages >[29269.970525] 25 pages in swap cache >[29269.970531] Swap cache stats: add 3006725, delete 3006818, find 2551524/4331033 >[29269.970537] Free swap = 1913088kB >[29269.970542] Total swap = 2052032kB >[29269.970547] 32768 pages RAM >[29269.970551] 0 pages HighMem/MovableOnly >[29269.970556] 2684 pages reserved >[29269.970560] 0 pages cma reserved >[29269.970564] 0 pages hwpoisoned >[29269.970568] Unreclaimable slab info: >[29269.970572] Name Used Total >[29269.970615] xfs_efd_item 1017KB 1017KB >[29269.970621] xfs_buf_item 1020KB 1020KB >[29269.970627] xfs_trans 1022KB 1022KB >[29269.970632] xfs_da_state 1020KB 1020KB >[29269.970637] xfs_log_ticket 1023KB 1023KB >[29269.970644] scsi_sense_cache 1024KB 1024KB >[29269.970654] RAWv6 504KB 504KB >[29269.970659] UDPv6 1012KB 1012KB >[29269.970665] tw_sock_TCPv6 127KB 127KB >[29269.970670] TCPv6 1386KB 1386KB >[29269.970676] cfq_io_cq 1023KB 1023KB >[29269.970681] cfq_queue 1023KB 1023KB >[29269.970687] mqueue_inode_cache 64KB 64KB >[29269.970693] kioctx 63KB 63KB >[29269.970699] posix_timers_cache 1022KB 1022KB >[29269.970705] ip4-frags 1023KB 1023KB >[29269.970710] secpath_cache 768KB 768KB >[29269.970715] RAW 896KB 896KB >[29269.970721] UDP 1008KB 1008KB >[29269.970726] tw_sock_TCP 127KB 127KB >[29269.970732] request_sock_TCP 191KB 191KB >[29269.970737] TCP 2040KB 2040KB >[29269.970742] hugetlbfs_inode_cache 127KB 127KB >[29269.970747] eventpoll_pwq 1023KB 1023KB >[29269.970753] inotify_inode_mark 1023KB 1023KB >[29269.970759] request_queue 755KB 755KB >[29269.970764] blkdev_ioc 1023KB 1023KB >[29269.970770] biovec-128 256KB 256KB >[29269.970775] biovec-64 128KB 128KB >[29269.970780] skbuff_fclone_cache 1024KB 1024KB >[29269.970786] skbuff_head_cache 1600KB 1600KB >[29269.970791] file_lock_cache 1021KB 1021KB >[29269.970796] net_namespace 510KB 510KB >[29269.970802] shmem_inode_cache 1087KB 1087KB >[29269.970807] dtl 10040KB 11520KB >[29269.970813] taskstats 828KB 828KB >[29269.970818] proc_dir_entry 640KB 640KB >[29269.970824] pde_opener 1023KB 1023KB >[29269.970829] seq_file 1024KB 1024KB >[29269.970834] sigqueue 1022KB 1022KB >[29269.970840] kernfs_node_cache 5302KB 5302KB >[29269.970845] mnt_cache 2996KB 2996KB >[29269.970851] names_cache 4096KB 4096KB >[29269.970856] ebitmap_node 1024KB 1024KB >[29269.970862] avtab_node 1023KB 1023KB >[29269.970867] avc_xperms_data 1024KB 1024KB >[29269.970872] selinux_file_security 1024KB 1024KB >[29269.970878] selinux_inode_security 1855KB 1855KB >[29269.970884] nsproxy 1023KB 1023KB >[29269.970889] vm_area_struct 1660KB 1660KB >[29269.970895] mm_struct 8056KB 8056KB >[29269.970900] files_cache 1211KB 1211KB >[29269.970905] signal_cache 1472KB 1472KB >[29269.970911] sighand_cache 2932KB 2932KB >[29269.970916] task_struct 12210KB 12210KB >[29269.970921] cred_jar 4800KB 4800KB >[29269.970927] thread_stack 17408KB 17408KB >[29269.970932] anon_vma 1086KB 1086KB >[29269.970938] pid 1280KB 1280KB >[29269.970943] numa_policy 383KB 383KB >[29269.970949] trace_event_file 255KB 255KB >[29269.970954] ftrace_event_field 447KB 447KB >[29269.970960] pool_workqueue 1024KB 1024KB >[29269.970965] task_group 1020KB 1020KB >[29269.970971] pgtable-2^11 9216KB 9216KB >[29269.970976] pgtable-2^8 4096KB 4096KB >[29269.970983] dma-kmalloc-512 128KB 128KB >[29269.970990] kmalloc-131072 35328KB 35328KB >[29269.970996] kmalloc-65536 9216KB 9216KB >[29269.971001] kmalloc-32768 8192KB 8192KB >[29269.971007] kmalloc-16384 18432KB 20480KB >[29269.971013] kmalloc-8192 8704KB 8704KB >[29269.971018] kmalloc-4096 8192KB 8192KB >[29269.971023] kmalloc-2048 4352KB 4352KB >[29269.971028] kmalloc-1024 19512KB 19520KB >[29269.971034] kmalloc-512 10165KB 11008KB >[29269.971040] kmalloc-256 1408KB 1408KB >[29269.971045] kmalloc-192 1790KB 1790KB >[29269.971051] kmalloc-128 1536KB 1536KB >[29269.971056] kmalloc-96 1470KB 1470KB >[29269.971061] kmalloc-64 1792KB 1792KB >[29269.971067] kmalloc-32 12992KB 12992KB >[29269.971072] kmalloc-16 1024KB 1024KB >[29269.971077] kmalloc-8 1024KB 1024KB >[29269.971083] kmem_cache_node 1024KB 1024KB >[29269.971088] kmem_cache 3315KB 3315KB >[29269.971093] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29269.971107] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29269.971116] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29269.971124] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29269.971131] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29269.971140] [ 2089] 0 2089 1288 0 51200 41 0 irqbalance >[29269.971148] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29269.971156] [ 2121] 0 2121 307 0 51200 72 -1000 sshd >[29269.971164] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29269.971171] [ 2139] 0 2139 97 0 51200 37 0 atd >[29269.979779] [ 2140] 0 2140 3304 0 51200 60 0 crond >[29269.979789] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29269.979797] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29269.979806] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29269.979813] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29269.979821] [ 2697] 0 2697 87 0 49152 30 0 log >[29269.979829] [ 2699] 0 2699 103 0 51200 39 0 config >[29269.979837] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29269.979846] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29269.979855] [ 8876] 0 8876 3282 0 51200 14 0 copyToOther >[29269.979864] [ 8919] 97 8919 167 0 51200 36 0 auth >[29269.979872] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29269.979881] [ 8932] 0 8932 302 2 40960 62 0 sshd >[29269.979889] [ 8937] 0 8937 302 0 40960 76 0 sshd >[29269.979897] [ 8940] 0 8940 3270 0 51200 20 0 rsync >[29269.979904] [ 8941] 0 8941 203 0 40960 50 0 ssh >[29269.979911] Out of memory: Kill process 8937 (sshd) score 1 or sacrifice child >[29269.979920] Killed process 8937 (sshd) total-vm:19328kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29270.000031] oom_reaper: reaped process 8937 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29271.920308] ssh invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29271.920321] ssh cpuset=/ mems_allowed=0 >[29271.920331] CPU: 12 PID: 8941 Comm: ssh Not tainted 4.18.7-200.fc28.ppc64 #1 >[29271.920336] Call Trace: >[29271.920343] [c0000000277971a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29271.920350] [c000000027797230] [c0000000002fb670] .dump_header+0x70/0x300 >[29271.920356] [c000000027797300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29271.920362] [c0000000277973d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29271.920368] [c000000027797480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29271.920376] [c000000027797670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29271.920383] [c000000027797730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29271.920390] [c0000000277977f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29271.920396] [c000000027797880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29271.920402] [c000000027797990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29271.920409] [c000000027797ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29271.920415] [c000000027797b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29271.920421] [c000000027797cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29271.920428] [c000000027797d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29271.920435] [c000000027797e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29271.920440] Mem-Info: >[29271.920447] active_anon:32 inactive_anon:40 isolated_anon:0 > active_file:33 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2797 slab_unreclaimable:15885 > mapped:30 shmem:0 pagetables:49 bounce:0 > free:1463 free_pcp:0 free_cma:0 >[29271.920465] Node 0 active_anon:2048kB inactive_anon:2560kB active_file:2112kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1920kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29271.920475] Node 0 DMA free:93632kB min:94144kB low:117632kB high:141120kB active_anon:192kB inactive_anon:256kB active_file:320kB inactive_file:0kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9008kB pagetables:3136kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29271.920489] lowmem_reserve[]: 0 0 0 0 >[29271.920495] Node 0 DMA: 362*64kB (UE) 363*128kB (UE) 101*256kB (UE) 4*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97536kB >[29271.920513] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29271.920518] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29271.920522] 97 total pagecache pages >[29271.920525] 30 pages in swap cache >[29271.920529] Swap cache stats: add 3007516, delete 3007604, find 2552325/4332423 >[29271.920533] Free swap = 1917184kB >[29271.920535] Total swap = 2052032kB >[29271.920538] 32768 pages RAM >[29271.920541] 0 pages HighMem/MovableOnly >[29271.920543] 2684 pages reserved >[29271.920546] 0 pages cma reserved >[29271.920549] 0 pages hwpoisoned >[29271.920552] Unreclaimable slab info: >[29271.920555] Name Used Total >[29271.920699] xfs_efd_item 1017KB 1017KB >[29271.920704] xfs_buf_item 1020KB 1020KB >[29271.920708] xfs_trans 1022KB 1022KB >[29271.920712] xfs_da_state 1020KB 1020KB >[29271.920716] xfs_log_ticket 1023KB 1023KB >[29271.920724] scsi_sense_cache 1024KB 1024KB >[29271.920739] RAWv6 504KB 504KB >[29271.920743] UDPv6 1012KB 1012KB >[29271.920747] tw_sock_TCPv6 127KB 127KB >[29271.920752] TCPv6 1386KB 1386KB >[29271.920756] cfq_io_cq 1023KB 1023KB >[29271.920760] cfq_queue 1023KB 1023KB >[29271.920765] mqueue_inode_cache 64KB 64KB >[29271.920769] kioctx 63KB 63KB >[29271.920774] posix_timers_cache 1022KB 1022KB >[29271.920777] ip4-frags 1023KB 1023KB >[29271.920781] secpath_cache 768KB 768KB >[29271.920785] RAW 896KB 896KB >[29271.920789] UDP 1008KB 1008KB >[29271.920793] tw_sock_TCP 127KB 127KB >[29271.920797] request_sock_TCP 191KB 191KB >[29271.920801] TCP 2040KB 2040KB >[29271.920805] hugetlbfs_inode_cache 127KB 127KB >[29271.920809] eventpoll_pwq 1023KB 1023KB >[29271.920812] inotify_inode_mark 1023KB 1023KB >[29271.920816] request_queue 755KB 755KB >[29271.920820] blkdev_ioc 1023KB 1023KB >[29271.920824] biovec-128 256KB 256KB >[29271.920828] biovec-64 128KB 128KB >[29271.920832] skbuff_fclone_cache 1024KB 1024KB >[29271.920836] skbuff_head_cache 1600KB 1600KB >[29271.920839] file_lock_cache 1021KB 1021KB >[29271.920843] net_namespace 510KB 510KB >[29271.920847] shmem_inode_cache 1087KB 1087KB >[29271.920852] dtl 10040KB 11520KB >[29271.920856] taskstats 828KB 828KB >[29271.920860] proc_dir_entry 640KB 640KB >[29271.920864] pde_opener 1023KB 1023KB >[29271.920868] seq_file 1024KB 1024KB >[29271.920872] sigqueue 1022KB 1022KB >[29271.920876] kernfs_node_cache 5302KB 5302KB >[29271.920880] mnt_cache 2996KB 2996KB >[29271.920884] names_cache 4096KB 4096KB >[29271.920888] ebitmap_node 1024KB 1024KB >[29271.920892] avtab_node 1023KB 1023KB >[29271.920896] avc_xperms_data 1024KB 1024KB >[29271.920900] selinux_file_security 1024KB 1024KB >[29271.920903] selinux_inode_security 1855KB 1855KB >[29271.920908] nsproxy 1023KB 1023KB >[29271.920912] vm_area_struct 1660KB 1660KB >[29271.920915] mm_struct 8056KB 8056KB >[29271.920919] files_cache 1211KB 1211KB >[29271.920922] signal_cache 1472KB 1472KB >[29271.920926] sighand_cache 2932KB 2932KB >[29271.920930] task_struct 12210KB 12210KB >[29271.920934] cred_jar 4800KB 4800KB >[29271.920938] thread_stack 17408KB 17408KB >[29271.920941] anon_vma 1086KB 1086KB >[29271.920945] pid 1280KB 1280KB >[29271.920949] numa_policy 383KB 383KB >[29271.920953] trace_event_file 255KB 255KB >[29271.920956] ftrace_event_field 447KB 447KB >[29271.920960] pool_workqueue 1024KB 1024KB >[29271.920964] task_group 1020KB 1020KB >[29271.920968] pgtable-2^11 9216KB 9216KB >[29271.920972] pgtable-2^8 4096KB 4096KB >[29271.920977] dma-kmalloc-512 128KB 128KB >[29271.920982] kmalloc-131072 35328KB 35328KB >[29271.920986] kmalloc-65536 9216KB 9216KB >[29271.920989] kmalloc-32768 8192KB 8192KB >[29271.920993] kmalloc-16384 18432KB 20480KB >[29271.920997] kmalloc-8192 8704KB 8704KB >[29271.921001] kmalloc-4096 8192KB 8192KB >[29271.921004] kmalloc-2048 4352KB 4352KB >[29271.921009] kmalloc-1024 19342KB 19456KB >[29271.921014] kmalloc-512 10164KB 11008KB >[29271.921018] kmalloc-256 1408KB 1408KB >[29271.921021] kmalloc-192 1790KB 1790KB >[29271.921025] kmalloc-128 1536KB 1536KB >[29271.921029] kmalloc-96 1470KB 1470KB >[29271.921032] kmalloc-64 1792KB 1792KB >[29271.921036] kmalloc-32 12992KB 12992KB >[29271.921040] kmalloc-16 1024KB 1024KB >[29271.921043] kmalloc-8 1024KB 1024KB >[29271.921047] kmem_cache_node 1024KB 1024KB >[29271.921051] kmem_cache 3315KB 3315KB >[29271.921054] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29271.921064] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29271.921071] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29271.921077] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29271.921082] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29271.921088] [ 2089] 0 2089 1288 0 51200 41 0 irqbalance >[29271.921095] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29271.921100] [ 2121] 0 2121 307 0 51200 88 -1000 sshd >[29271.921106] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29271.921112] [ 2139] 0 2139 97 0 51200 37 0 atd >[29271.921117] [ 2140] 0 2140 3304 0 51200 60 0 crond >[29271.921123] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29271.921129] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29271.921135] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29271.921141] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29271.921146] [ 2697] 0 2697 87 0 49152 30 0 log >[29271.921152] [ 2699] 0 2699 103 0 51200 39 0 config >[29271.921158] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29271.921164] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29271.921171] [ 8876] 0 8876 3282 0 51200 14 0 copyToOther >[29271.929950] [ 8919] 97 8919 167 0 51200 36 0 auth >[29271.929957] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29271.929964] [ 8932] 0 8932 302 2 40960 72 0 sshd >[29271.929971] [ 8940] 0 8940 3270 0 51200 20 0 rsync >[29271.929976] [ 8941] 0 8941 203 12 40960 36 0 ssh >[29271.929981] Out of memory: Kill process 8932 (sshd) score 1 or sacrifice child >[29271.929992] Killed process 8932 (sshd) total-vm:19328kB, anon-rss:0kB, file-rss:128kB, shmem-rss:0kB >[29271.950083] oom_reaper: reaped process 8932 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29273.879955] systemd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29273.879971] systemd cpuset=/ mems_allowed=0 >[29273.879981] CPU: 9 PID: 1 Comm: systemd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29273.879986] Call Trace: >[29273.879993] [c00000007e9ff1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29273.880000] [c00000007e9ff230] [c0000000002fb670] .dump_header+0x70/0x300 >[29273.880006] [c00000007e9ff300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29273.880013] [c00000007e9ff3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29273.880019] [c00000007e9ff480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29273.880027] [c00000007e9ff670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29273.880033] [c00000007e9ff730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29273.880040] [c00000007e9ff7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29273.880046] [c00000007e9ff880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29273.880053] [c00000007e9ff990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29273.880059] [c00000007e9ffac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29273.880065] [c00000007e9ffb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29273.880072] [c00000007e9ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29273.880079] [c00000007e9ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29273.880086] [c00000007e9ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29273.880091] Mem-Info: >[29273.880099] active_anon:16 inactive_anon:20 isolated_anon:0 > active_file:0 inactive_file:27 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15896 > mapped:15 shmem:0 pagetables:50 bounce:0 > free:1457 free_pcp:0 free_cma:0 >[29273.880117] Node 0 active_anon:1024kB inactive_anon:1280kB active_file:0kB inactive_file:1728kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:960kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29273.880127] Node 0 DMA free:93248kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:2432kB active_file:0kB inactive_file:256kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:9008kB pagetables:3200kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29273.880141] lowmem_reserve[]: 0 0 0 0 >[29273.880147] Node 0 DMA: 356*64kB (UE) 360*128kB (UE) 102*256kB (UE) 4*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97024kB >[29273.880165] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29273.880170] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29273.880175] 87 total pagecache pages >[29273.880178] 23 pages in swap cache >[29273.880182] Swap cache stats: add 3008395, delete 3008490, find 2553062/4333730 >[29273.880186] Free swap = 1921280kB >[29273.880188] Total swap = 2052032kB >[29273.880191] 32768 pages RAM >[29273.880194] 0 pages HighMem/MovableOnly >[29273.880197] 2684 pages reserved >[29273.880200] 0 pages cma reserved >[29273.880202] 0 pages hwpoisoned >[29273.880206] Unreclaimable slab info: >[29273.880209] Name Used Total >[29273.880354] xfs_efd_item 1017KB 1017KB >[29273.880359] xfs_buf_item 1020KB 1020KB >[29273.880363] xfs_trans 1022KB 1022KB >[29273.880367] xfs_da_state 1020KB 1020KB >[29273.880371] xfs_log_ticket 1023KB 1023KB >[29273.880379] scsi_sense_cache 1024KB 1024KB >[29273.880395] RAWv6 504KB 504KB >[29273.880399] UDPv6 1012KB 1012KB >[29273.880403] tw_sock_TCPv6 127KB 127KB >[29273.880407] TCPv6 1386KB 1386KB >[29273.880412] cfq_io_cq 1023KB 1023KB >[29273.880416] cfq_queue 1023KB 1023KB >[29273.880420] mqueue_inode_cache 64KB 64KB >[29273.880424] kioctx 63KB 63KB >[29273.880429] posix_timers_cache 1022KB 1022KB >[29273.880433] ip4-frags 1023KB 1023KB >[29273.880437] secpath_cache 768KB 768KB >[29273.880441] RAW 896KB 896KB >[29273.880445] UDP 1008KB 1008KB >[29273.880449] tw_sock_TCP 127KB 127KB >[29273.880452] request_sock_TCP 191KB 191KB >[29273.880456] TCP 2040KB 2040KB >[29273.880460] hugetlbfs_inode_cache 127KB 127KB >[29273.880464] eventpoll_pwq 1023KB 1023KB >[29273.880468] inotify_inode_mark 1023KB 1023KB >[29273.880472] request_queue 755KB 755KB >[29273.880476] blkdev_ioc 1023KB 1023KB >[29273.880480] biovec-128 256KB 256KB >[29273.880483] biovec-64 128KB 128KB >[29273.880487] skbuff_fclone_cache 1024KB 1024KB >[29273.880491] skbuff_head_cache 1600KB 1600KB >[29273.880495] file_lock_cache 1021KB 1021KB >[29273.880499] net_namespace 510KB 510KB >[29273.880503] shmem_inode_cache 1087KB 1087KB >[29273.880508] dtl 10040KB 11520KB >[29273.880511] taskstats 828KB 828KB >[29273.880515] proc_dir_entry 640KB 640KB >[29273.880519] pde_opener 1023KB 1023KB >[29273.880523] seq_file 1024KB 1024KB >[29273.880527] sigqueue 1022KB 1022KB >[29273.880531] kernfs_node_cache 5302KB 5302KB >[29273.880534] mnt_cache 2996KB 2996KB >[29273.880538] names_cache 4096KB 4096KB >[29273.880542] ebitmap_node 1024KB 1024KB >[29273.880546] avtab_node 1023KB 1023KB >[29273.880550] avc_xperms_data 1024KB 1024KB >[29273.880554] selinux_file_security 1024KB 1024KB >[29273.880557] selinux_inode_security 1855KB 1855KB >[29273.880562] nsproxy 1023KB 1023KB >[29273.880566] vm_area_struct 1660KB 1660KB >[29273.880570] mm_struct 8056KB 8056KB >[29273.880573] files_cache 1211KB 1211KB >[29273.880577] signal_cache 1472KB 1472KB >[29273.880581] sighand_cache 2932KB 2932KB >[29273.880584] task_struct 12210KB 12210KB >[29273.880588] cred_jar 4800KB 4800KB >[29273.880592] thread_stack 17408KB 17408KB >[29273.880596] anon_vma 1086KB 1086KB >[29273.880599] pid 1280KB 1280KB >[29273.880603] numa_policy 383KB 383KB >[29273.880607] trace_event_file 255KB 255KB >[29273.880611] ftrace_event_field 447KB 447KB >[29273.880615] pool_workqueue 1024KB 1024KB >[29273.880618] task_group 1020KB 1020KB >[29273.880622] pgtable-2^11 9216KB 9216KB >[29273.880626] pgtable-2^8 4096KB 4096KB >[29273.880631] dma-kmalloc-512 128KB 128KB >[29273.880636] kmalloc-131072 35328KB 35328KB >[29273.880639] kmalloc-65536 9216KB 9216KB >[29273.880643] kmalloc-32768 8192KB 8192KB >[29273.880648] kmalloc-16384 18432KB 20480KB >[29273.880651] kmalloc-8192 8704KB 8704KB >[29273.880655] kmalloc-4096 8192KB 8192KB >[29273.880659] kmalloc-2048 4352KB 4352KB >[29273.880662] kmalloc-1024 19456KB 19456KB >[29273.880667] kmalloc-512 10227KB 11008KB >[29273.880671] kmalloc-256 1408KB 1408KB >[29273.880675] kmalloc-192 1790KB 1790KB >[29273.880678] kmalloc-128 1536KB 1536KB >[29273.880682] kmalloc-96 1470KB 1470KB >[29273.880686] kmalloc-64 1792KB 1792KB >[29273.880690] kmalloc-32 12992KB 12992KB >[29273.880693] kmalloc-16 1024KB 1024KB >[29273.880697] kmalloc-8 1024KB 1024KB >[29273.880701] kmem_cache_node 1024KB 1024KB >[29273.880705] kmem_cache 3315KB 3315KB >[29273.880708] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29273.880724] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29273.880731] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29273.880738] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29273.880744] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29273.880750] [ 2089] 0 2089 1288 0 51200 42 0 irqbalance >[29273.880757] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29273.880763] [ 2121] 0 2121 307 0 51200 73 -1000 sshd >[29273.880769] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29273.880775] [ 2139] 0 2139 97 0 51200 37 0 atd >[29273.880781] [ 2140] 0 2140 3304 0 51200 60 0 crond >[29273.880787] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29273.880792] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29273.880800] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29273.880806] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29273.880812] [ 2697] 0 2697 87 0 49152 30 0 log >[29273.880818] [ 2699] 0 2699 103 0 51200 39 0 config >[29273.880824] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29273.880832] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29273.880840] [ 8876] 0 8876 3282 0 51200 14 0 copyToOther >[29273.880847] [ 8919] 97 8919 167 0 51200 36 0 auth >[29273.880852] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29273.880859] [ 8940] 0 8940 3270 0 51200 20 0 rsync >[29273.880864] [ 8941] 0 8941 205 34 51200 29 0 ssh >[29273.880870] [ 8943] 0 8943 210 17 40960 32 0 sshd >[29273.880874] Out of memory: Kill process 8941 (ssh) score 1 or sacrifice child >[29273.880884] Killed process 8941 (ssh) total-vm:13120kB, anon-rss:704kB, file-rss:1472kB, shmem-rss:0kB >[29273.900119] oom_reaper: reaped process 8941 (ssh), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29274.940474] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29274.940488] sshd cpuset=/ mems_allowed=0 >[29274.940499] CPU: 1 PID: 8943 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29274.940504] Call Trace: >[29274.940511] [c00000002abd31a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29274.940520] [c00000002abd3230] [c0000000002fb670] .dump_header+0x70/0x300 >[29274.940526] [c00000002abd3300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29274.940533] [c00000002abd33d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29274.940541] [c00000002abd3480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29274.940549] [c00000002abd3670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29274.940556] [c00000002abd3730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29274.940564] [c00000002abd37f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29274.940571] [c00000002abd3880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29274.940578] [c00000002abd3990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29274.940585] [c00000002abd3ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29274.940592] [c00000002abd3b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29274.940599] [c00000002abd3cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29274.940606] [c00000002abd3d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29274.940614] [c00000002abd3e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29274.940619] Mem-Info: >[29274.940628] active_anon:2 inactive_anon:17 isolated_anon:0 > active_file:6 inactive_file:2 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15918 > mapped:21 shmem:0 pagetables:51 bounce:0 > free:1464 free_pcp:0 free_cma:0 >[29274.940649] Node 0 active_anon:128kB inactive_anon:1088kB active_file:384kB inactive_file:128kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1344kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29274.940661] Node 0 DMA free:93696kB min:94144kB low:117632kB high:141120kB active_anon:128kB inactive_anon:1088kB active_file:384kB inactive_file:128kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8992kB pagetables:3264kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29274.940678] lowmem_reserve[]: 0 0 0 0 >[29274.940684] Node 0 DMA: 375*64kB (UE) 359*128kB (UE) 102*256kB (UE) 2*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97088kB >[29274.940704] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29274.940711] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29274.940716] 58 total pagecache pages >[29274.940719] 8 pages in swap cache >[29274.940723] Swap cache stats: add 3008715, delete 3008826, find 2553315/4334198 >[29274.940728] Free swap = 1925376kB >[29274.940731] Total swap = 2052032kB >[29274.940735] 32768 pages RAM >[29274.940737] 0 pages HighMem/MovableOnly >[29274.940741] 2684 pages reserved >[29274.940744] 0 pages cma reserved >[29274.940747] 0 pages hwpoisoned >[29274.940751] Unreclaimable slab info: >[29274.940754] Name Used Total >[29274.940892] xfs_efd_item 1017KB 1017KB >[29274.940897] xfs_buf_item 1020KB 1020KB >[29274.940901] xfs_trans 1022KB 1022KB >[29274.940906] xfs_da_state 1020KB 1020KB >[29274.940910] xfs_log_ticket 1023KB 1023KB >[29274.940919] scsi_sense_cache 1024KB 1024KB >[29274.940935] RAWv6 504KB 504KB >[29274.940939] UDPv6 1012KB 1012KB >[29274.940944] tw_sock_TCPv6 127KB 127KB >[29274.940949] TCPv6 1386KB 1386KB >[29274.940954] cfq_io_cq 1023KB 1023KB >[29274.940958] cfq_queue 1023KB 1023KB >[29274.940963] mqueue_inode_cache 64KB 64KB >[29274.940968] kioctx 63KB 63KB >[29274.940973] posix_timers_cache 1022KB 1022KB >[29274.940978] ip4-frags 1023KB 1023KB >[29274.940982] secpath_cache 768KB 768KB >[29274.940987] RAW 896KB 896KB >[29274.940991] UDP 1008KB 1008KB >[29274.940995] tw_sock_TCP 127KB 127KB >[29274.940999] request_sock_TCP 191KB 191KB >[29274.941004] TCP 2040KB 2040KB >[29274.941008] hugetlbfs_inode_cache 127KB 127KB >[29274.941013] eventpoll_pwq 1023KB 1023KB >[29274.941018] inotify_inode_mark 1023KB 1023KB >[29274.941022] request_queue 755KB 755KB >[29274.941026] blkdev_ioc 1023KB 1023KB >[29274.941030] biovec-128 256KB 256KB >[29274.941035] biovec-64 128KB 128KB >[29274.941039] skbuff_fclone_cache 1024KB 1024KB >[29274.941044] skbuff_head_cache 1600KB 1600KB >[29274.941049] file_lock_cache 1021KB 1021KB >[29274.941053] net_namespace 510KB 510KB >[29274.941058] shmem_inode_cache 1087KB 1087KB >[29274.941063] dtl 10040KB 11520KB >[29274.941068] taskstats 828KB 828KB >[29274.941072] proc_dir_entry 640KB 640KB >[29274.941076] pde_opener 1023KB 1023KB >[29274.941081] seq_file 1024KB 1024KB >[29274.941085] sigqueue 1022KB 1022KB >[29274.941090] kernfs_node_cache 5302KB 5302KB >[29274.941094] mnt_cache 2996KB 2996KB >[29274.941099] names_cache 4096KB 4096KB >[29274.941104] ebitmap_node 1024KB 1024KB >[29274.941108] avtab_node 1023KB 1023KB >[29274.941113] avc_xperms_data 1024KB 1024KB >[29274.941117] selinux_file_security 1024KB 1024KB >[29274.941121] selinux_inode_security 1855KB 1855KB >[29274.941127] nsproxy 1023KB 1023KB >[29274.941131] vm_area_struct 1660KB 1660KB >[29274.941135] mm_struct 8056KB 8056KB >[29274.941140] files_cache 1211KB 1211KB >[29274.941144] signal_cache 1472KB 1472KB >[29274.941148] sighand_cache 2932KB 2932KB >[29274.941153] task_struct 12210KB 12210KB >[29274.941157] cred_jar 4800KB 4800KB >[29274.941161] thread_stack 17408KB 17408KB >[29274.941166] anon_vma 1086KB 1086KB >[29274.941170] pid 1280KB 1280KB >[29274.949945] numa_policy 383KB 383KB >[29274.949956] trace_event_file 255KB 255KB >[29274.949960] ftrace_event_field 447KB 447KB >[29274.949964] pool_workqueue 1024KB 1024KB >[29274.949968] task_group 1020KB 1020KB >[29274.949972] pgtable-2^11 9216KB 9216KB >[29274.949974] pgtable-2^8 4096KB 4096KB >[29274.949978] dma-kmalloc-512 128KB 128KB >[29274.949982] kmalloc-131072 35328KB 35328KB >[29274.949985] kmalloc-65536 9216KB 9216KB >[29274.949988] kmalloc-32768 8192KB 8192KB >[29274.949991] kmalloc-16384 18432KB 20480KB >[29274.949994] kmalloc-8192 8704KB 8704KB >[29274.949997] kmalloc-4096 8192KB 8192KB >[29274.950000] kmalloc-2048 4352KB 4352KB >[29274.950003] kmalloc-1024 19456KB 19456KB >[29274.950007] kmalloc-512 10227KB 11008KB >[29274.950010] kmalloc-256 1408KB 1408KB >[29274.950012] kmalloc-192 1790KB 1790KB >[29274.950015] kmalloc-128 1536KB 1536KB >[29274.950018] kmalloc-96 1470KB 1470KB >[29274.950021] kmalloc-64 1792KB 1792KB >[29274.950024] kmalloc-32 12992KB 12992KB >[29274.950027] kmalloc-16 1024KB 1024KB >[29274.950030] kmalloc-8 1024KB 1024KB >[29274.950033] kmem_cache_node 1024KB 1024KB >[29274.950036] kmem_cache 3315KB 3315KB >[29274.950038] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29274.950052] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29274.950057] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29274.950062] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29274.950067] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29274.950071] [ 2089] 0 2089 1288 0 51200 42 0 irqbalance >[29274.950077] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29274.950081] [ 2121] 0 2121 307 0 51200 73 -1000 sshd >[29274.950086] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29274.950090] [ 2139] 0 2139 97 0 51200 37 0 atd >[29274.950095] [ 2140] 0 2140 3304 0 51200 60 0 crond >[29274.950099] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29274.950104] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29274.950110] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29274.950114] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29274.950119] [ 2697] 0 2697 87 0 49152 30 0 log >[29274.950123] [ 2699] 0 2699 103 0 51200 39 0 config >[29274.950128] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29274.950134] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29274.950140] [ 8876] 0 8876 3282 0 51200 21 0 copyToOther >[29274.950146] [ 8919] 97 8919 167 0 51200 36 0 auth >[29274.950150] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29274.950155] [ 8943] 0 8943 237 11 40960 41 0 sshd >[29274.950160] [ 8944] 0 8944 3250 0 51200 17 0 rm >[29274.950163] Out of memory: Kill process 2140 (crond) score 0 or sacrifice child >[29274.950173] Killed process 2140 (crond) total-vm:211456kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29274.970019] oom_reaper: reaped process 2140 (crond), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29275.031069] rm invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29275.031074] rm cpuset=/ mems_allowed=0 >[29275.031080] CPU: 10 PID: 8944 Comm: rm Not tainted 4.18.7-200.fc28.ppc64 #1 >[29275.031083] Call Trace: >[29275.031087] [c00000001701f380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29275.031093] [c00000001701f410] [c0000000002fb670] .dump_header+0x70/0x300 >[29275.031097] [c00000001701f4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29275.031102] [c00000001701f5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29275.031107] [c00000001701f660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29275.031112] [c00000001701f850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29275.031117] [c00000001701f8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29275.031122] [c00000001701f980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29275.039950] [c00000001701fa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29275.039957] [c00000001701fb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29275.039961] [c00000001701fb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29275.039966] [c00000001701fcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29275.039971] [c00000001701fd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29275.039976] [c00000001701fe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29275.039982] Mem-Info: >[29275.039988] active_anon:3 inactive_anon:16 isolated_anon:0 > active_file:10 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:8 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15918 > mapped:21 shmem:0 pagetables:51 bounce:0 > free:1469 free_pcp:3 free_cma:0 >[29275.040003] Node 0 active_anon:192kB inactive_anon:1024kB active_file:640kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1344kB dirty:0kB writeback:512kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29275.040010] Node 0 DMA free:94016kB min:94144kB low:117632kB high:141120kB active_anon:192kB inactive_anon:1024kB active_file:640kB inactive_file:0kB unevictable:0kB writepending:512kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8992kB pagetables:3264kB bounce:0kB free_pcp:192kB local_pcp:0kB free_cma:0kB >[29275.040021] lowmem_reserve[]: 0 0 0 0 >[29275.040026] Node 0 DMA: 373*64kB (UE) 359*128kB (UE) 102*256kB (UE) 2*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 96960kB >[29275.040038] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29275.040042] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29275.040046] 61 total pagecache pages >[29275.040048] 8 pages in swap cache >[29275.040051] Swap cache stats: add 3008717, delete 3008828, find 2553317/4334202 >[29275.040054] Free swap = 1929472kB >[29275.040056] Total swap = 2052032kB >[29275.040058] 32768 pages RAM >[29275.040060] 0 pages HighMem/MovableOnly >[29275.040062] 2684 pages reserved >[29275.040064] 0 pages cma reserved >[29275.040066] 0 pages hwpoisoned >[29275.040069] Unreclaimable slab info: >[29275.040071] Name Used Total >[29275.040148] xfs_efd_item 1017KB 1017KB >[29275.040151] xfs_buf_item 1020KB 1020KB >[29275.040154] xfs_trans 1022KB 1022KB >[29275.040157] xfs_da_state 1020KB 1020KB >[29275.040160] xfs_log_ticket 1023KB 1023KB >[29275.040165] scsi_sense_cache 1024KB 1024KB >[29275.040177] RAWv6 504KB 504KB >[29275.040180] UDPv6 1012KB 1012KB >[29275.040183] tw_sock_TCPv6 127KB 127KB >[29275.040186] TCPv6 1386KB 1386KB >[29275.040189] cfq_io_cq 1023KB 1023KB >[29275.040192] cfq_queue 1023KB 1023KB >[29275.040195] mqueue_inode_cache 64KB 64KB >[29275.040199] kioctx 63KB 63KB >[29275.040202] posix_timers_cache 1022KB 1022KB >[29275.040205] ip4-frags 1023KB 1023KB >[29275.040208] secpath_cache 768KB 768KB >[29275.040211] RAW 896KB 896KB >[29275.040214] UDP 1008KB 1008KB >[29275.040217] tw_sock_TCP 127KB 127KB >[29275.040220] request_sock_TCP 191KB 191KB >[29275.040223] TCP 2040KB 2040KB >[29275.040225] hugetlbfs_inode_cache 127KB 127KB >[29275.040228] eventpoll_pwq 1023KB 1023KB >[29275.040231] inotify_inode_mark 1023KB 1023KB >[29275.040234] request_queue 755KB 755KB >[29275.040237] blkdev_ioc 1023KB 1023KB >[29275.040240] biovec-128 256KB 256KB >[29275.040243] biovec-64 128KB 128KB >[29275.040246] skbuff_fclone_cache 1024KB 1024KB >[29275.040249] skbuff_head_cache 1600KB 1600KB >[29275.040252] file_lock_cache 1021KB 1021KB >[29275.040255] net_namespace 510KB 510KB >[29275.040258] shmem_inode_cache 1087KB 1087KB >[29275.040261] dtl 10040KB 11520KB >[29275.040264] taskstats 828KB 828KB >[29275.040267] proc_dir_entry 640KB 640KB >[29275.040270] pde_opener 1023KB 1023KB >[29275.040273] seq_file 1024KB 1024KB >[29275.040276] sigqueue 1022KB 1022KB >[29275.040278] kernfs_node_cache 5302KB 5302KB >[29275.040281] mnt_cache 2996KB 2996KB >[29275.040284] names_cache 4096KB 4096KB >[29275.040287] ebitmap_node 1024KB 1024KB >[29275.040290] avtab_node 1023KB 1023KB >[29275.040293] avc_xperms_data 1024KB 1024KB >[29275.040296] selinux_file_security 1024KB 1024KB >[29275.040299] selinux_inode_security 1855KB 1855KB >[29275.040302] nsproxy 1023KB 1023KB >[29275.040305] vm_area_struct 1660KB 1660KB >[29275.040308] mm_struct 8056KB 8056KB >[29275.040310] files_cache 1211KB 1211KB >[29275.040313] signal_cache 1472KB 1472KB >[29275.040316] sighand_cache 2932KB 2932KB >[29275.040319] task_struct 12210KB 12210KB >[29275.040322] cred_jar 4800KB 4800KB >[29275.040325] thread_stack 17408KB 17408KB >[29275.040328] anon_vma 1086KB 1086KB >[29275.040331] pid 1280KB 1280KB >[29275.040333] numa_policy 383KB 383KB >[29275.040336] trace_event_file 255KB 255KB >[29275.040339] ftrace_event_field 447KB 447KB >[29275.040342] pool_workqueue 1024KB 1024KB >[29275.040345] task_group 1020KB 1020KB >[29275.040348] pgtable-2^11 9216KB 9216KB >[29275.040351] pgtable-2^8 4096KB 4096KB >[29275.040355] dma-kmalloc-512 128KB 128KB >[29275.040358] kmalloc-131072 35328KB 35328KB >[29275.040361] kmalloc-65536 9216KB 9216KB >[29275.040364] kmalloc-32768 8192KB 8192KB >[29275.040367] kmalloc-16384 18432KB 20480KB >[29275.040370] kmalloc-8192 8704KB 8704KB >[29275.040372] kmalloc-4096 8192KB 8192KB >[29275.040375] kmalloc-2048 4352KB 4352KB >[29275.040378] kmalloc-1024 19456KB 19456KB >[29275.040381] kmalloc-512 10227KB 11008KB >[29275.040384] kmalloc-256 1408KB 1408KB >[29275.040387] kmalloc-192 1790KB 1790KB >[29275.040390] kmalloc-128 1536KB 1536KB >[29275.040392] kmalloc-96 1470KB 1470KB >[29275.040395] kmalloc-64 1792KB 1792KB >[29275.040398] kmalloc-32 12992KB 12992KB >[29275.040401] kmalloc-16 1024KB 1024KB >[29275.040404] kmalloc-8 1024KB 1024KB >[29275.040407] kmem_cache_node 1024KB 1024KB >[29275.040409] kmem_cache 3315KB 3315KB >[29275.040412] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29275.040420] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29275.040425] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29275.040429] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29275.040434] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29275.040438] [ 2089] 0 2089 1288 0 51200 42 0 irqbalance >[29275.040443] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29275.040447] [ 2121] 0 2121 307 0 51200 73 -1000 sshd >[29275.040451] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29275.040456] [ 2139] 0 2139 97 0 51200 37 0 atd >[29275.040460] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29275.040464] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29275.040469] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29275.040473] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29275.040477] [ 2697] 0 2697 87 0 49152 30 0 log >[29275.040482] [ 2699] 0 2699 103 0 51200 39 0 config >[29275.040486] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29275.040491] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29275.040496] [ 8876] 0 8876 3282 0 51200 21 0 copyToOther >[29275.040501] [ 8919] 97 8919 167 0 51200 36 0 auth >[29275.040505] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29275.040510] [ 8943] 0 8943 237 11 40960 41 0 sshd >[29275.040514] [ 8944] 0 8944 3250 0 51200 17 0 rm >[29275.040517] Out of memory: Kill process 8943 (sshd) score 0 or sacrifice child >[29275.040524] Killed process 8943 (sshd) total-vm:15168kB, anon-rss:0kB, file-rss:704kB, shmem-rss:0kB >[29275.060009] oom_reaper: reaped process 8943 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29276.360966] systemd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29276.360976] systemd cpuset=/ mems_allowed=0 >[29276.360983] CPU: 0 PID: 1 Comm: systemd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29276.360987] Call Trace: >[29276.360993] [c00000007e9ff1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29276.360999] [c00000007e9ff230] [c0000000002fb670] .dump_header+0x70/0x300 >[29276.361003] [c00000007e9ff300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29276.361008] [c00000007e9ff3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29276.361013] [c00000007e9ff480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29276.361018] [c00000007e9ff670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29276.361024] [c00000007e9ff730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29276.361028] [c00000007e9ff7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29276.361033] [c00000007e9ff880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29276.361040] [c00000007e9ff990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29276.361045] [c00000007e9ffac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29276.361050] [c00000007e9ffb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29276.361055] [c00000007e9ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29276.361060] [c00000007e9ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29276.361065] [c00000007e9ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29276.361079] Mem-Info: >[29276.361085] active_anon:0 inactive_anon:0 isolated_anon:0 > active_file:29 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:7 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15928 > mapped:29 shmem:0 pagetables:50 bounce:0 > free:1472 free_pcp:0 free_cma:0 >[29276.361099] Node 0 active_anon:0kB inactive_anon:0kB active_file:1856kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1856kB dirty:0kB writeback:448kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29276.361107] Node 0 DMA free:94208kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:1024kB active_file:320kB inactive_file:0kB unevictable:0kB writepending:448kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8976kB pagetables:3200kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29276.361118] lowmem_reserve[]: 0 0 0 0 >[29276.361122] Node 0 DMA: 363*64kB (UE) 357*128kB (UE) 100*256kB (UE) 3*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 96064kB >[29276.361136] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29276.361140] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29276.361143] 34 total pagecache pages >[29276.361146] 12 pages in swap cache >[29276.361148] Swap cache stats: add 3009158, delete 3009265, find 2553660/4334838 >[29276.361151] Free swap = 1927424kB >[29276.361153] Total swap = 2052032kB >[29276.361156] 32768 pages RAM >[29276.361157] 0 pages HighMem/MovableOnly >[29276.369953] 2684 pages reserved >[29276.369956] 0 pages cma reserved >[29276.369958] 0 pages hwpoisoned >[29276.369961] Unreclaimable slab info: >[29276.369963] Name Used Total >[29276.370116] xfs_efd_item 1017KB 1017KB >[29276.370119] xfs_buf_item 1020KB 1020KB >[29276.370123] xfs_trans 1022KB 1022KB >[29276.370126] xfs_da_state 1020KB 1020KB >[29276.370129] xfs_log_ticket 1023KB 1023KB >[29276.370136] scsi_sense_cache 1024KB 1024KB >[29276.370163] RAWv6 504KB 504KB >[29276.370166] UDPv6 1012KB 1012KB >[29276.370169] tw_sock_TCPv6 127KB 127KB >[29276.370172] TCPv6 1386KB 1386KB >[29276.370176] cfq_io_cq 1023KB 1023KB >[29276.370179] cfq_queue 1023KB 1023KB >[29276.370182] mqueue_inode_cache 64KB 64KB >[29276.370186] kioctx 63KB 63KB >[29276.370189] posix_timers_cache 1022KB 1022KB >[29276.370192] ip4-frags 1023KB 1023KB >[29276.370195] secpath_cache 768KB 768KB >[29276.370198] RAW 896KB 896KB >[29276.370201] UDP 1008KB 1008KB >[29276.370204] tw_sock_TCP 127KB 127KB >[29276.370207] request_sock_TCP 191KB 191KB >[29276.370210] TCP 2040KB 2040KB >[29276.370213] hugetlbfs_inode_cache 127KB 127KB >[29276.370217] eventpoll_pwq 1023KB 1023KB >[29276.370220] inotify_inode_mark 1023KB 1023KB >[29276.370223] request_queue 755KB 755KB >[29276.370226] blkdev_ioc 1023KB 1023KB >[29276.370229] biovec-128 256KB 256KB >[29276.370232] biovec-64 128KB 128KB >[29276.370235] skbuff_fclone_cache 1024KB 1024KB >[29276.370238] skbuff_head_cache 1600KB 1600KB >[29276.370241] file_lock_cache 1021KB 1021KB >[29276.370244] net_namespace 510KB 510KB >[29276.370247] shmem_inode_cache 1087KB 1087KB >[29276.370251] dtl 10040KB 11520KB >[29276.370262] taskstats 828KB 828KB >[29276.370265] proc_dir_entry 640KB 640KB >[29276.370268] pde_opener 1023KB 1023KB >[29276.370271] seq_file 1024KB 1024KB >[29276.370274] sigqueue 1022KB 1022KB >[29276.370277] kernfs_node_cache 5302KB 5302KB >[29276.370280] mnt_cache 2996KB 2996KB >[29276.370283] names_cache 4096KB 4096KB >[29276.370286] ebitmap_node 1024KB 1024KB >[29276.370297] avtab_node 1023KB 1023KB >[29276.370300] avc_xperms_data 1024KB 1024KB >[29276.370303] selinux_file_security 1024KB 1024KB >[29276.370305] selinux_inode_security 1855KB 1855KB >[29276.370309] nsproxy 1023KB 1023KB >[29276.370312] vm_area_struct 1660KB 1660KB >[29276.370315] mm_struct 8056KB 8056KB >[29276.370318] files_cache 1211KB 1211KB >[29276.370321] signal_cache 1472KB 1472KB >[29276.370323] sighand_cache 2932KB 2932KB >[29276.370326] task_struct 12210KB 12210KB >[29276.370329] cred_jar 4800KB 4800KB >[29276.370332] thread_stack 17408KB 17408KB >[29276.370335] anon_vma 1086KB 1086KB >[29276.370338] pid 1280KB 1280KB >[29276.370341] numa_policy 383KB 383KB >[29276.370344] trace_event_file 255KB 255KB >[29276.370347] ftrace_event_field 447KB 447KB >[29276.370350] pool_workqueue 1024KB 1024KB >[29276.370353] task_group 1020KB 1020KB >[29276.370356] pgtable-2^11 9216KB 9216KB >[29276.370359] pgtable-2^8 4096KB 4096KB >[29276.370363] dma-kmalloc-512 128KB 128KB >[29276.370367] kmalloc-131072 35328KB 35328KB >[29276.370370] kmalloc-65536 9216KB 9216KB >[29276.370372] kmalloc-32768 8192KB 8192KB >[29276.370376] kmalloc-16384 18432KB 20480KB >[29276.370387] kmalloc-8192 8704KB 8704KB >[29276.370390] kmalloc-4096 8192KB 8192KB >[29276.370393] kmalloc-2048 4352KB 4352KB >[29276.370395] kmalloc-1024 19520KB 19520KB >[29276.370399] kmalloc-512 10348KB 11008KB >[29276.370402] kmalloc-256 1408KB 1408KB >[29276.370405] kmalloc-192 1790KB 1790KB >[29276.370408] kmalloc-128 1536KB 1536KB >[29276.370411] kmalloc-96 1470KB 1470KB >[29276.370413] kmalloc-64 1792KB 1792KB >[29276.370416] kmalloc-32 12992KB 12992KB >[29276.370419] kmalloc-16 1024KB 1024KB >[29276.370422] kmalloc-8 1024KB 1024KB >[29276.370425] kmem_cache_node 1024KB 1024KB >[29276.370428] kmem_cache 3315KB 3315KB >[29276.370430] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29276.370444] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29276.370450] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29276.370455] [ 2068] 0 2068 158 0 51200 49 0 smartd >[29276.370459] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29276.370464] [ 2089] 0 2089 1288 0 51200 42 0 irqbalance >[29276.370469] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29276.370483] [ 2121] 0 2121 307 0 51200 70 -1000 sshd >[29276.370488] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29276.370492] [ 2139] 0 2139 97 0 51200 37 0 atd >[29276.370497] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29276.370501] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29276.370507] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29276.370512] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29276.370516] [ 2697] 0 2697 87 0 49152 30 0 log >[29276.370521] [ 2699] 0 2699 103 0 51200 39 0 config >[29276.370526] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29276.370531] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29276.370539] [ 8919] 97 8919 167 0 51200 36 0 auth >[29276.370543] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29276.370549] [ 8945] 0 8945 133 9 43008 17 0 sshd >[29276.370553] Out of memory: Kill process 2068 (smartd) score 0 or sacrifice child >[29276.370562] Killed process 2068 (smartd) total-vm:10112kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29276.390011] oom_reaper: reaped process 2068 (smartd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29276.940649] irqbalance invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29276.940660] irqbalance cpuset=/ mems_allowed=0 >[29276.940669] CPU: 1 PID: 2089 Comm: irqbalance Not tainted 4.18.7-200.fc28.ppc64 #1 >[29276.940673] Call Trace: >[29276.940680] [c0000000607db1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29276.940687] [c0000000607db230] [c0000000002fb670] .dump_header+0x70/0x300 >[29276.940693] [c0000000607db300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29276.940699] [c0000000607db3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29276.940705] [c0000000607db480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29276.940712] [c0000000607db670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29276.940719] [c0000000607db730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29276.940726] [c0000000607db7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29276.940732] [c0000000607db880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29276.940738] [c0000000607db990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29276.940745] [c0000000607dbac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29276.940751] [c0000000607dbb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29276.940757] [c0000000607dbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29276.940764] [c0000000607dbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29276.940771] [c0000000607dbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29276.940776] Mem-Info: >[29276.940783] active_anon:3 inactive_anon:18 isolated_anon:0 > active_file:37 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15928 > mapped:57 shmem:0 pagetables:49 bounce:0 > free:1467 free_pcp:0 free_cma:0 >[29276.940801] Node 0 active_anon:192kB inactive_anon:1152kB active_file:2368kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3648kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29276.940811] Node 0 DMA free:93888kB min:94144kB low:117632kB high:141120kB active_anon:192kB inactive_anon:1152kB active_file:768kB inactive_file:640kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8976kB pagetables:3136kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29276.940825] lowmem_reserve[]: 0 0 0 0 >[29276.940831] Node 0 DMA: 389*64kB (UE) 357*128kB (UE) 100*256kB (UE) 3*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97728kB >[29276.940849] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29276.940854] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29276.940859] 68 total pagecache pages >[29276.940862] 8 pages in swap cache >[29276.940865] Swap cache stats: add 3009319, delete 3009430, find 2553778/4335065 >[29276.940869] Free swap = 1931520kB >[29276.940872] Total swap = 2052032kB >[29276.940875] 32768 pages RAM >[29276.940877] 0 pages HighMem/MovableOnly >[29276.940880] 2684 pages reserved >[29276.940882] 0 pages cma reserved >[29276.940885] 0 pages hwpoisoned >[29276.940888] Unreclaimable slab info: >[29276.940891] Name Used Total >[29276.941001] xfs_efd_item 1017KB 1017KB >[29276.941005] xfs_buf_item 1020KB 1020KB >[29276.941009] xfs_trans 1022KB 1022KB >[29276.941013] xfs_da_state 1020KB 1020KB >[29276.941017] xfs_log_ticket 1023KB 1023KB >[29276.941023] scsi_sense_cache 1024KB 1024KB >[29276.941037] RAWv6 504KB 504KB >[29276.941041] UDPv6 1012KB 1012KB >[29276.941045] tw_sock_TCPv6 127KB 127KB >[29276.941049] TCPv6 1386KB 1386KB >[29276.941054] cfq_io_cq 1023KB 1023KB >[29276.941057] cfq_queue 1023KB 1023KB >[29276.941062] mqueue_inode_cache 64KB 64KB >[29276.941066] kioctx 63KB 63KB >[29276.941070] posix_timers_cache 1022KB 1022KB >[29276.941074] ip4-frags 1023KB 1023KB >[29276.941078] secpath_cache 768KB 768KB >[29276.941082] RAW 896KB 896KB >[29276.941085] UDP 1008KB 1008KB >[29276.941089] tw_sock_TCP 127KB 127KB >[29276.941093] request_sock_TCP 191KB 191KB >[29276.941097] TCP 2040KB 2040KB >[29276.941100] hugetlbfs_inode_cache 127KB 127KB >[29276.941104] eventpoll_pwq 1023KB 1023KB >[29276.941108] inotify_inode_mark 1023KB 1023KB >[29276.941112] request_queue 755KB 755KB >[29276.941116] blkdev_ioc 1023KB 1023KB >[29276.941120] biovec-128 256KB 256KB >[29276.941123] biovec-64 128KB 128KB >[29276.941127] skbuff_fclone_cache 1024KB 1024KB >[29276.941131] skbuff_head_cache 1600KB 1600KB >[29276.941135] file_lock_cache 1021KB 1021KB >[29276.941139] net_namespace 510KB 510KB >[29276.941142] shmem_inode_cache 1087KB 1087KB >[29276.941147] dtl 10040KB 11520KB >[29276.941151] taskstats 828KB 828KB >[29276.941154] proc_dir_entry 640KB 640KB >[29276.949935] pde_opener 1023KB 1023KB >[29276.949954] seq_file 1024KB 1024KB >[29276.949959] sigqueue 1022KB 1022KB >[29276.949964] kernfs_node_cache 5302KB 5302KB >[29276.949968] mnt_cache 2996KB 2996KB >[29276.949973] names_cache 4096KB 4096KB >[29276.949977] ebitmap_node 1024KB 1024KB >[29276.949981] avtab_node 1023KB 1023KB >[29276.949984] avc_xperms_data 1024KB 1024KB >[29276.949988] selinux_file_security 1024KB 1024KB >[29276.949992] selinux_inode_security 1855KB 1855KB >[29276.949996] nsproxy 1023KB 1023KB >[29276.950000] vm_area_struct 1660KB 1660KB >[29276.950003] mm_struct 8056KB 8056KB >[29276.950007] files_cache 1211KB 1211KB >[29276.950011] signal_cache 1472KB 1472KB >[29276.950015] sighand_cache 2932KB 2932KB >[29276.950019] task_struct 12210KB 12210KB >[29276.950023] cred_jar 4800KB 4800KB >[29276.950026] thread_stack 17408KB 17408KB >[29276.950030] anon_vma 1086KB 1086KB >[29276.950034] pid 1280KB 1280KB >[29276.950037] numa_policy 383KB 383KB >[29276.950041] trace_event_file 255KB 255KB >[29276.950045] ftrace_event_field 447KB 447KB >[29276.950049] pool_workqueue 1024KB 1024KB >[29276.950053] task_group 1020KB 1020KB >[29276.950057] pgtable-2^11 9216KB 9216KB >[29276.950060] pgtable-2^8 4096KB 4096KB >[29276.950065] dma-kmalloc-512 128KB 128KB >[29276.950070] kmalloc-131072 35328KB 35328KB >[29276.950074] kmalloc-65536 9216KB 9216KB >[29276.950078] kmalloc-32768 8192KB 8192KB >[29276.950082] kmalloc-16384 18432KB 20480KB >[29276.950085] kmalloc-8192 8704KB 8704KB >[29276.950089] kmalloc-4096 8192KB 8192KB >[29276.950093] kmalloc-2048 4352KB 4352KB >[29276.950097] kmalloc-1024 19520KB 19520KB >[29276.950101] kmalloc-512 10348KB 11008KB >[29276.950105] kmalloc-256 1408KB 1408KB >[29276.950109] kmalloc-192 1790KB 1790KB >[29276.950112] kmalloc-128 1536KB 1536KB >[29276.950116] kmalloc-96 1470KB 1470KB >[29276.950120] kmalloc-64 1792KB 1792KB >[29276.950123] kmalloc-32 12992KB 12992KB >[29276.950127] kmalloc-16 1024KB 1024KB >[29276.950130] kmalloc-8 1024KB 1024KB >[29276.950134] kmem_cache_node 1024KB 1024KB >[29276.950138] kmem_cache 3315KB 3315KB >[29276.950142] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29276.950158] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29276.950165] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29276.950172] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29276.950178] [ 2089] 0 2089 1288 0 51200 36 0 irqbalance >[29276.950185] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29276.950191] [ 2121] 0 2121 307 0 51200 70 -1000 sshd >[29276.950197] [ 2125] 0 2125 216 1 49152 45 0 xinetd >[29276.950203] [ 2139] 0 2139 97 0 51200 37 0 atd >[29276.950209] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29276.950215] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29276.950222] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29276.950228] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29276.950234] [ 2697] 0 2697 87 0 49152 30 0 log >[29276.950239] [ 2699] 0 2699 103 0 51200 39 0 config >[29276.950246] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29276.950253] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29276.950263] [ 8919] 97 8919 167 0 51200 36 0 auth >[29276.950269] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29276.950277] [ 8945] 0 8945 169 0 43008 40 0 sshd >[29276.950281] Out of memory: Kill process 2125 (xinetd) score 0 or sacrifice child >[29276.950296] Killed process 2125 (xinetd) total-vm:13824kB, anon-rss:0kB, file-rss:64kB, shmem-rss:0kB >[29276.970024] oom_reaper: reaped process 2125 (xinetd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29277.030300] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29277.030305] sshd cpuset=/ mems_allowed=0 >[29277.030310] CPU: 11 PID: 8945 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29277.030313] Call Trace: >[29277.030317] [c00000000e3df1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29277.030322] [c00000000e3df230] [c0000000002fb670] .dump_header+0x70/0x300 >[29277.030326] [c00000000e3df300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29277.030331] [c00000000e3df3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29277.030336] [c00000000e3df480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29277.030341] [c00000000e3df670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29277.030345] [c00000000e3df730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29277.030350] [c00000000e3df7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29277.030355] [c00000000e3df880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29277.030360] [c00000000e3df990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29277.030364] [c00000000e3dfac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29277.030369] [c00000000e3dfb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29277.030373] [c00000000e3dfcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29277.030378] [c00000000e3dfd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29277.030383] [c00000000e3dfe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29277.030387] Mem-Info: >[29277.030392] active_anon:3 inactive_anon:18 isolated_anon:0 > active_file:37 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15928 > mapped:57 shmem:0 pagetables:49 bounce:0 > free:1465 free_pcp:0 free_cma:0 >[29277.030406] Node 0 active_anon:192kB inactive_anon:1152kB active_file:2368kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3648kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29277.030413] Node 0 DMA free:93760kB min:94144kB low:117632kB high:141120kB active_anon:192kB inactive_anon:1152kB active_file:768kB inactive_file:640kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8976kB pagetables:3136kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29277.030424] lowmem_reserve[]: 0 0 0 0 >[29277.030428] Node 0 DMA: 386*64kB (UE) 357*128kB (UE) 100*256kB (UE) 3*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97536kB >[29277.030441] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29277.030445] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29277.030448] 70 total pagecache pages >[29277.030450] 8 pages in swap cache >[29277.030453] Swap cache stats: add 3009321, delete 3009432, find 2553779/4335068 >[29277.030456] Free swap = 1931520kB >[29277.030458] Total swap = 2052032kB >[29277.030460] 32768 pages RAM >[29277.030462] 0 pages HighMem/MovableOnly >[29277.030464] 2684 pages reserved >[29277.030466] 0 pages cma reserved >[29277.030468] 0 pages hwpoisoned >[29277.030470] Unreclaimable slab info: >[29277.030472] Name Used Total >[29277.030502] xfs_efd_item 1017KB 1017KB >[29277.030506] xfs_buf_item 1020KB 1020KB >[29277.030509] xfs_trans 1022KB 1022KB >[29277.030511] xfs_da_state 1020KB 1020KB >[29277.030514] xfs_log_ticket 1023KB 1023KB >[29277.030518] scsi_sense_cache 1024KB 1024KB >[29277.030524] RAWv6 504KB 504KB >[29277.030527] UDPv6 1012KB 1012KB >[29277.030530] tw_sock_TCPv6 127KB 127KB >[29277.030533] TCPv6 1386KB 1386KB >[29277.030536] cfq_io_cq 1023KB 1023KB >[29277.030539] cfq_queue 1023KB 1023KB >[29277.030542] mqueue_inode_cache 64KB 64KB >[29277.030545] kioctx 63KB 63KB >[29277.030548] posix_timers_cache 1022KB 1022KB >[29277.030551] ip4-frags 1023KB 1023KB >[29277.030553] secpath_cache 768KB 768KB >[29277.030556] RAW 896KB 896KB >[29277.030559] UDP 1008KB 1008KB >[29277.030562] tw_sock_TCP 127KB 127KB >[29277.030565] request_sock_TCP 191KB 191KB >[29277.030568] TCP 2040KB 2040KB >[29277.030571] hugetlbfs_inode_cache 127KB 127KB >[29277.030574] eventpoll_pwq 1023KB 1023KB >[29277.030576] inotify_inode_mark 1023KB 1023KB >[29277.030579] request_queue 755KB 755KB >[29277.030582] blkdev_ioc 1023KB 1023KB >[29277.030585] biovec-128 256KB 256KB >[29277.030588] biovec-64 128KB 128KB >[29277.030591] skbuff_fclone_cache 1024KB 1024KB >[29277.030594] skbuff_head_cache 1600KB 1600KB >[29277.030596] file_lock_cache 1021KB 1021KB >[29277.030599] net_namespace 510KB 510KB >[29277.030602] shmem_inode_cache 1087KB 1087KB >[29277.030605] dtl 10040KB 11520KB >[29277.030608] taskstats 828KB 828KB >[29277.030611] proc_dir_entry 640KB 640KB >[29277.030614] pde_opener 1023KB 1023KB >[29277.030617] seq_file 1024KB 1024KB >[29277.030620] sigqueue 1022KB 1022KB >[29277.030623] kernfs_node_cache 5302KB 5302KB >[29277.030625] mnt_cache 2996KB 2996KB >[29277.030628] names_cache 4096KB 4096KB >[29277.030631] ebitmap_node 1024KB 1024KB >[29277.030634] avtab_node 1023KB 1023KB >[29277.030637] avc_xperms_data 1024KB 1024KB >[29277.030640] selinux_file_security 1024KB 1024KB >[29277.030643] selinux_inode_security 1855KB 1855KB >[29277.030646] nsproxy 1023KB 1023KB >[29277.030649] vm_area_struct 1660KB 1660KB >[29277.030652] mm_struct 8056KB 8056KB >[29277.030655] files_cache 1211KB 1211KB >[29277.030657] signal_cache 1472KB 1472KB >[29277.030660] sighand_cache 2932KB 2932KB >[29277.030663] task_struct 12210KB 12210KB >[29277.030666] cred_jar 4800KB 4800KB >[29277.030669] thread_stack 17408KB 17408KB >[29277.030672] anon_vma 1086KB 1086KB >[29277.030674] pid 1280KB 1280KB >[29277.030677] numa_policy 383KB 383KB >[29277.030680] trace_event_file 255KB 255KB >[29277.030683] ftrace_event_field 447KB 447KB >[29277.030686] pool_workqueue 1024KB 1024KB >[29277.030689] task_group 1020KB 1020KB >[29277.030692] pgtable-2^11 9216KB 9216KB >[29277.030694] pgtable-2^8 4096KB 4096KB >[29277.030698] dma-kmalloc-512 128KB 128KB >[29277.030702] kmalloc-131072 35328KB 35328KB >[29277.030705] kmalloc-65536 9216KB 9216KB >[29277.030708] kmalloc-32768 8192KB 8192KB >[29277.030711] kmalloc-16384 18432KB 20480KB >[29277.030714] kmalloc-8192 8704KB 8704KB >[29277.030716] kmalloc-4096 8192KB 8192KB >[29277.030719] kmalloc-2048 4352KB 4352KB >[29277.030722] kmalloc-1024 19520KB 19520KB >[29277.030725] kmalloc-512 10348KB 11008KB >[29277.030728] kmalloc-256 1408KB 1408KB >[29277.030731] kmalloc-192 1790KB 1790KB >[29277.030734] kmalloc-128 1536KB 1536KB >[29277.030737] kmalloc-96 1470KB 1470KB >[29277.030739] kmalloc-64 1792KB 1792KB >[29277.030742] kmalloc-32 12992KB 12992KB >[29277.030745] kmalloc-16 1024KB 1024KB >[29277.030748] kmalloc-8 1024KB 1024KB >[29277.030751] kmem_cache_node 1024KB 1024KB >[29277.030753] kmem_cache 3315KB 3315KB >[29277.030756] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29277.030764] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29277.030769] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29277.030774] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29277.030778] [ 2089] 0 2089 1288 0 51200 36 0 irqbalance >[29277.030783] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29277.030788] [ 2121] 0 2121 307 0 51200 70 -1000 sshd >[29277.030792] [ 2139] 0 2139 97 0 51200 37 0 atd >[29277.030796] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29277.030800] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29277.030805] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29277.030810] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29277.030814] [ 2697] 0 2697 87 0 49152 30 0 log >[29277.030818] [ 2699] 0 2699 103 0 51200 39 0 config >[29277.030823] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29277.030828] [ 7941] 1000 7941 156 0 51200 41 0 imap >[29277.030833] [ 8919] 97 8919 167 0 51200 36 0 auth >[29277.030837] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29277.030842] [ 8945] 0 8945 169 0 43008 40 0 sshd >[29277.030846] Out of memory: Kill process 7941 (imap) score 0 or sacrifice child >[29277.030854] Killed process 7941 (imap) total-vm:9984kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29277.050011] oom_reaper: reaped process 7941 (imap), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29277.160914] irqbalance invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29277.160923] irqbalance cpuset=/ mems_allowed=0 >[29277.160934] CPU: 1 PID: 2089 Comm: irqbalance Not tainted 4.18.7-200.fc28.ppc64 #1 >[29277.160941] Call Trace: >[29277.160949] [c0000000607db1a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29277.160960] [c0000000607db230] [c0000000002fb670] .dump_header+0x70/0x300 >[29277.160969] [c0000000607db300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29277.160979] [c0000000607db3d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29277.160991] [c0000000607db480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29277.161001] [c0000000607db670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29277.161012] [c0000000607db730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29277.161022] [c0000000607db7f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29277.161032] [c0000000607db880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29277.161042] [c0000000607db990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29277.161051] [c0000000607dbac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29277.161064] [c0000000607dbb90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29277.161077] [c0000000607dbcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29277.161090] [c0000000607dbd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29277.161103] [c0000000607dbe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29277.161111] Mem-Info: >[29277.161121] active_anon:3 inactive_anon:18 isolated_anon:0 > active_file:37 inactive_file:0 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15928 > mapped:57 shmem:0 pagetables:49 bounce:0 > free:1465 free_pcp:2 free_cma:0 >[29277.161149] Node 0 active_anon:192kB inactive_anon:1152kB active_file:2368kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:3648kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29277.161164] Node 0 DMA free:93760kB min:94144kB low:117632kB high:141120kB active_anon:192kB inactive_anon:1152kB active_file:768kB inactive_file:640kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8976kB pagetables:3136kB bounce:0kB free_pcp:128kB local_pcp:0kB free_cma:0kB >[29277.169969] lowmem_reserve[]: 0 0 0 0 >[29277.169980] Node 0 DMA: 386*64kB (UE) 357*128kB (UE) 100*256kB (UE) 3*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97536kB >[29277.170006] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29277.170014] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29277.170021] 70 total pagecache pages >[29277.170026] 8 pages in swap cache >[29277.170031] Swap cache stats: add 3009323, delete 3009434, find 2553780/4335071 >[29277.170037] Free swap = 1935616kB >[29277.170042] Total swap = 2052032kB >[29277.170047] 32768 pages RAM >[29277.170050] 0 pages HighMem/MovableOnly >[29277.170055] 2684 pages reserved >[29277.170059] 0 pages cma reserved >[29277.170063] 0 pages hwpoisoned >[29277.170067] Unreclaimable slab info: >[29277.170072] Name Used Total >[29277.170115] xfs_efd_item 1017KB 1017KB >[29277.170121] xfs_buf_item 1020KB 1020KB >[29277.170127] xfs_trans 1022KB 1022KB >[29277.170134] xfs_da_state 1020KB 1020KB >[29277.170139] xfs_log_ticket 1023KB 1023KB >[29277.170147] scsi_sense_cache 1024KB 1024KB >[29277.170157] RAWv6 504KB 504KB >[29277.170163] UDPv6 1012KB 1012KB >[29277.170169] tw_sock_TCPv6 127KB 127KB >[29277.170176] TCPv6 1386KB 1386KB >[29277.170182] cfq_io_cq 1023KB 1023KB >[29277.170188] cfq_queue 1023KB 1023KB >[29277.170194] mqueue_inode_cache 64KB 64KB >[29277.170200] kioctx 63KB 63KB >[29277.170207] posix_timers_cache 1022KB 1022KB >[29277.170213] ip4-frags 1023KB 1023KB >[29277.170218] secpath_cache 768KB 768KB >[29277.170224] RAW 896KB 896KB >[29277.170230] UDP 1008KB 1008KB >[29277.170236] tw_sock_TCP 127KB 127KB >[29277.170241] request_sock_TCP 191KB 191KB >[29277.170247] TCP 2040KB 2040KB >[29277.170253] hugetlbfs_inode_cache 127KB 127KB >[29277.170259] eventpoll_pwq 1023KB 1023KB >[29277.170265] inotify_inode_mark 1023KB 1023KB >[29277.170271] request_queue 755KB 755KB >[29277.170277] blkdev_ioc 1023KB 1023KB >[29277.170283] biovec-128 256KB 256KB >[29277.170289] biovec-64 128KB 128KB >[29277.170295] skbuff_fclone_cache 1024KB 1024KB >[29277.170301] skbuff_head_cache 1600KB 1600KB >[29277.170306] file_lock_cache 1021KB 1021KB >[29277.170312] net_namespace 510KB 510KB >[29277.170318] shmem_inode_cache 1087KB 1087KB >[29277.170324] dtl 10040KB 11520KB >[29277.170330] taskstats 828KB 828KB >[29277.170336] proc_dir_entry 640KB 640KB >[29277.170342] pde_opener 1023KB 1023KB >[29277.170348] seq_file 1024KB 1024KB >[29277.170354] sigqueue 1022KB 1022KB >[29277.170360] kernfs_node_cache 5302KB 5302KB >[29277.170366] mnt_cache 2996KB 2996KB >[29277.170372] names_cache 4096KB 4096KB >[29277.170378] ebitmap_node 1024KB 1024KB >[29277.170384] avtab_node 1023KB 1023KB >[29277.170390] avc_xperms_data 1024KB 1024KB >[29277.170396] selinux_file_security 1024KB 1024KB >[29277.170401] selinux_inode_security 1855KB 1855KB >[29277.170408] nsproxy 1023KB 1023KB >[29277.170414] vm_area_struct 1660KB 1660KB >[29277.170420] mm_struct 8056KB 8056KB >[29277.170425] files_cache 1211KB 1211KB >[29277.170431] signal_cache 1472KB 1472KB >[29277.170437] sighand_cache 2932KB 2932KB >[29277.170442] task_struct 12210KB 12210KB >[29277.170448] cred_jar 4800KB 4800KB >[29277.170454] thread_stack 17408KB 17408KB >[29277.170460] anon_vma 1086KB 1086KB >[29277.170465] pid 1280KB 1280KB >[29277.170471] numa_policy 383KB 383KB >[29277.170477] trace_event_file 255KB 255KB >[29277.170482] ftrace_event_field 447KB 447KB >[29277.170488] pool_workqueue 1024KB 1024KB >[29277.170494] task_group 1020KB 1020KB >[29277.170500] pgtable-2^11 9216KB 9216KB >[29277.170506] pgtable-2^8 4096KB 4096KB >[29277.170514] dma-kmalloc-512 128KB 128KB >[29277.170523] kmalloc-131072 35328KB 35328KB >[29277.170529] kmalloc-65536 9216KB 9216KB >[29277.170535] kmalloc-32768 8192KB 8192KB >[29277.170541] kmalloc-16384 18432KB 20480KB >[29277.170547] kmalloc-8192 8704KB 8704KB >[29277.170553] kmalloc-4096 8192KB 8192KB >[29277.170558] kmalloc-2048 4352KB 4352KB >[29277.170564] kmalloc-1024 19520KB 19520KB >[29277.170571] kmalloc-512 10348KB 11008KB >[29277.170577] kmalloc-256 1408KB 1408KB >[29277.170583] kmalloc-192 1790KB 1790KB >[29277.170589] kmalloc-128 1536KB 1536KB >[29277.170594] kmalloc-96 1470KB 1470KB >[29277.170600] kmalloc-64 1792KB 1792KB >[29277.170606] kmalloc-32 12992KB 12992KB >[29277.170612] kmalloc-16 1024KB 1024KB >[29277.170618] kmalloc-8 1024KB 1024KB >[29277.170623] kmem_cache_node 1024KB 1024KB >[29277.170629] kmem_cache 3315KB 3315KB >[29277.170634] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29277.170656] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29277.170666] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29277.170676] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29277.170686] [ 2089] 0 2089 1288 0 51200 36 0 irqbalance >[29277.170696] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29277.170705] [ 2121] 0 2121 307 0 51200 70 -1000 sshd >[29277.170715] [ 2139] 0 2139 97 0 51200 37 0 atd >[29277.170724] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29277.170733] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29277.170743] [ 2695] 0 2695 91 0 49152 22 0 dovecot >[29277.170752] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29277.170761] [ 2697] 0 2697 87 0 49152 30 0 log >[29277.170770] [ 2699] 0 2699 103 0 51200 39 0 config >[29277.170780] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29277.170796] [ 8919] 97 8919 167 0 51200 36 0 auth >[29277.170805] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29277.170815] [ 8945] 0 8945 169 0 43008 41 0 sshd >[29277.170823] Out of memory: Kill process 8945 (sshd) score 0 or sacrifice child >[29277.170834] Killed process 8945 (sshd) total-vm:10816kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29277.190013] oom_reaper: reaped process 8945 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29279.270987] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29279.271002] sshd cpuset=/ mems_allowed=0 >[29279.271012] CPU: 8 PID: 8946 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29279.271016] Call Trace: >[29279.271023] [c00000000e3d31a0] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29279.271030] [c00000000e3d3230] [c0000000002fb670] .dump_header+0x70/0x300 >[29279.271036] [c00000000e3d3300] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29279.271043] [c00000000e3d33d0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29279.271049] [c00000000e3d3480] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29279.271056] [c00000000e3d3670] [c000000000396f8c] .alloc_pages_vma+0xac/0x300 >[29279.271063] [c00000000e3d3730] [c000000000379eac] .__read_swap_cache_async+0x1ec/0x330 >[29279.271069] [c00000000e3d37f0] [c00000000037a038] .read_swap_cache_async+0x48/0xb0 >[29279.271076] [c00000000e3d3880] [c00000000037a424] .swap_cluster_readahead+0x384/0x3e0 >[29279.271082] [c00000000e3d3990] [c00000000037a960] .swapin_readahead+0x350/0x5d0 >[29279.271089] [c00000000e3d3ac0] [c000000000352170] .do_swap_page+0x4b0/0xba0 >[29279.271095] [c00000000e3d3b90] [c000000000356d48] .__handle_mm_fault+0xf08/0x19c0 >[29279.271101] [c00000000e3d3cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29279.271109] [c00000000e3d3d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29279.271116] [c00000000e3d3e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29279.271121] Mem-Info: >[29279.271129] active_anon:0 inactive_anon:14 isolated_anon:0 > active_file:4 inactive_file:33 isolated_file:0 > unevictable:0 dirty:0 writeback:2 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15939 > mapped:18 shmem:0 pagetables:49 bounce:0 > free:1471 free_pcp:0 free_cma:0 >[29279.271147] Node 0 active_anon:0kB inactive_anon:896kB active_file:256kB inactive_file:2112kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1152kB dirty:0kB writeback:128kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no >[29279.271157] Node 0 DMA free:94144kB min:94144kB low:117632kB high:141120kB active_anon:0kB inactive_anon:832kB active_file:128kB inactive_file:1344kB unevictable:0kB writepending:128kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8976kB pagetables:3136kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29279.271171] lowmem_reserve[]: 0 0 0 0 >[29279.279949] Node 0 DMA: 387*64kB (UE) 357*128kB (UE) 100*256kB (UE) 2*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 97088kB >[29279.279966] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29279.279972] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29279.279976] 87 total pagecache pages >[29279.279980] 6 pages in swap cache >[29279.279983] Swap cache stats: add 3009961, delete 3010074, find 2554256/4335968 >[29279.279987] Free swap = 1937216kB >[29279.279990] Total swap = 2052032kB >[29279.279993] 32768 pages RAM >[29279.279996] 0 pages HighMem/MovableOnly >[29279.279998] 2684 pages reserved >[29279.280001] 0 pages cma reserved >[29279.280004] 0 pages hwpoisoned >[29279.280007] Unreclaimable slab info: >[29279.280010] Name Used Total >[29279.280151] xfs_efd_item 1017KB 1017KB >[29279.280156] xfs_buf_item 1020KB 1020KB >[29279.280160] xfs_trans 1022KB 1022KB >[29279.280164] xfs_da_state 1020KB 1020KB >[29279.280168] xfs_log_ticket 1023KB 1023KB >[29279.280176] scsi_sense_cache 1024KB 1024KB >[29279.280191] RAWv6 504KB 504KB >[29279.280195] UDPv6 1012KB 1012KB >[29279.280199] tw_sock_TCPv6 127KB 127KB >[29279.280204] TCPv6 1386KB 1386KB >[29279.280208] cfq_io_cq 1023KB 1023KB >[29279.280212] cfq_queue 1023KB 1023KB >[29279.280217] mqueue_inode_cache 64KB 64KB >[29279.280221] kioctx 63KB 63KB >[29279.280226] posix_timers_cache 1022KB 1022KB >[29279.280229] ip4-frags 1023KB 1023KB >[29279.280233] secpath_cache 768KB 768KB >[29279.280237] RAW 896KB 896KB >[29279.280241] UDP 1008KB 1008KB >[29279.280244] tw_sock_TCP 127KB 127KB >[29279.280248] request_sock_TCP 191KB 191KB >[29279.280252] TCP 2040KB 2040KB >[29279.280256] hugetlbfs_inode_cache 127KB 127KB >[29279.280260] eventpoll_pwq 1023KB 1023KB >[29279.280264] inotify_inode_mark 1023KB 1023KB >[29279.280267] request_queue 755KB 755KB >[29279.280271] blkdev_ioc 1023KB 1023KB >[29279.280275] biovec-128 256KB 256KB >[29279.280279] biovec-64 128KB 128KB >[29279.280283] skbuff_fclone_cache 1024KB 1024KB >[29279.280287] skbuff_head_cache 1600KB 1600KB >[29279.280290] file_lock_cache 1021KB 1021KB >[29279.280294] net_namespace 510KB 510KB >[29279.280298] shmem_inode_cache 1087KB 1087KB >[29279.280303] dtl 10040KB 11520KB >[29279.280307] taskstats 828KB 828KB >[29279.280310] proc_dir_entry 640KB 640KB >[29279.280314] pde_opener 1023KB 1023KB >[29279.280318] seq_file 1024KB 1024KB >[29279.280322] sigqueue 1022KB 1022KB >[29279.280326] kernfs_node_cache 5302KB 5302KB >[29279.280330] mnt_cache 2996KB 2996KB >[29279.280333] names_cache 4096KB 4096KB >[29279.280338] ebitmap_node 1024KB 1024KB >[29279.280341] avtab_node 1023KB 1023KB >[29279.280345] avc_xperms_data 1024KB 1024KB >[29279.280349] selinux_file_security 1024KB 1024KB >[29279.280353] selinux_inode_security 1855KB 1855KB >[29279.280357] nsproxy 1023KB 1023KB >[29279.280361] vm_area_struct 1660KB 1660KB >[29279.280364] mm_struct 8056KB 8056KB >[29279.280368] files_cache 1211KB 1211KB >[29279.280372] signal_cache 1472KB 1472KB >[29279.280375] sighand_cache 2932KB 2932KB >[29279.280379] task_struct 12210KB 12210KB >[29279.280383] cred_jar 4800KB 4800KB >[29279.280387] thread_stack 17408KB 17408KB >[29279.280391] anon_vma 1086KB 1086KB >[29279.280394] pid 1280KB 1280KB >[29279.280398] numa_policy 383KB 383KB >[29279.280402] trace_event_file 255KB 255KB >[29279.280406] ftrace_event_field 447KB 447KB >[29279.280409] pool_workqueue 1024KB 1024KB >[29279.280413] task_group 1020KB 1020KB >[29279.280417] pgtable-2^11 9216KB 9216KB >[29279.280421] pgtable-2^8 4096KB 4096KB >[29279.280426] dma-kmalloc-512 128KB 128KB >[29279.280431] kmalloc-131072 35328KB 35328KB >[29279.280435] kmalloc-65536 9216KB 9216KB >[29279.280438] kmalloc-32768 8192KB 8192KB >[29279.280442] kmalloc-16384 18432KB 20480KB >[29279.280446] kmalloc-8192 8704KB 8704KB >[29279.280450] kmalloc-4096 8192KB 8192KB >[29279.280454] kmalloc-2048 4352KB 4352KB >[29279.280458] kmalloc-1024 19515KB 19520KB >[29279.280462] kmalloc-512 10475KB 11008KB >[29279.280466] kmalloc-256 1408KB 1408KB >[29279.280469] kmalloc-192 1790KB 1790KB >[29279.280473] kmalloc-128 1536KB 1536KB >[29279.280477] kmalloc-96 1470KB 1470KB >[29279.280480] kmalloc-64 1792KB 1792KB >[29279.280484] kmalloc-32 12992KB 12992KB >[29279.280488] kmalloc-16 1024KB 1024KB >[29279.280491] kmalloc-8 1024KB 1024KB >[29279.280495] kmem_cache_node 1024KB 1024KB >[29279.280499] kmem_cache 3315KB 3315KB >[29279.280503] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29279.280519] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29279.280526] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29279.280533] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29279.280539] [ 2089] 0 2089 1288 0 51200 45 0 irqbalance >[29279.280546] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29279.280552] [ 2121] 0 2121 307 0 51200 80 -1000 sshd >[29279.280558] [ 2139] 0 2139 97 0 51200 37 0 atd >[29279.280564] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29279.280570] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29279.280578] [ 2695] 0 2695 91 0 49152 29 0 dovecot >[29279.280584] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29279.280590] [ 2697] 0 2697 87 0 49152 30 0 log >[29279.280596] [ 2699] 0 2699 103 0 51200 39 0 config >[29279.280602] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29279.280615] [ 8919] 97 8919 167 0 51200 36 0 auth >[29279.280620] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29279.280627] [ 8946] 0 8946 164 0 38912 33 0 sshd >[29279.280632] Out of memory: Kill process 2089 (irqbalance) score 0 or sacrifice child >[29279.280653] Killed process 2089 (irqbalance) total-vm:82432kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29279.300038] oom_reaper: reaped process 2089 (irqbalance), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29282.289984] sshd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29282.289998] sshd cpuset=/ mems_allowed=0 >[29282.290008] CPU: 8 PID: 8946 Comm: sshd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29282.290013] Call Trace: >[29282.290020] [c00000000e3d3380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29282.290029] [c00000000e3d3410] [c0000000002fb670] .dump_header+0x70/0x300 >[29282.290036] [c00000000e3d34e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29282.290043] [c00000000e3d35b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29282.290049] [c00000000e3d3660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29282.290058] [c00000000e3d3850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29282.290065] [c00000000e3d38f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29282.290073] [c00000000e3d3980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29282.290171] [c00000000e3d3a50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29282.290179] [c00000000e3d3b00] [c00000000034d494] .__do_fault+0x44/0x200 >[29282.290186] [c00000000e3d3b90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29282.290193] [c00000000e3d3cc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29282.290201] [c00000000e3d3d50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29282.290209] [c00000000e3d3e30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29282.290214] Mem-Info: >[29282.290222] active_anon:0 inactive_anon:13 isolated_anon:0 > active_file:0 inactive_file:17 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15939 > mapped:5 shmem:0 pagetables:47 bounce:0 > free:1458 free_pcp:0 free_cma:0 >[29282.290244] Node 0 active_anon:0kB inactive_anon:832kB active_file:0kB inactive_file:1088kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:320kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29282.290255] Node 0 DMA free:93312kB min:94144kB low:117632kB high:141120kB active_anon:512kB inactive_anon:0kB active_file:320kB inactive_file:128kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8960kB pagetables:3008kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29282.290271] lowmem_reserve[]: 0 0 0 0 >[29282.290278] Node 0 DMA: 384*64kB (UE) 357*128kB (UE) 100*256kB (UE) 2*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 96896kB >[29282.290299] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29282.290305] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29282.290310] 72 total pagecache pages >[29282.290313] 13 pages in swap cache >[29282.290317] Swap cache stats: add 3011180, delete 3011286, find 2555234/4337836 >[29282.290322] Free swap = 1939328kB >[29282.290325] Total swap = 2052032kB >[29282.290328] 32768 pages RAM >[29282.290331] 0 pages HighMem/MovableOnly >[29282.290334] 2684 pages reserved >[29282.290337] 0 pages cma reserved >[29282.290341] 0 pages hwpoisoned >[29282.290344] Unreclaimable slab info: >[29282.290347] Name Used Total >[29282.290488] xfs_efd_item 1017KB 1017KB >[29282.290493] xfs_buf_item 1020KB 1020KB >[29282.290497] xfs_trans 1022KB 1022KB >[29282.290502] xfs_da_state 1020KB 1020KB >[29282.290506] xfs_log_ticket 1023KB 1023KB >[29282.290515] scsi_sense_cache 1024KB 1024KB >[29282.290531] RAWv6 504KB 504KB >[29282.290536] UDPv6 1012KB 1012KB >[29282.290540] tw_sock_TCPv6 127KB 127KB >[29282.290545] TCPv6 1386KB 1386KB >[29282.290551] cfq_io_cq 1023KB 1023KB >[29282.290555] cfq_queue 1023KB 1023KB >[29282.290559] mqueue_inode_cache 64KB 64KB >[29282.290563] kioctx 63KB 63KB >[29282.290567] posix_timers_cache 1022KB 1022KB >[29282.290570] ip4-frags 1023KB 1023KB >[29282.290573] secpath_cache 768KB 768KB >[29282.290576] RAW 896KB 896KB >[29282.290579] UDP 1008KB 1008KB >[29282.290582] tw_sock_TCP 127KB 127KB >[29282.290585] request_sock_TCP 191KB 191KB >[29282.290588] TCP 2040KB 2040KB >[29282.290590] hugetlbfs_inode_cache 127KB 127KB >[29282.290594] eventpoll_pwq 1023KB 1023KB >[29282.290597] inotify_inode_mark 1023KB 1023KB >[29282.290600] request_queue 755KB 755KB >[29282.290603] blkdev_ioc 1023KB 1023KB >[29282.290606] biovec-128 256KB 256KB >[29282.290608] biovec-64 128KB 128KB >[29282.290612] skbuff_fclone_cache 1024KB 1024KB >[29282.290614] skbuff_head_cache 1600KB 1600KB >[29282.290617] file_lock_cache 1021KB 1021KB >[29282.290620] net_namespace 510KB 510KB >[29282.290623] shmem_inode_cache 1087KB 1087KB >[29282.290627] dtl 10040KB 11520KB >[29282.290630] taskstats 828KB 828KB >[29282.290633] proc_dir_entry 640KB 640KB >[29282.290636] pde_opener 1023KB 1023KB >[29282.290639] seq_file 1024KB 1024KB >[29282.290642] sigqueue 1022KB 1022KB >[29282.290645] kernfs_node_cache 5302KB 5302KB >[29282.290648] mnt_cache 2996KB 2996KB >[29282.290651] names_cache 4096KB 4096KB >[29282.290654] ebitmap_node 1024KB 1024KB >[29282.290657] avtab_node 1023KB 1023KB >[29282.290660] avc_xperms_data 1024KB 1024KB >[29282.290663] selinux_file_security 1024KB 1024KB >[29282.290665] selinux_inode_security 1855KB 1855KB >[29282.290669] nsproxy 1023KB 1023KB >[29282.290672] vm_area_struct 1660KB 1660KB >[29282.290675] mm_struct 8056KB 8056KB >[29282.290677] files_cache 1211KB 1211KB >[29282.290680] signal_cache 1472KB 1472KB >[29282.290683] sighand_cache 2932KB 2932KB >[29282.290686] task_struct 12210KB 12210KB >[29282.290689] cred_jar 4800KB 4800KB >[29282.290691] thread_stack 17408KB 17408KB >[29282.290694] anon_vma 1086KB 1086KB >[29282.290697] pid 1280KB 1280KB >[29282.290700] numa_policy 383KB 383KB >[29282.290703] trace_event_file 255KB 255KB >[29282.290706] ftrace_event_field 447KB 447KB >[29282.290709] pool_workqueue 1024KB 1024KB >[29282.290712] task_group 1020KB 1020KB >[29282.290715] pgtable-2^11 9216KB 9216KB >[29282.290718] pgtable-2^8 4096KB 4096KB >[29282.290721] dma-kmalloc-512 128KB 128KB >[29282.290725] kmalloc-131072 35328KB 35328KB >[29282.290728] kmalloc-65536 9216KB 9216KB >[29282.290731] kmalloc-32768 8192KB 8192KB >[29282.290734] kmalloc-16384 18432KB 20480KB >[29282.290737] kmalloc-8192 8704KB 8704KB >[29282.290740] kmalloc-4096 8192KB 8192KB >[29282.290742] kmalloc-2048 4352KB 4352KB >[29282.290745] kmalloc-1024 19515KB 19520KB >[29282.290749] kmalloc-512 10475KB 11008KB >[29282.290752] kmalloc-256 1408KB 1408KB >[29282.290754] kmalloc-192 1790KB 1790KB >[29282.290757] kmalloc-128 1536KB 1536KB >[29282.290760] kmalloc-96 1470KB 1470KB >[29282.290763] kmalloc-64 1792KB 1792KB >[29282.290765] kmalloc-32 12992KB 12992KB >[29282.290768] kmalloc-16 1024KB 1024KB >[29282.290771] kmalloc-8 1024KB 1024KB >[29282.290774] kmem_cache_node 1024KB 1024KB >[29282.290777] kmem_cache 3315KB 3315KB >[29282.290779] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29282.290792] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29282.290798] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29282.290802] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29282.290808] [ 2100] 992 2100 1269 0 53248 40 0 chronyd >[29282.290812] [ 2121] 0 2121 307 0 51200 82 -1000 sshd >[29282.290817] [ 2139] 0 2139 97 0 51200 37 0 atd >[29282.290821] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29282.290825] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29282.290831] [ 2695] 0 2695 91 0 49152 29 0 dovecot >[29282.290835] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29282.290840] [ 2697] 0 2697 87 0 49152 30 0 log >[29282.290844] [ 2699] 0 2699 103 0 51200 39 0 config >[29282.290849] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29282.290855] [ 8919] 97 8919 167 0 51200 36 0 auth >[29282.290860] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29282.290865] [ 8946] 0 8946 275 1 40960 49 0 sshd >[29282.290868] Out of memory: Kill process 8946 (sshd) score 0 or sacrifice child >[29282.290875] Killed process 8946 (sshd) total-vm:17600kB, anon-rss:64kB, file-rss:0kB, shmem-rss:0kB >[29282.290996] oom_reaper: reaped process 8946 (sshd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29286.030551] systemd invoked oom-killer: gfp_mask=0x6200ca(GFP_HIGHUSER_MOVABLE), nodemask=(null), order=0, oom_score_adj=0 >[29286.030561] systemd cpuset=/ mems_allowed=0 >[29286.030568] CPU: 0 PID: 1 Comm: systemd Not tainted 4.18.7-200.fc28.ppc64 #1 >[29286.030572] Call Trace: >[29286.030577] [c00000007e9ff380] [c000000000c0ee9c] .dump_stack+0xa8/0xec (unreliable) >[29286.030583] [c00000007e9ff410] [c0000000002fb670] .dump_header+0x70/0x300 >[29286.030587] [c00000007e9ff4e0] [c0000000002fbdb0] .oom_kill_process+0x150/0x640 >[29286.030592] [c00000007e9ff5b0] [c0000000002fce38] .out_of_memory+0x208/0x710 >[29286.030596] [c00000007e9ff660] [c000000000306260] .__alloc_pages_nodemask+0xfc0/0x1060 >[29286.030602] [c00000007e9ff850] [c000000000394c0c] .alloc_pages_current+0xac/0x150 >[29286.030607] [c00000007e9ff8f0] [c0000000002f483c] .__page_cache_alloc+0x13c/0x180 >[29286.030612] [c00000007e9ff980] [c0000000002f6d98] .filemap_fault+0x5a8/0x840 >[29286.030673] [c00000007e9ffa50] [d000000002c17248] .__xfs_filemap_fault+0xe8/0x310 [xfs] >[29286.030679] [c00000007e9ffb00] [c00000000034d494] .__do_fault+0x44/0x200 >[29286.030683] [c00000007e9ffb90] [c000000000356df4] .__handle_mm_fault+0xfb4/0x19c0 >[29286.030688] [c00000007e9ffcc0] [c000000000357938] .handle_mm_fault+0x138/0x250 >[29286.030693] [c00000007e9ffd50] [c00000000006f8ec] .__do_page_fault+0x24c/0xb10 >[29286.030699] [c00000007e9ffe30] [c00000000000adf4] handle_page_fault+0x18/0x38 >[29286.030702] Mem-Info: >[29286.030708] active_anon:30 inactive_anon:6 isolated_anon:0 > active_file:0 inactive_file:35 isolated_file:0 > unevictable:0 dirty:0 writeback:0 unstable:0 > slab_reclaimable:2798 slab_unreclaimable:15939 > mapped:44 shmem:0 pagetables:47 bounce:0 > free:1458 free_pcp:0 free_cma:0 >[29286.030722] Node 0 active_anon:1920kB inactive_anon:384kB active_file:0kB inactive_file:2240kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:2816kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes >[29286.030730] Node 0 DMA free:93312kB min:94144kB low:117632kB high:141120kB active_anon:320kB inactive_anon:384kB active_file:832kB inactive_file:768kB unevictable:0kB writepending:0kB present:2097152kB managed:1925376kB mlocked:0kB kernel_stack:8944kB pagetables:3008kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB >[29286.030741] lowmem_reserve[]: 0 0 0 0 >[29286.030745] Node 0 DMA: 381*64kB (UE) 357*128kB (UE) 100*256kB (UE) 2*512kB (U) 0*1024kB 0*2048kB 0*4096kB 0*8192kB 0*16384kB = 96704kB >[29286.030758] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16384kB >[29286.030763] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=16777216kB >[29286.030766] 77 total pagecache pages >[29286.030768] 9 pages in swap cache >[29286.030771] Swap cache stats: add 3012645, delete 3012755, find 2556286/4339833 >[29286.030774] Free swap = 1937664kB >[29286.030776] Total swap = 2052032kB >[29286.030778] 32768 pages RAM >[29286.030780] 0 pages HighMem/MovableOnly >[29286.030782] 2684 pages reserved >[29286.030784] 0 pages cma reserved >[29286.030786] 0 pages hwpoisoned >[29286.030789] Unreclaimable slab info: >[29286.030791] Name Used Total >[29286.030917] xfs_efd_item 1017KB 1017KB >[29286.030921] xfs_buf_item 1020KB 1020KB >[29286.030924] xfs_trans 1022KB 1022KB >[29286.030927] xfs_da_state 1020KB 1020KB >[29286.030930] xfs_log_ticket 1023KB 1023KB >[29286.030937] scsi_sense_cache 1024KB 1024KB >[29286.030953] RAWv6 504KB 504KB >[29286.030956] UDPv6 1012KB 1012KB >[29286.030959] tw_sock_TCPv6 127KB 127KB >[29286.030962] TCPv6 1386KB 1386KB >[29286.030966] cfq_io_cq 1023KB 1023KB >[29286.030969] cfq_queue 1023KB 1023KB >[29286.030972] mqueue_inode_cache 64KB 64KB >[29286.030975] kioctx 63KB 63KB >[29286.030979] posix_timers_cache 1022KB 1022KB >[29286.030982] ip4-frags 1023KB 1023KB >[29286.030985] secpath_cache 768KB 768KB >[29286.030988] RAW 896KB 896KB >[29286.030991] UDP 1008KB 1008KB >[29286.030993] tw_sock_TCP 127KB 127KB >[29286.030996] request_sock_TCP 191KB 191KB >[29286.030999] TCP 2040KB 2040KB >[29286.031002] hugetlbfs_inode_cache 127KB 127KB >[29286.031005] eventpoll_pwq 1023KB 1023KB >[29286.031008] inotify_inode_mark 1023KB 1023KB >[29286.031011] request_queue 755KB 755KB >[29286.031014] blkdev_ioc 1023KB 1023KB >[29286.031017] biovec-128 256KB 256KB >[29286.031020] biovec-64 128KB 128KB >[29286.031023] skbuff_fclone_cache 1024KB 1024KB >[29286.031026] skbuff_head_cache 1600KB 1600KB >[29286.031029] file_lock_cache 1021KB 1021KB >[29286.031032] net_namespace 510KB 510KB >[29286.031035] shmem_inode_cache 1087KB 1087KB >[29286.031038] dtl 10040KB 11520KB >[29286.031041] taskstats 828KB 828KB >[29286.031047] proc_dir_entry 640KB 640KB >[29286.031050] pde_opener 1023KB 1023KB >[29286.031053] seq_file 1024KB 1024KB >[29286.031056] sigqueue 1022KB 1022KB >[29286.031058] kernfs_node_cache 5302KB 5302KB >[29286.031061] mnt_cache 2996KB 2996KB >[29286.031065] names_cache 4096KB 4096KB >[29286.031068] ebitmap_node 1024KB 1024KB >[29286.031070] avtab_node 1023KB 1023KB >[29286.031073] avc_xperms_data 1024KB 1024KB >[29286.031076] selinux_file_security 1024KB 1024KB >[29286.031079] selinux_inode_security 1855KB 1855KB >[29286.031083] nsproxy 1023KB 1023KB >[29286.031086] vm_area_struct 1660KB 1660KB >[29286.031088] mm_struct 8056KB 8056KB >[29286.031091] files_cache 1211KB 1211KB >[29286.031094] signal_cache 1472KB 1472KB >[29286.031097] sighand_cache 2932KB 2932KB >[29286.031100] task_struct 12210KB 12210KB >[29286.031103] cred_jar 4800KB 4800KB >[29286.031105] thread_stack 17408KB 17408KB >[29286.031108] anon_vma 1086KB 1086KB >[29286.031111] pid 1280KB 1280KB >[29286.031114] numa_policy 383KB 383KB >[29286.031117] trace_event_file 255KB 255KB >[29286.031120] ftrace_event_field 447KB 447KB >[29286.031123] pool_workqueue 1024KB 1024KB >[29286.031126] task_group 1020KB 1020KB >[29286.031129] pgtable-2^11 9216KB 9216KB >[29286.031131] pgtable-2^8 4096KB 4096KB >[29286.031135] dma-kmalloc-512 128KB 128KB >[29286.031139] kmalloc-131072 35328KB 35328KB >[29286.031142] kmalloc-65536 9216KB 9216KB >[29286.031145] kmalloc-32768 8192KB 8192KB >[29286.031148] kmalloc-16384 18432KB 20480KB >[29286.031150] kmalloc-8192 8704KB 8704KB >[29286.031153] kmalloc-4096 8192KB 8192KB >[29286.031156] kmalloc-2048 4352KB 4352KB >[29286.031160] kmalloc-1024 19242KB 19520KB >[29286.031163] kmalloc-512 10475KB 11008KB >[29286.031166] kmalloc-256 1408KB 1408KB >[29286.031168] kmalloc-192 1790KB 1790KB >[29286.031171] kmalloc-128 1536KB 1536KB >[29286.031174] kmalloc-96 1470KB 1470KB >[29286.031177] kmalloc-64 1792KB 1792KB >[29286.031180] kmalloc-32 12992KB 12992KB >[29286.031182] kmalloc-16 1024KB 1024KB >[29286.031185] kmalloc-8 1024KB 1024KB >[29286.031188] kmem_cache_node 1024KB 1024KB >[29286.031191] kmem_cache 3315KB 3315KB >[29286.031193] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name >[29286.031201] [ 748] 0 748 542 0 53248 117 -1000 systemd-udevd >[29286.031207] [ 2033] 0 2033 346 0 49152 73 -1000 auditd >[29286.031211] [ 2069] 81 2069 208 0 51200 58 -900 dbus-daemon >[29286.031217] [ 2100] 992 2100 1269 0 53248 41 0 chronyd >[29286.031221] [ 2121] 0 2121 307 0 51200 81 -1000 sshd >[29286.031225] [ 2139] 0 2139 97 0 51200 37 0 atd >[29286.031230] [ 2150] 0 2150 3253 0 51200 11 0 agetty >[29286.031234] [ 2153] 0 2153 3260 0 51200 11 0 agetty >[29286.031239] [ 2695] 0 2695 91 0 49152 29 0 dovecot >[29286.031243] [ 2696] 97 2696 84 0 51200 31 0 anvil >[29286.031248] [ 2697] 0 2697 87 0 49152 30 0 log >[29286.031252] [ 2699] 0 2699 103 0 51200 39 0 config >[29286.031256] [ 2971] 1000 2971 156 0 51200 38 0 imap >[29286.031262] [ 8919] 97 8919 167 0 51200 36 0 auth >[29286.031267] [ 8920] 990 8920 159 0 51200 34 0 pop3-login >[29286.031271] Out of memory: Kill process 2100 (chronyd) score 0 or sacrifice child >[29286.031283] Killed process 2100 (chronyd) total-vm:81216kB, anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29286.042037] oom_reaper: reaped process 2100 (chronyd), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB >[29291.600359] systemd[1]: systemd-logind.service: Main process exited, code=killed, status=9/KILL >[29291.600691] systemd[1]: systemd-logind.service: Failed with result 'signal'. >[29291.610562] systemd[1]: smartd.service: Main process exited, code=killed, status=9/KILL >[29291.610684] systemd[1]: smartd.service: Failed with result 'signal'. >[29291.620220] systemd[1]: irqbalance.service: Main process exited, code=killed, status=9/KILL >[29291.620331] systemd[1]: irqbalance.service: Failed with result 'signal'. >[29291.621027] systemd[1]: NetworkManager.service: Main process exited, code=killed, status=9/KILL >[29291.621035] systemd[1]: NetworkManager.service: Failed with result 'signal'. >[29291.630635] systemd[1]: rsyslog.service: Main process exited, code=killed, status=9/KILL >[29291.630742] systemd[1]: rsyslog.service: Failed with result 'signal'. >[29317.055098] systemd[1]: mariadb.service: Service hold-off time over, scheduling restart. >[29317.055530] systemd[1]: mariadb.service: Scheduled restart job, restart counter is at 1. >[29317.055734] systemd[1]: Stopped MariaDB 10.2 database server. >[29317.069995] systemd[1]: Starting MariaDB 10.2 database server... >[29317.070092] systemd[1]: Stopped System Logging Service. >[29317.090899] systemd[1]: Starting System Logging Service... >[29317.090954] systemd[1]: Stopped Network Manager Wait Online. >[29317.091047] systemd[1]: Stopping Network Manager Wait Online... >[29317.091095] systemd[1]: Stopped Network Manager. >[29317.110848] systemd[1]: Starting Network Manager... >[29317.500754] systemd-journald[8963]: File /var/log/journal/533a0a923b964860b33da8018f01c95b/system.journal corrupted or uncleanly shut down, renaming and replacing.
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1630370
: 1484372