Bug 177648 - oom problems with test kernel
oom problems with test kernel
Status: CLOSED CURRENTRELEASE
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
4
All Linux
medium Severity medium
: ---
: ---
Assigned To: Dave Jones
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-01-12 12:20 EST by Orion Poplawski
Modified: 2015-01-04 17:24 EST (History)
2 users (show)

See Also:
Fixed In Version: 2.6.15-1.1830_FC4
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-02-06 12:47:40 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
dmesg output (16.63 KB, text/plain)
2006-01-24 11:14 EST, Orion Poplawski
no flags Details

  None (edit)
Description Orion Poplawski 2006-01-12 12:20:38 EST
Description of problem:
Started running the latest updates-testing kernel and seeing oom kills.  Don't
think anything else pertinent has changed.  Here's the long from the first one:

Jan 11 20:49:24 wind kernel: oom-killer: gfp_mask=0x200d2, order=0
Jan 11 20:49:24 wind kernel: Mem-info:
Jan 11 20:49:24 wind kernel: DMA per-cpu:
Jan 11 20:49:24 wind kernel: cpu 0 hot: low 0, high 0, batch 1 used:0
Jan 11 20:49:24 wind kernel: cpu 0 cold: low 0, high 0, batch 1 used:0
Jan 11 20:49:24 wind kernel: DMA32 per-cpu: empty
Jan 11 20:49:24 wind kernel: Normal per-cpu:
Jan 11 20:49:24 wind kernel: cpu 0 hot: low 0, high 90, batch 15 used:0
Jan 11 20:49:24 wind kernel: cpu 0 cold: low 0, high 30, batch 7 used:23
Jan 11 20:49:24 wind kernel: HighMem per-cpu: empty
Jan 11 20:49:24 wind kernel: Free pages:        3916kB (0kB HighMem)
Jan 11 20:49:24 wind kernel: Active:3120 inactive:568 dirty:0 writeback:97
unstable:0 fre
e:979 slab:57172 mapped:2466 pagetables:892
Jan 11 20:49:25 wind kernel: DMA free:1080kB min:124kB low:152kB high:184kB
active:76kB i
nactive:48kB present:16384kB pages_scanned:203 all_unreclaimable? yes
Jan 11 20:49:28 wind kernel: lowmem_reserve[]: 0 0 239 239
Jan 11 20:49:29 wind kernel: DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB
inactive:
0kB present:0kB pages_scanned:0 all_unreclaimable? no
Jan 11 20:49:30 wind kernel: lowmem_reserve[]: 0 0 239 239
Jan 11 20:49:32 wind kernel: Normal free:2836kB min:1916kB low:2392kB
high:2872kB active:
12404kB inactive:2224kB present:245744kB pages_scanned:43 all_unreclaimable? no
Jan 11 20:49:35 wind kernel: lowmem_reserve[]: 0 0 0 0
Jan 11 20:49:40 wind kernel: HighMem free:0kB min:128kB low:128kB high:128kB
active:0kB i
nactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
Jan 11 20:49:41 wind kernel: lowmem_reserve[]: 0 0 0 0
Jan 11 20:49:42 wind kernel: DMA: 0*4kB 5*8kB 1*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512k
B 1*1024kB 0*2048kB 0*4096kB = 1080kB
Jan 11 20:49:45 wind kernel: DMA32: empty
Jan 11 20:49:49 wind kernel: Normal: 235*4kB 7*8kB 27*16kB 0*32kB 0*64kB 1*128kB
1*256kB
0*512kB 1*1024kB 0*2048kB 0*4096kB = 2836kB
Jan 11 20:49:55 wind kernel: HighMem: empty
Jan 11 20:49:56 wind kernel: Swap cache: add 1847963, delete 1846570, find
3101047/347173
5, race 4+128
Jan 11 20:49:57 wind kernel: Free swap  = 1891084kB
Jan 11 20:50:00 wind kernel: Total swap = 2097144kB
Jan 11 20:50:01 wind kernel: Free swap:       1891084kB
Jan 11 20:50:06 wind kernel: 65532 pages of RAM
Jan 11 20:50:07 wind kernel: 0 pages of HIGHMEM
Jan 11 20:50:08 wind kernel: 1565 reserved pages
Jan 11 20:50:09 wind kernel: 12087 pages shared
Jan 11 20:50:12 wind kernel: 1393 pages swap cached
Jan 11 20:50:12 wind kernel: 0 pages dirty
Jan 11 20:50:13 wind kernel: 97 pages writeback
Jan 11 20:50:16 wind kernel: 2466 pages mapped
Jan 11 20:50:18 wind kernel: 57172 pages slab
Jan 11 20:50:20 wind kernel: 892 pages pagetables
Jan 11 20:50:22 wind kernel: Out of Memory: Killed process 16875 (mysqld).


Version-Release number of selected component (if applicable):
kernel-2.6.15-1.1823_FC4
Comment 1 Orion Poplawski 2006-01-17 16:24:35 EST
Getting these fairly regularly.  Anything else to try/report?
Comment 2 Orion Poplawski 2006-01-20 11:56:05 EST
Still happening in 2.6.15-1.1824_FC4

Jan 19 05:58:01 wind kernel: oom-killer: gfp_mask=0x200d2, order=0
Jan 19 05:58:01 wind kernel: Mem-info:
Jan 19 05:58:01 wind kernel: DMA per-cpu:
Jan 19 05:58:01 wind kernel: cpu 0 hot: low 0, high 0, batch 1 used:0
Jan 19 05:58:01 wind kernel: cpu 0 cold: low 0, high 0, batch 1 used:0
Jan 19 05:58:01 wind kernel: DMA32 per-cpu: empty
Jan 19 05:58:01 wind kernel: Normal per-cpu:
Jan 19 05:58:01 wind kernel: cpu 0 hot: low 0, high 90, batch 15 used:5
Jan 19 05:58:01 wind kernel: cpu 0 cold: low 0, high 30, batch 7 used:17
Jan 19 05:58:01 wind kernel: HighMem per-cpu: empty
Jan 19 05:58:01 wind kernel: Free pages:        3612kB (0kB HighMem)
Jan 19 05:58:01 wind kernel: Active:345 inactive:2594 dirty:0 writeback:97
unstable:0 fre
e:903 slab:57903 mapped:1391 pagetables:979
Jan 19 05:58:07 wind kernel: DMA free:1080kB min:124kB low:152kB high:184kB
active:56kB i
nactive:24kB present:16384kB pages_scanned:144 all_unreclaimable? yes
Jan 19 05:58:07 wind kernel: lowmem_reserve[]: 0 0 239 239
Jan 19 05:58:09 wind kernel: DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB
inactive:
0kB present:0kB pages_scanned:0 all_unreclaimable? no
Jan 19 05:58:14 wind kernel: lowmem_reserve[]: 0 0 239 239
Jan 19 05:58:21 wind kernel: Normal free:2532kB min:1916kB low:2392kB
high:2872kB active:
1324kB inactive:10352kB present:245744kB pages_scanned:402 all_unreclaimable? no
Jan 19 05:58:26 wind kernel: lowmem_reserve[]: 0 0 0 0
Jan 19 05:58:27 wind kernel: HighMem free:0kB min:128kB low:128kB high:128kB
active:0kB i
nactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
Jan 19 05:58:28 wind kernel: lowmem_reserve[]: 0 0 0 0
Jan 19 05:58:30 wind kernel: DMA: 0*4kB 1*8kB 3*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512k
B 1*1024kB 0*2048kB 0*4096kB = 1080kB
Jan 19 05:58:37 wind kernel: DMA32: empty
Jan 19 05:58:37 wind kernel: Normal: 159*4kB 13*8kB 18*16kB 5*32kB 1*64kB
0*128kB 1*256kB
 0*512kB 1*1024kB 0*2048kB 0*4096kB = 2532kB
Jan 19 05:58:37 wind kernel: HighMem: empty
Jan 19 05:58:40 wind kernel: Swap cache: add 1102504, delete 1101835, find
1236027/145767
6, race 2+105
Jan 19 05:58:42 wind kernel: Free swap  = 1926408kB
Jan 19 05:58:47 wind kernel: Total swap = 2097144kB
Jan 19 05:58:49 wind kernel: Free swap:       1926408kB
Jan 19 05:58:53 wind kernel: 65532 pages of RAM
Jan 19 05:58:54 wind kernel: 0 pages of HIGHMEM
Jan 19 05:58:56 wind kernel: 1564 reserved pages
Jan 19 05:59:04 wind kernel: 11829 pages shared
Jan 19 05:59:11 wind kernel: 669 pages swap cached
Jan 19 05:59:16 wind kernel: 0 pages dirty
Jan 19 05:59:17 wind kernel: 97 pages writeback
Jan 19 05:59:21 wind kernel: 1391 pages mapped
Jan 19 05:59:28 wind kernel: 57903 pages slab
Jan 19 05:59:34 wind kernel: 979 pages pagetables
Jan 19 05:59:44 wind kernel: Out of Memory: Killed process 31946 (mysqld).

Hmmm, maybe time to start monitoring mysqld?  Although the oom killer doesn't
just take out the largest process, does it?
Comment 3 Dave Jones 2006-01-24 01:51:13 EST
can I get the output of 'dmesg -s 128000' please ? (As an attachment to the bug
rather than pasted as a comment)
Comment 4 Orion Poplawski 2006-01-24 11:14:16 EST
Created attachment 123618 [details]
dmesg output

This is just the boot up stuff.  System becomes unusable quickly once the oom
stuff starts and I can't do anything on the machine, but hopefully this helps.
Comment 5 Dave Jones 2006-01-24 14:30:10 EST
can you paste the output of slabtop -sc after one of the kills please ?
Comment 6 Orion Poplawski 2006-01-30 11:15:33 EST
 Active / Total Caches (% used)     : 95 / 140 (67.9%)
 Active / Total Size (% used)       : 209553.75K / 211282.30K (99.2%)
 Minimum / Average / Maximum Object : 0.02K / 0.05K / 128.00K

  OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME
1885200 1884784  99%    0.08K  39275       48    157100K bio
1885145 1884696  99%    0.02K  13001      145     52004K biovec-1
289941 289923  99%    0.03K   2283      127      9132K avtab_node
 55272  55140  99%    0.04K    658       84      2632K size-32
   260    260 100%    3.01K    130        2      1040K biovec-(256)
   221    219  99%    4.00K    221        1       884K size-4096
  4922   1854  37%    0.16K    214       23       856K dentry_cache
  7644   7372  96%    0.10K    196       39       784K vm_area_struct
  1044    561  53%    0.61K    174        6       696K ext3_inode_cache
  8300   5050  60%    0.07K    166       50       664K size-64
   109    109 100%    4.00K    109        1       436K pgd
   981    921  93%    0.43K    109        9       436K inode_cache
   265    264  99%    1.51K     53        5       424K biovec-128
  1886   1751  92%    0.17K     82       23       328K filp
   114    108  94%    2.01K     38        3       304K size-2048
   370    343  92%    0.76K     74        5       296K biovec-64
   858    441  51%    0.28K     66       13       264K radix_tree_node
   210    205  97%    1.01K     30        7       240K size-1024
   171    166  97%    1.31K     57        3       228K task_struct
   147    142  96%    1.28K     49        3       196K sighand_cache
  1344   1284  95%    0.14K     48       28       192K size-128
   266    253  95%    0.54K     38        7       152K shmem_inode_cache
   252    252 100%    0.51K     36        7       144K size-512
  2232   2165  96%    0.05K     31       72       124K sysfs_dir_cache
   406    393  96%    0.26K     29       14       116K size-256
    33     32  96%    2.01K     11        3        88K sgpool-128
  2794   1894  67%    0.03K     22      127        88K dm_io
  2794   1894  67%    0.03K     22      127        88K dm_tio
  1919   1819  94%    0.04K     19      101        76K anon_vma
   152    134  88%    0.48K     19        8        76K sock_inode_cache
   361    313  86%    0.20K     19       19        76K biovec-16
   150    143  95%    0.36K     15       10        60K signal_cache
   112    109  97%    0.45K     14        8        56K files_cache
   112    109  97%    0.49K     14        8        56K mm_struct
   299    132  44%    0.17K     13       23        52K blkdev_requests
Connection to wind closed by remote host.
Comment 7 Dave Jones 2006-02-03 01:18:51 EST
This is a mass-update to all currently open kernel bugs.

A new kernel update has been released (Version: 2.6.15-1.1830_FC4)
based upon a new upstream kernel release.

Please retest against this new kernel, as a large number of patches
go into each upstream release, possibly including changes that
may address this problem.

This bug has been placed in NEEDINFO_REPORTER state.
Due to the large volume of inactive bugs in bugzilla, if this bug is
still in this state in two weeks time, it will be closed.

Should this bug still be relevant after this period, the reporter
can reopen the bug at any time. Any other users on the Cc: list
of this bug can request that the bug be reopened by adding a
comment to the bug.

If this bug is a problem preventing you from installing the
release this version is filed against, please see bug 169613.

Thank you.
Comment 8 Orion Poplawski 2006-02-06 12:47:40 EST
Does not appear to be present in 2.6.15-1.1830_FC4

Note You need to log in before you can comment on or make changes to this bug.