Bug 486138 - Discrepancy of free RAM reported by 2.6.9-78.0.13.ELsmp
Discrepancy of free RAM reported by 2.6.9-78.0.13.ELsmp
Status: CLOSED WONTFIX
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: kernel (Show other bugs)
4.7
x86_64 Linux
low Severity low
: rc
: ---
Assigned To: Red Hat Kernel Manager
Red Hat Kernel QE team
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-02-18 11:29 EST by Joseph W. Breu
Modified: 2012-06-20 09:27 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-06-20 09:27:06 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
C utility to alloc 10GB of RAM (919 bytes, text/x-csrc)
2009-02-18 11:29 EST, Joseph W. Breu
no flags Details

  None (edit)
Description Joseph W. Breu 2009-02-18 11:29:40 EST
Created attachment 332409 [details]
C utility to alloc 10GB of RAM

Description of problem:

I've been trying to track down an issue with free RAM reported by the kernel.  The machine in question is a dual quad core Dell 2950 with 32GB of RAM running the 64bit SMP kernel.  The server in question is running mysqld and memcached.

nobody    4425  0.4  5.0 1692088 1654448 ?   Ssl  Feb14  29:36 memcached -d -p 11211 -u nobody -c 10000 -m 15360 -t 4
mysql     4396 54.2 15.1 5163460 4976068 ?   Sl   Feb14 3437:45 /usr/libexec/mysqld --defaults-file=/etc/my.cnf

RSS and VSS sizes reported by ps
ps -eorss,vsz | awk '{RSS+=$1}{VSZ+=$2}END{print "RSS=" RSS/1024/1024 " " "VSZ=" VSZ/1024/1024}'
RSS=6.54633 VSZ=9.81006

Version-Release number of selected component (if applicable):
2.6.9-78.0.5.ELsmp
2.6.9-78.0.13.ELsmp


How reproducible:
Very reproducible - even after reboot

Steps to Reproduce:
1. boot kernel
2. let run for several days
3. check the output of free -m
   - kernel will report almost between 28GB and 29GB of RAM used
4. run attached memtest.c application
   - application allocates 10GB of RAM
5. check free -m output
   - free reports 10GB of RAM available
6. let run for several days
7. check the output of free again.
  
Actual results:


Expected results:


Additional info:


Original 'free -m' output:
--------------------------
             total       used       free     shared    buffers     cached
Mem:         32167      31789        378          0        556       2163
-/+ buffers/cache:      29070       3097
Swap:         1027          0       1027


*** compile and run the attached C application


Output of 'free -m' after the run:
----------------------------------
             total       used       free     shared    buffers     cached
Mem:         32167      21487      10680          0        292        901
-/+ buffers/cache:      20292      11875
Swap:         1027          0       1027


wait for several hours/days and run 'free -m' one more time:
------------------------------------------------------------
             total       used       free     shared    buffers     cached
Mem:         32167      31891        276          0        311       1158
-/+ buffers/cache:      30420       1747
Swap:         1027          0       1027


slabtop shows less than 600MB of total slabs 



cat /proc/meminfo
-----------------
MemTotal:     32939848 kB
MemFree:       5128816 kB
Buffers:        333912 kB
Cached:        1796392 kB
SwapCached:          0 kB
Active:       24179644 kB
Inactive:      2980908 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32939848 kB
LowFree:       5128816 kB
SwapTotal:     1052216 kB
SwapFree:      1051992 kB
Dirty:             260 kB
Writeback:           0 kB
Mapped:        6719992 kB
Slab:           609108 kB
CommitLimit:  17522140 kB
Committed_AS:  7554744 kB
PageTables:      18032 kB
VmallocTotal: 536870911 kB
VmallocUsed:    263080 kB
VmallocChunk: 536607731 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB


cat /proc/slabinfo
------------------ 
slabinfo - version: 2.0
# name            <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <batchcount> <limit> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
rpc_buffers            8      8   2048    2    1 : tunables   24   12    8 : slabdata      4      4      0
rpc_tasks              8     12    320   12    1 : tunables   54   27    8 : slabdata      1      1      0
rpc_inode_cache        6      8    832    4    1 : tunables   54   27    8 : slabdata      2      2      0
msi_cache              2      2   5760    1    2 : tunables    8    4    0 : slabdata      2      2      0
ip_fib_alias          15    119     32  119    1 : tunables  120   60    8 : slabdata      1      1      0
ip_fib_hash           15     61     64   61    1 : tunables  120   60    8 : slabdata      1      1      0
dm_tio                 0      0     24  156    1 : tunables  120   60    8 : slabdata      0      0      0
dm_io                  0      0     40   96    1 : tunables  120   60    8 : slabdata      0      0      0
dm-bvec-(256)          0      0   4096    1    1 : tunables   24   12    8 : slabdata      0      0      0
dm-bvec-128            0      0   2048    2    1 : tunables   24   12    8 : slabdata      0      0      0
dm-bvec-64             0      0   1024    4    1 : tunables   54   27    8 : slabdata      0      0      0
dm-bvec-16             0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
dm-bvec-4              0      0     64   61    1 : tunables  120   60    8 : slabdata      0      0      0
dm-bvec-1              0      0     16  225    1 : tunables  120   60    8 : slabdata      0      0      0
dm-bio                 0      0    128   31    1 : tunables  120   60    8 : slabdata      0      0      0
uhci_urb_priv          0      0     88   45    1 : tunables  120   60    8 : slabdata      0      0      0
ext3_inode_cache   57160  57192    840    4    1 : tunables   54   27    8 : slabdata  14297  14298      0
ext3_xattr           118    225     88   45    1 : tunables  120   60    8 : slabdata      5      5      0
journal_handle       320    324     48   81    1 : tunables  120   60    8 : slabdata      4      4      0
journal_head         889   3526     96   41    1 : tunables  120   60    8 : slabdata     74     86    360
revoke_table           6    225     16  225    1 : tunables  120   60    8 : slabdata      1      1      0
revoke_record         21    119     32  119    1 : tunables  120   60    8 : slabdata      1      1      0
scsi_cmd_cache       297    385    512    7    1 : tunables   54   27    8 : slabdata     55     55    101
sgpool-128            32     32   4096    1    1 : tunables   24   12    8 : slabdata     32     32      0
sgpool-64             32     32   2048    2    1 : tunables   24   12    8 : slabdata     16     16      0
sgpool-32             32     32   1024    4    1 : tunables   54   27    8 : slabdata      8      8      0
sgpool-16             40     48    512    8    1 : tunables   54   27    8 : slabdata      6      6      0
sgpool-8             447    675    256   15    1 : tunables  120   60    8 : slabdata     45     45      0
unix_sock            122    207    832    9    2 : tunables   54   27    8 : slabdata     23     23      0
ip_mrt_cache           0      0    128   31    1 : tunables  120   60    8 : slabdata      0      0      0
tcp_tw_bucket        101    180    192   20    1 : tunables  120   60    8 : slabdata      9      9      0
tcp_bind_bucket       36    238     32  119    1 : tunables  120   60    8 : slabdata      2      2      0
tcp_open_request      47     62    128   31    1 : tunables  120   60    8 : slabdata      2      2      0
inet_peer_cache        3     31    128   31    1 : tunables  120   60    8 : slabdata      1      1      0
secpath_cache          0      0    192   20    1 : tunables  120   60    8 : slabdata      0      0      0
xfrm_dst_cache         0      0    384   10    1 : tunables   54   27    8 : slabdata      0      0      0
ip_dst_cache          59    110    384   10    1 : tunables   54   27    8 : slabdata     11     11      0
arp_cache             16     30    256   15    1 : tunables  120   60    8 : slabdata      2      2      0
raw_sock               9      9    832    9    2 : tunables   54   27    8 : slabdata      1      1      0
udp_sock              10     45    832    9    2 : tunables   54   27    8 : slabdata      5      5      0
tcp_sock              76    110   1536    5    2 : tunables   24   12    8 : slabdata     22     22      1
flow_cache             0      0    128   31    1 : tunables  120   60    8 : slabdata      0      0      0
mqueue_inode_cache      1      4    896    4    1 : tunables   54   27    8 : slabdata      1      1      0
relayfs_inode_cache      0      0    576    7    1 : tunables   54   27    8 : slabdata      0      0      0
isofs_inode_cache      0      0    616    6    1 : tunables   54   27    8 : slabdata      0      0      0
hugetlbfs_inode_cache      1      6    608    6    1 : tunables   54   27    8 : slabdata      1      1      0
ext2_inode_cache       0      0    736    5    1 : tunables   54   27    8 : slabdata      0      0      0
ext2_xattr             0      0     88   45    1 : tunables  120   60    8 : slabdata      0      0      0
dquot                  0      0    224   17    1 : tunables  120   60    8 : slabdata      0      0      0
eventpoll_pwq        125    324     72   54    1 : tunables  120   60    8 : slabdata      6      6      0
eventpoll_epi        126    280    192   20    1 : tunables  120   60    8 : slabdata     14     14      0
kioctx                 0      0    384   10    1 : tunables   54   27    8 : slabdata      0      0      0
kiocb                  0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
dnotify_cache          1     96     40   96    1 : tunables  120   60    8 : slabdata      1      1      0
fasync_cache           1    156     24  156    1 : tunables  120   60    8 : slabdata      1      1      0
shmem_inode_cache    353    390    800    5    1 : tunables   54   27    8 : slabdata     78     78      0
posix_timers_cache      0      0    184   21    1 : tunables  120   60    8 : slabdata      0      0      0
uid_cache              7     62    128   31    1 : tunables  120   60    8 : slabdata      2      2      0
cfq_pool             446    552     56   69    1 : tunables  120   60    8 : slabdata      8      8      0
crq_pool             482    972     72   54    1 : tunables  120   60    8 : slabdata     18     18     37
deadline_drq           0      0     96   41    1 : tunables  120   60    8 : slabdata      0      0      0
as_arq                 0      0    112   35    1 : tunables  120   60    8 : slabdata      0      0      0
blkdev_ioc            86    238     32  119    1 : tunables  120   60    8 : slabdata      2      2      0
blkdev_queue          22     27    848    9    2 : tunables   54   27    8 : slabdata      3      3      0
blkdev_requests      281    405    264   15    1 : tunables   54   27    8 : slabdata     27     27      0
biovec-(256)         256    256   4096    1    1 : tunables   24   12    8 : slabdata    256    256      0
biovec-128           256    256   2048    2    1 : tunables   24   12    8 : slabdata    128    128      0
biovec-64            256    256   1024    4    1 : tunables   54   27    8 : slabdata     64     64      0
biovec-16            256    270    256   15    1 : tunables  120   60    8 : slabdata     18     18      0
biovec-4             256    305     64   61    1 : tunables  120   60    8 : slabdata      5      5      0
biovec-1             870   2700     16  225    1 : tunables  120   60    8 : slabdata     12     12    132
bio                  912   1736    128   31    1 : tunables  120   60    8 : slabdata     56     56    156
file_lock_cache       53    125    160   25    1 : tunables  120   60    8 : slabdata      5      5      0
sock_inode_cache     264    325    704    5    1 : tunables   54   27    8 : slabdata     65     65      0
skbuff_head_cache    664    804    320   12    1 : tunables   54   27    8 : slabdata     67     67     54
sock                   6      6    640    6    1 : tunables   54   27    8 : slabdata      1      1      0
proc_inode_cache    2904   2934    600    6    1 : tunables   54   27    8 : slabdata    489    489      0
sigqueue             529    529    168   23    1 : tunables  120   60    8 : slabdata     23     23      0
radix_tree_node    27977  63049    536    7    1 : tunables   54   27    8 : slabdata   9007   9007    184
bdev_cache            22     35    768    5    1 : tunables   54   27    8 : slabdata      7      7      0
mnt_cache             28     80    192   20    1 : tunables  120   60    8 : slabdata      4      4      0
audit_watch_cache      0      0     88   45    1 : tunables  120   60    8 : slabdata      0      0      0
inode_cache         2178   2233    568    7    1 : tunables   54   27    8 : slabdata    319    319      0
dentry_cache       85292  85520    240   16    1 : tunables  120   60    8 : slabdata   5345   5345      0
filp                1902   2025    256   15    1 : tunables  120   60    8 : slabdata    135    135      0
names_cache           22     23   4096    1    1 : tunables   24   12    8 : slabdata     22     23      0
avc_node              12    540     72   54    1 : tunables  120   60    8 : slabdata     10     10      0
key_jar               14     40    192   20    1 : tunables  120   60    8 : slabdata      2      2      0
idr_layer_cache       62     63    528    7    1 : tunables   54   27    8 : slabdata      9      9      0
buffer_head       4879932 5324085     88   45    1 : tunables  120   60    8 : slabdata 118313 118313    480
mm_struct             73    175   1152    7    2 : tunables   24   12    8 : slabdata     25     25      0
vm_area_struct      3668   4642    176   22    1 : tunables  120   60    8 : slabdata    211    211      0
fs_cache              83    549     64   61    1 : tunables  120   60    8 : slabdata      9      9      0
files_cache           84    207    832    9    2 : tunables   54   27    8 : slabdata     23     23      0
signal_cache         152    324    320   12    1 : tunables   54   27    8 : slabdata     27     27      0
sighand_cache        131    174   2112    3    2 : tunables   24   12    8 : slabdata     58     58      0
task_struct          252    268   1984    2    1 : tunables   24   12    8 : slabdata    134    134      0
anon_vma            1123   2184     24  156    1 : tunables  120   60    8 : slabdata     14     14      0
shared_policy_node      0      0     56   69    1 : tunables  120   60    8 : slabdata      0      0      0
numa_policy           58    450     16  225    1 : tunables  120   60    8 : slabdata      2      2      0
size-131072(DMA)       0      0 131072    1   32 : tunables    8    4    0 : slabdata      0      0      0
size-131072            1      1 131072    1   32 : tunables    8    4    0 : slabdata      1      1      0
size-65536(DMA)        0      0  65536    1   16 : tunables    8    4    0 : slabdata      0      0      0
size-65536             0      0  65536    1   16 : tunables    8    4    0 : slabdata      0      0      0
size-32768(DMA)        0      0  32768    1    8 : tunables    8    4    0 : slabdata      0      0      0
size-32768             0      0  32768    1    8 : tunables    8    4    0 : slabdata      0      0      0
size-16384(DMA)        0      0  16384    1    4 : tunables    8    4    0 : slabdata      0      0      0
size-16384             2      2  16384    1    4 : tunables    8    4    0 : slabdata      2      2      0
size-8192(DMA)         0      0   8192    1    2 : tunables    8    4    0 : slabdata      0      0      0
size-8192             27     27   8192    1    2 : tunables    8    4    0 : slabdata     27     27      0
size-4096(DMA)         0      0   4096    1    1 : tunables   24   12    8 : slabdata      0      0      0
size-4096            200    200   4096    1    1 : tunables   24   12    8 : slabdata    200    200     36
size-2048(DMA)         0      0   2048    2    1 : tunables   24   12    8 : slabdata      0      0      0
size-2048            659    698   2048    2    1 : tunables   24   12    8 : slabdata    347    349      0
size-1620(DMA)         0      0   1664    4    2 : tunables   24   12    8 : slabdata      0      0      0
size-1620             59     80   1664    4    2 : tunables   24   12    8 : slabdata     20     20      0
size-1024(DMA)         0      0   1024    4    1 : tunables   54   27    8 : slabdata      0      0      0
size-1024            791   2120   1024    4    1 : tunables   54   27    8 : slabdata    530    530     27
size-512(DMA)          0      0    512    8    1 : tunables   54   27    8 : slabdata      0      0      0
size-512             501   2304    512    8    1 : tunables   54   27    8 : slabdata    288    288      0
size-256(DMA)          0      0    256   15    1 : tunables  120   60    8 : slabdata      0      0      0
size-256            1242   2640    256   15    1 : tunables  120   60    8 : slabdata    176    176      0
size-128(DMA)          0      0    128   31    1 : tunables  120   60    8 : slabdata      0      0      0
size-128            6043   6386    128   31    1 : tunables  120   60    8 : slabdata    206    206      0
size-64(DMA)           0      0     64   61    1 : tunables  120   60    8 : slabdata      0      0      0
size-64             4265  17568     64   61    1 : tunables  120   60    8 : slabdata    288    288      2
size-32(DMA)           0      0     32  119    1 : tunables  120   60    8 : slabdata      0      0      0
size-32              865   9163     32  119    1 : tunables  120   60    8 : slabdata     77     77      0
kmem_cache           195    195    256   15    1 : tunables  120   60    8 : slabdata     13     13      0

cat /proc/vmstat
----------------
nr_dirty 67
nr_writeback 0
nr_unstable 0
nr_page_table_pages 4508
nr_mapped 1679906
nr_slab 152082
pgpgin 161740656
pgpgout 2044078034
pswpin 0
pswpout 56
pgalloc_high 0
pgalloc_normal 1929898247
pgalloc_dma 18996
pgfree 1931195021
pgactivate 1556410332
pgdeactivate 166636526
pgfault 139552830
pgmajfault 13054
pgrefill_high 0
pgrefill_normal 216940503
pgrefill_dma 0
pgsteal_high 0
pgsteal_normal 211093846
pgsteal_dma 0
pgscan_kswapd_high 0
pgscan_kswapd_normal 217853559
pgscan_kswapd_dma 0
pgscan_direct_high 0
pgscan_direct_normal 24552
pgscan_direct_dma 0
pginodesteal 0
slabs_scanned 4224
kswapd_steal 211085861
kswapd_inodesteal 0
pageoutrun 6628867
allocstall 238
pgrotated 114


sysctl -a
---------
sunrpc.tcp_slot_table_entries = 16
sunrpc.udp_slot_table_entries = 16
sunrpc.max_resvport = 1023
sunrpc.min_resvport = 665
sunrpc.nlm_debug = 0
sunrpc.nfsd_debug = 0
sunrpc.nfs_debug = 0
sunrpc.rpc_debug = 0
mc.check_pci_parity = 0
mc.panic_on_pci_parity = 0
mc.poll_msec = 1000
mc.log_ce = 1
mc.log_ue = 1
mc.panic_on_ue = 1
abi.vsyscall32 = 1
dev.scsi.logging_level = 0
dev.raid.speed_limit_max = 200000
dev.raid.speed_limit_min = 1000
dev.cdrom.check_media = 0
dev.cdrom.lock = 1
dev.cdrom.debug = 0
dev.cdrom.autoeject = 0
dev.cdrom.autoclose = 1
dev.cdrom.info = CD-ROM information, Id: cdrom.c 3.20 2003/12/17
dev.cdrom.info = 
dev.cdrom.info = drive name:	
dev.cdrom.info = drive speed:	
dev.cdrom.info = drive # of slots:
dev.cdrom.info = Can close tray:	
dev.cdrom.info = Can open tray:	
dev.cdrom.info = Can lock tray:	
dev.cdrom.info = Can change speed:
dev.cdrom.info = Can select disk:
dev.cdrom.info = Can read multisession:
dev.cdrom.info = Can read MCN:	
dev.cdrom.info = Reports media changed:
dev.cdrom.info = Can play audio:	
dev.cdrom.info = Can write CD-R:	
dev.cdrom.info = Can write CD-RW:
dev.cdrom.info = Can read DVD:	
dev.cdrom.info = Can write DVD-R:
dev.cdrom.info = Can write DVD-RAM:
dev.cdrom.info = Can read MRW:	
dev.cdrom.info = Can write MRW:	
dev.cdrom.info = Can write RAM:	
dev.cdrom.info = 
dev.cdrom.info = 
dev.rtc.max-user-freq = 64
debug.exception-trace = 1
net.unix.max_dgram_qlen = 10
net.token-ring.rif_timeout = 600000
net.ipv4.conf.eth1.force_igmp_version = 0
net.ipv4.conf.eth1.disable_policy = 0
net.ipv4.conf.eth1.disable_xfrm = 0
net.ipv4.conf.eth1.arp_ignore = 0
net.ipv4.conf.eth1.arp_announce = 0
net.ipv4.conf.eth1.arp_filter = 0
net.ipv4.conf.eth1.tag = 0
net.ipv4.conf.eth1.log_martians = 0
net.ipv4.conf.eth1.bootp_relay = 0
net.ipv4.conf.eth1.medium_id = 0
net.ipv4.conf.eth1.proxy_arp = 0
net.ipv4.conf.eth1.accept_source_route = 0
net.ipv4.conf.eth1.send_redirects = 1
net.ipv4.conf.eth1.rp_filter = 1
net.ipv4.conf.eth1.shared_media = 1
net.ipv4.conf.eth1.secure_redirects = 1
net.ipv4.conf.eth1.accept_redirects = 1
net.ipv4.conf.eth1.mc_forwarding = 0
net.ipv4.conf.eth1.forwarding = 0
net.ipv4.conf.eth0.force_igmp_version = 0
net.ipv4.conf.eth0.disable_policy = 0
net.ipv4.conf.eth0.disable_xfrm = 0
net.ipv4.conf.eth0.arp_ignore = 0
net.ipv4.conf.eth0.arp_announce = 0
net.ipv4.conf.eth0.arp_filter = 0
net.ipv4.conf.eth0.tag = 0
net.ipv4.conf.eth0.log_martians = 0
net.ipv4.conf.eth0.bootp_relay = 0
net.ipv4.conf.eth0.medium_id = 0
net.ipv4.conf.eth0.proxy_arp = 0
net.ipv4.conf.eth0.accept_source_route = 0
net.ipv4.conf.eth0.send_redirects = 1
net.ipv4.conf.eth0.rp_filter = 1
net.ipv4.conf.eth0.shared_media = 1
net.ipv4.conf.eth0.secure_redirects = 1
net.ipv4.conf.eth0.accept_redirects = 1
net.ipv4.conf.eth0.mc_forwarding = 0
net.ipv4.conf.eth0.forwarding = 0
net.ipv4.conf.lo.force_igmp_version = 0
net.ipv4.conf.lo.disable_policy = 0
net.ipv4.conf.lo.disable_xfrm = 0
net.ipv4.conf.lo.arp_ignore = 0
net.ipv4.conf.lo.arp_announce = 0
net.ipv4.conf.lo.arp_filter = 0
net.ipv4.conf.lo.tag = 0
net.ipv4.conf.lo.log_martians = 0
net.ipv4.conf.lo.bootp_relay = 0
net.ipv4.conf.lo.medium_id = 0
net.ipv4.conf.lo.proxy_arp = 0
net.ipv4.conf.lo.accept_source_route = 0
net.ipv4.conf.lo.send_redirects = 1
net.ipv4.conf.lo.rp_filter = 1
net.ipv4.conf.lo.shared_media = 1
net.ipv4.conf.lo.secure_redirects = 1
net.ipv4.conf.lo.accept_redirects = 1
net.ipv4.conf.lo.mc_forwarding = 0
net.ipv4.conf.lo.forwarding = 0
net.ipv4.conf.default.force_igmp_version = 0
net.ipv4.conf.default.disable_policy = 0
net.ipv4.conf.default.disable_xfrm = 0
net.ipv4.conf.default.arp_ignore = 0
net.ipv4.conf.default.arp_announce = 0
net.ipv4.conf.default.arp_filter = 0
net.ipv4.conf.default.tag = 0
net.ipv4.conf.default.log_martians = 0
net.ipv4.conf.default.bootp_relay = 0
net.ipv4.conf.default.medium_id = 0
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.shared_media = 1
net.ipv4.conf.default.secure_redirects = 1
net.ipv4.conf.default.accept_redirects = 1
net.ipv4.conf.default.mc_forwarding = 0
net.ipv4.conf.default.forwarding = 0
net.ipv4.conf.all.force_igmp_version = 0
net.ipv4.conf.all.disable_policy = 0
net.ipv4.conf.all.disable_xfrm = 0
net.ipv4.conf.all.arp_ignore = 0
net.ipv4.conf.all.arp_announce = 0
net.ipv4.conf.all.arp_filter = 0
net.ipv4.conf.all.tag = 0
net.ipv4.conf.all.log_martians = 0
net.ipv4.conf.all.bootp_relay = 0
net.ipv4.conf.all.medium_id = 0
net.ipv4.conf.all.proxy_arp = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.all.send_redirects = 1
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.all.shared_media = 1
net.ipv4.conf.all.secure_redirects = 1
net.ipv4.conf.all.accept_redirects = 1
net.ipv4.conf.all.mc_forwarding = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.neigh.eth1.locktime = 99
net.ipv4.neigh.eth1.proxy_delay = 79
net.ipv4.neigh.eth1.anycast_delay = 99
net.ipv4.neigh.eth1.proxy_qlen = 64
net.ipv4.neigh.eth1.unres_qlen = 3
net.ipv4.neigh.eth1.gc_stale_time = 60
net.ipv4.neigh.eth1.delay_first_probe_time = 5
net.ipv4.neigh.eth1.base_reachable_time = 30
net.ipv4.neigh.eth1.retrans_time = 99
net.ipv4.neigh.eth1.app_solicit = 0
net.ipv4.neigh.eth1.ucast_solicit = 3
net.ipv4.neigh.eth1.mcast_solicit = 3
net.ipv4.neigh.eth0.locktime = 99
net.ipv4.neigh.eth0.proxy_delay = 79
net.ipv4.neigh.eth0.anycast_delay = 99
net.ipv4.neigh.eth0.proxy_qlen = 64
net.ipv4.neigh.eth0.unres_qlen = 3
net.ipv4.neigh.eth0.gc_stale_time = 60
net.ipv4.neigh.eth0.delay_first_probe_time = 5
net.ipv4.neigh.eth0.base_reachable_time = 30
net.ipv4.neigh.eth0.retrans_time = 99
net.ipv4.neigh.eth0.app_solicit = 0
net.ipv4.neigh.eth0.ucast_solicit = 3
net.ipv4.neigh.eth0.mcast_solicit = 3
net.ipv4.neigh.lo.locktime = 99
net.ipv4.neigh.lo.proxy_delay = 79
net.ipv4.neigh.lo.anycast_delay = 99
net.ipv4.neigh.lo.proxy_qlen = 64
net.ipv4.neigh.lo.unres_qlen = 3
net.ipv4.neigh.lo.gc_stale_time = 60
net.ipv4.neigh.lo.delay_first_probe_time = 5
net.ipv4.neigh.lo.base_reachable_time = 30
net.ipv4.neigh.lo.retrans_time = 99
net.ipv4.neigh.lo.app_solicit = 0
net.ipv4.neigh.lo.ucast_solicit = 3
net.ipv4.neigh.lo.mcast_solicit = 3
net.ipv4.neigh.default.gc_thresh3 = 1024
net.ipv4.neigh.default.gc_thresh2 = 512
net.ipv4.neigh.default.gc_thresh1 = 128
net.ipv4.neigh.default.gc_interval = 30
net.ipv4.neigh.default.locktime = 99
net.ipv4.neigh.default.proxy_delay = 79
net.ipv4.neigh.default.anycast_delay = 99
net.ipv4.neigh.default.proxy_qlen = 64
net.ipv4.neigh.default.unres_qlen = 3
net.ipv4.neigh.default.gc_stale_time = 60
net.ipv4.neigh.default.delay_first_probe_time = 5
net.ipv4.neigh.default.base_reachable_time = 30
net.ipv4.neigh.default.retrans_time = 99
net.ipv4.neigh.default.app_solicit = 0
net.ipv4.neigh.default.ucast_solicit = 3
net.ipv4.neigh.default.mcast_solicit = 3
net.ipv4.tcp_slow_start_after_idle = 1
net.ipv4.tcp_workaround_signed_windows = 1
net.ipv4.tcp_bic_beta = 819
net.ipv4.tcp_tso_win_divisor = 8
net.ipv4.tcp_moderate_rcvbuf = 1
net.ipv4.tcp_bic_low_window = 14
net.ipv4.tcp_bic_fast_convergence = 1
net.ipv4.tcp_bic = 1
net.ipv4.tcp_vegas_gamma = 2
net.ipv4.tcp_vegas_beta = 6
net.ipv4.tcp_vegas_alpha = 2
net.ipv4.tcp_vegas_cong_avoid = 0
net.ipv4.tcp_westwood = 0
net.ipv4.tcp_no_metrics_save = 0
net.ipv4.ipfrag_secret_interval = 600
net.ipv4.tcp_low_latency = 0
net.ipv4.tcp_frto = 0
net.ipv4.tcp_tw_reuse = 0
net.ipv4.icmp_ratemask = 6168
net.ipv4.icmp_ratelimit = 1000
net.ipv4.tcp_adv_win_scale = 2
net.ipv4.tcp_app_win = 31
net.ipv4.tcp_rmem = 4096	87380	174760
net.ipv4.tcp_wmem = 4096	16384	131072
net.ipv4.tcp_mem = 786432	1048576	1572864
net.ipv4.tcp_dsack = 1
net.ipv4.tcp_ecn = 0
net.ipv4.tcp_reordering = 3
net.ipv4.tcp_fack = 1
net.ipv4.tcp_orphan_retries = 0
net.ipv4.inet_peer_gc_maxtime = 120
net.ipv4.inet_peer_gc_mintime = 10
net.ipv4.inet_peer_maxttl = 600
net.ipv4.inet_peer_minttl = 120
net.ipv4.inet_peer_threshold = 65664
net.ipv4.igmp_max_msf = 10
net.ipv4.igmp_max_memberships = 20
net.ipv4.route.secret_interval = 600
net.ipv4.route.min_adv_mss = 256
net.ipv4.route.min_pmtu = 552
net.ipv4.route.mtu_expires = 600
net.ipv4.route.gc_elasticity = 8
net.ipv4.route.error_burst = 5000
net.ipv4.route.error_cost = 1000
net.ipv4.route.redirect_silence = 20480
net.ipv4.route.redirect_number = 9
net.ipv4.route.redirect_load = 20
net.ipv4.route.gc_interval = 60
net.ipv4.route.gc_timeout = 300
net.ipv4.route.gc_min_interval = 0
net.ipv4.route.max_size = 8388608
net.ipv4.route.gc_thresh = 524288
net.ipv4.route.max_delay = 10
net.ipv4.route.min_delay = 2
net.ipv4.icmp_errors_use_inbound_ifaddr = 0
net.ipv4.icmp_ignore_bogus_error_responses = 0
net.ipv4.icmp_echo_ignore_broadcasts = 0
net.ipv4.icmp_echo_ignore_all = 0
net.ipv4.ip_local_port_range = 32768	61000
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_rfc1337 = 0
net.ipv4.tcp_stdurg = 0
net.ipv4.tcp_abort_on_overflow = 0
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_fin_timeout = 60
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_keepalive_intvl = 75
net.ipv4.tcp_keepalive_probes = 9
net.ipv4.tcp_keepalive_time = 7200
net.ipv4.ipfrag_time = 30
net.ipv4.ip_dynaddr = 0
net.ipv4.ipfrag_low_thresh = 196608
net.ipv4.ipfrag_high_thresh = 262144
net.ipv4.tcp_max_tw_buckets = 180000
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_synack_retries = 5
net.ipv4.tcp_syn_retries = 5
net.ipv4.ip_nonlocal_bind = 0
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.ip_autoconfig = 0
net.ipv4.ip_default_ttl = 64
net.ipv4.ip_forward = 0
net.ipv4.tcp_retrans_collapse = 1
net.ipv4.tcp_sack = 1
net.ipv4.tcp_window_scaling = 1
net.ipv4.tcp_timestamps = 1
net.core.somaxconn = 128
net.core.divert_version = 0.46
net.core.optmem_max = 20480
net.core.message_burst = 10
net.core.message_cost = 5
net.core.mod_cong = 290
net.core.lo_cong = 100
net.core.no_cong = 20
net.core.no_cong_thresh = 10
net.core.netdev_max_backlog = 300
net.core.dev_weight = 64
net.core.rmem_default = 135168
net.core.wmem_default = 135168
net.core.rmem_max = 135168
net.core.wmem_max = 135168
vm.write-mapped = 1
vm.drop_caches = 0
vm.inactive_percent = 0
vm.pagecache = 100
vm.percpu_pagelist_fraction = 0
vm.max_queue_depth = 0
vm.oom-kill = 1
vm.legacy_va_layout = 0
vm.vfs_cache_pressure = 100
vm.block_dump = 0
vm.laptop_mode = 0
vm.max_map_count = 65536
vm.min_free_kbytes = 23163
vm.lower_zone_protection = 0
vm.hugetlb_shm_group = 0
vm.nr_hugepages = 0
vm.swappiness = 60
vm.nr_pdflush_threads = 2
vm.dirty_expire_centisecs = 3000
vm.dirty_writeback_centisecs = 500
vm.dirty_ratio = 40
vm.dirty_background_ratio = 10
vm.page-cluster = 3
vm.overcommit_ratio = 50
vm.panic_on_oom = 0
vm.overcommit_memory = 0
kernel.panic_on_unrecovered_nmi = 0
kernel.wake_balance = 1
kernel.suid_dumpable = 0
kernel.unknown_nmi_panic = 0
kernel.ngroups_max = 65536
kernel.printk_ratelimit_burst = 10
kernel.printk_ratelimit = 5
kernel.panic_on_oops = 1
kernel.pid_max = 32768
kernel.sercons_esc = -1
kernel.overflowgid = 65534
kernel.overflowuid = 65534
kernel.pty.nr = 2
kernel.pty.max = 4096
kernel.random.uuid = 1ba1a531-fdfa-431d-ab4c-7bda5d846470
kernel.random.boot_id = e2e96ec5-921b-4b63-a7af-03395928fd3e
kernel.random.write_wakeup_threshold = 128
kernel.random.read_wakeup_threshold = 64
kernel.random.entropy_avail = 3840
kernel.random.poolsize = 512
kernel.threads-max = 557056
kernel.cad_pid = 1
kernel.sysrq = 0
kernel.sem = 250	32000	32	128
kernel.msgmnb = 16384
kernel.msgmni = 16
kernel.msgmax = 8192
kernel.shmmni = 4096
kernel.shmall = 2097152
kernel.shmmax = 33554432
kernel.acct = 4	2	30
kernel.hotplug = /sbin/hotplug
kernel.modprobe = /sbin/modprobe
kernel.printk = 6	4	1	7
kernel.ctrl-alt-del = 0
kernel.real-root-dev = 0
kernel.cap-bound = -257
kernel.tainted = 0
kernel.core_pattern = /root/core
kernel.core_uses_pid = 1
kernel.print-fatal-signals = 0
kernel.exec-shield-randomize = 1
kernel.exec-shield = 1
kernel.panic = 0
kernel.domainname = (none)
kernel.hostname = HOSTNAME.DOMAIN.TLD
kernel.version = #1 SMP Wed Jan 7 17:45:52 EST 2009
kernel.osrelease = 2.6.9-78.0.13.ELsmp
kernel.ostype = Linux
fs.mqueue.msgsize_max = 8192
fs.mqueue.msg_max = 10
fs.mqueue.queues_max = 256
fs.quota.syncs = 18
fs.quota.free_dquots = 0
fs.quota.allocated_dquots = 0
fs.quota.cache_hits = 0
fs.quota.writes = 0
fs.quota.reads = 0
fs.quota.drops = 0
fs.quota.lookups = 0
fs.aio-max-nr = 65536
fs.aio-nr = 0
fs.lease-break-time = 45
fs.dir-notify-enable = 1
fs.leases-enable = 1
fs.overflowgid = 65534
fs.overflowuid = 65534
fs.dentry-state = 85109	70641	45	0	0	0
fs.file-max = 3158299
fs.file-nr = 2100	0	3158299
fs.inode-state = 62765	0	0	0	0	0	0
fs.inode-nr = 62765	0
Comment 1 Sriram 2009-02-27 15:53:22 EST
I had a similar issue with the 2.6.18-92.1.6.el5 #1 SMP  kernel also.

/proc/pid/status does not get updated with the correct RSS. It stays like that for several hours.

It only reclaims memory when I run a program that requests more memory.

I understand that is some kind of delayed garbage collection. But utilities like free and top are fooled into reporting incorrect results.

Sriram
Comment 2 Jiri Pallich 2012-06-20 09:27:06 EDT
Thank you for submitting this issue for consideration in Red Hat Enterprise Linux. The release for which you requested us to review is now End of Life. 
Please See https://access.redhat.com/support/policy/updates/errata/

If you would like Red Hat to re-consider your feature request for an active release, please re-open the request via appropriate support channels and provide additional supporting details about the importance of this issue.

Note You need to log in before you can comment on or make changes to this bug.