Bug 97586 - ps reports incorrect size for multithreaded programs
Summary: ps reports incorrect size for multithreaded programs
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: procps   
(Show other bugs)
Version: 9
Hardware: i686
OS: Linux
Target Milestone: ---
Assignee: Alexander Larsson
QA Contact: Brian Brock
Depends On:
TreeView+ depends on / blocked
Reported: 2003-06-18 01:20 UTC by Joseph Shraibman
Modified: 2007-04-18 16:54 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2003-09-26 09:26:50 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

Description Joseph Shraibman 2003-06-18 01:20:06 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030529

Description of problem:
[jks@xx ~] ps ax  -w -w -o pid,rss,size,args --sort size
30993 66024 132424 ./mozilla-bin
 8093 9212 178764 java <snip>
 8277 10832 182112 java <snip>
 8208 16840 183296 java <snip>
  671 6140 184484 rmiregistry
 8335 11392 378684 java <snip>
21648 31948 661340 java <snip>
 8151 17660 907776 java <snip>
[jks@xx ~] free
             total       used       free     shared    buffers     cached
Mem:        514592     507468       7124          0      15996     142692
-/+ buffers/cache:     348780     165812
Swap:      1305568     312140     993428

If the total memory used on the system is about 820 megs, then obviously one
java app can't be taking up more than 900 megs by itself.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Run a program with multiple threads (like mozilla)
2. compare ps size with rss. If swap space isn't being used then size should be
higher than rss.

Comment 1 Alexander Larsson 2003-07-10 09:59:14 UTC
This is almost certainly the virtual memory for the thread stacks showing up.
Unless apps change the thread stack size they get an 8 meg stack by default.
However, the stack is mapped in a way that the memory isn't actually allocated
until it is used.

Looking at 'pmap <mozilla-pid>' on my machine i see this:
42134000 (4 KB)        ---p (00:00 0)        
42135000 (8192 KB)     rwxp (00:00 0)        
42935000 (4 KB)        ---p (00:00 0)        
42936000 (8192 KB)     rwxp (00:00 0)        
43136000 (4 KB)        ---p (00:00 0)        
43137000 (8192 KB)     rwxp (00:00 0)        
43937000 (4 KB)        ---p (00:00 0)        
43938000 (8192 KB)     rwxp (00:00 0)        
44138000 (4 KB)        ---p (00:00 0)        
44139000 (8192 KB)     rwxp (00:00 0)        
44939000 (4 KB)        ---p (00:00 0)        
4493a000 (8192 KB)     rwxp (00:00 0)        

Those are the thread stacks and the stack overflow protection pages.

Comment 2 Joseph Shraibman 2003-07-10 16:38:07 UTC
So how is this not a bug?  It is still reporting more memory used than is
actually used.

Comment 3 Alexander Larsson 2003-07-11 07:39:34 UTC
No, its not. The RSS field reports how much memory is used, and the size field
how much is mapped.

Comment 4 Joseph Shraibman 2003-07-11 15:29:58 UTC
RSS shows how much of actual memory the process is using, excluding memory that
currently resides in swap.  size is supposed to be the total size, not the "size
that is mapped".

If you want to say that SIZE is supposed to show how much memory is mapped, then
we need another option to show the real size, so this is a bug either way.

Comment 5 Alexander Larsson 2003-08-05 09:06:39 UTC
How do you define "total size" then?

Comment 6 Joseph Shraibman 2003-08-05 17:49:57 UTC
size should report the memory allocated, including memory in swap.  If you kill
the program that is how much free memory you will gain.  I really don't see how
you can be confused by this.

Comment 7 Alexander Larsson 2003-08-06 11:15:19 UTC
Maybe mapped was a bad word. The threadstack memory is anonymous memory
allocated to the process, and not mapped to a file, so its real allocated
memory. However, its pages are initially only allocated in the page table, and
actual pages are not allocated on swap/ram until the page in question is written
to (copy-on-write versions of the zero page). There is no way to find out from
the kernel how much of this memory is "really" allocated.

However, looking at this i noticed a strange thing, it seems the threadstack
memory gets displayed as data, and not swap. I wonder why VM_GROWSDOWN is not
set on the thread stack mappings.

Comment 8 Joseph Shraibman 2003-09-25 22:34:19 UTC
It's amazing: ps on redhat 8 doesn't work like this, top on redhat 9 doesn't
work like this, but you still won't acknowledge that this is a bug.

Comment 9 Dan Kegel 2003-09-25 23:22:48 UTC
Doesn't look like a bug to me.  SZ is allocated address space,
not allocated memory.  Always has been.

If an app has buttloads of threads, most of which have
never used much of their stack allocation, the total of the SZ column in ps
could indeed be lower than the total of the 'used' column in the
output of free.  SZ can include address space for which neither
physical memory nor swap space has been committed yet. 

Comment 10 Joseph Shraibman 2003-09-25 23:46:28 UTC
Maybe I misstated the problem.  Why is ps on redhat 9 using SZ when I passed in
'size' as the argument?  redhat 8 doesn't have this problem.

rh8># ps -w -w -o pid,rss,size,sz,args 478
  478 17364 17368 238551 java <snip>

rh9># ps -w -w -o pid,rss,size,sz,args 5359
 5359 14320 179740 55927 java <snip>

... so maybe the ps on rh9 got sz and size backward?  Except 55927 looks a
little big for what I expect my java program to be (more like 20000).

Comment 11 Joseph Shraibman 2003-09-26 00:03:17 UTC
Actually I might have a rawhide version of procps on my rh8 machine, so I can't
say that size worked in redhat 8.

Comment 12 Joseph Shraibman 2003-09-26 00:14:18 UTC
Looking at the source code, it seems that in redhat 9 m_size is what I want,
which on redhat 8 produced "-".  Was there any reason in particular to rename
size to m_size?

On rh9 the 'size' argument calls this:
/* approximation to: kB of address space that could end up in swap */
static int pr_swapable(void)
	return sprintf(outbuf, "%lld", pp->vm_data + pp->vm_stack);

In redhat 8 there doesn't appear to be any argument that resembeles that.

Comment 13 Alexander Larsson 2003-09-26 09:26:50 UTC
Robert Love, the current procps maintainer has this to say:

#97586 is not a bug

So. For the n:th time i will close this bug. reopening it again will not help
you. If you have ideas about how procps should report size, bring it up at
procps-list@redhat.com where the procps developers can answer you.

Note You need to log in before you can comment on or make changes to this bug.