Bug 88047 - /proc/<pid>/cmdline is empty in RHL8
Summary: /proc/<pid>/cmdline is empty in RHL8
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: kernel
Version: 9
Hardware: athlon
OS: Linux
Target Milestone: ---
Assignee: Arjan van de Ven
QA Contact: Brian Brock
Depends On:
TreeView+ depends on / blocked
Reported: 2003-04-04 21:08 UTC by Jordan Russell
Modified: 2005-10-31 22:00 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2003-06-08 14:51:48 UTC

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2003:172 0 normal SHIPPED_LIVE : Updated 2.4 kernel fixes security vulnerabilities and various bugs 2003-09-05 04:00:00 UTC

Description Jordan Russell 2003-04-04 21:08:53 UTC
Description of problem:
Inexplicably, with the new 2.4.20 kernel, some processes (usually daemons) have 
an empty (zero-byte) /proc/<pid>/cmdline file. Consequently, "ps ax" will not 
show the processes' command lines.

I read that this can happen if processes are swapped out of memory, so I tried 
disabling swap (swapoff -a) and restarting the processes, but the problem was 
still there.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
# ps ax | grep sendmail
 1634 ?        S      0:00 [sendmail]
 1643 ?        S      0:00 [sendmail]
# cat /proc/1634/cmdline
# cat /proc/1643/cmdline

Expected Results:  "ps" and "cat" should show the full command lines, as was 
the case in Red Hat Linux 8 and earlier.

Comment 1 Jordan Russell 2003-04-04 21:23:37 UTC
The problem is also present in Red Hat 8.0's 2.4.18-27.8.0 errata kernel.
The stock 2.4.18-14 kernel included with Red Hat 8.0 does not have this problem.

Comment 2 James Ralston 2003-04-04 22:06:16 UTC
JR, could you please raise the priority of this bug to "High"?

Not only does this bug make it impossible to see status information from
affected processes (e.g., sendmail), but init.d scripts fail because long
program names are truncated:

 1083 ?        S      0:00 [mimedefang-mult]
 1112 ?        S      0:15  \_ /usr/bin/perl -w /usr/bin/mimedefang.pl -server
 1989 ?        S      0:01  \_ /usr/bin/perl -w /usr/bin/mimedefang.pl -server
 1101 ?        S      0:00 [mimedefang]
 1109 ?        S      0:00  \_ [mimedefang]
 1110 ?        S      0:00      \_ [mimedefang]
 1127 ?        S      0:00 [sendmail]
 1136 ?        S      0:00 [sendmail]

(The "[mimedefang-mult]" process is mimedefang-multiplexor.  Because the
/etc/init.d/mimedefang script (understandable) calls "killproc
mimedefang-multiplexor", running "service mimedefang stop" always fails.)

I'm not sure what the common element here is.  (It had looked like only
processes that attempted to modify argv[0] were broken, but a test program I
wrote worked just fine, so that's not the case.)

Comment 3 Jordan Russell 2003-04-04 22:18:08 UTC
Ouch, you're right! Changing the priority...

BTW, I haven't verified it but I *suspect* this bug has something to do with 
the recent fix for the ptrace security hole. See for example:


Comment 4 James Ralston 2003-04-05 00:02:16 UTC
Yeah, that's the problem:


From your link, as of 2003-04-04T19:00-0500, it looks like people are still
arguing on lkml about how to properly fix the problem.

If you see a patch float by that Alan, Arjan, et. al. accept, please attach it
here.  (I'll be watching as well.)

Comment 5 James Ralston 2003-04-05 00:12:11 UTC
Actually, from skimming through the past few weeks of the lkml archive, I think
it's pretty clear that there's going to be another version of the ptrace patch,
as the current one is broken:


Hopefully the /proc/*/cmdline issues can be addressed in the second patch...

Comment 6 Deron Meranda 2003-04-07 05:58:32 UTC
Yes, this was introduced in the ptrace patch.  Both the /proc/pid/cmdline and
/proc/pid/environ pseudo-files (at least) are affected, and return an empty
string, and also indirectly affects the output of ps(1).  This happens whenever
the process is considered non-dumpable, and occurs for daemons (like sendmail)
whenever the process euid != uid.  This is unrelated to a process changing its
argv or environ arrays.

The problem was introduced in the access_process_vm() function in
linux/kernel/ptrace.c, which now returns -EPERM when the process is not
dumpable.  And that function is used by both proc_pid_cmdline() and
proc_pid_environ() functions in the fs/proc/base.c file for the proc filesystem.

I believe this is not platform specific, should this change in this bug?

Comment 7 Deron Meranda 2003-04-07 06:30:50 UTC
A purported patch was just made by Tachino Nobuhiro on the linux.kernel mailing
list.  Looks reasonable to me, here's the thread to follow...


Comment 8 Arjan van de Ven 2003-04-07 08:55:59 UTC
I'm not at all convinced this is safe.
The thing is, euid!=uid programs are "special" security wise and ANY access to
them needs to be very very careful. Including seeing the arguments as user,
since that sort of is privileged already.

Comment 9 Deron Meranda 2003-04-07 17:38:56 UTC
Yes, there is certainly the argument that even read-only access is not valid,
but that would be a change in the way Linux has traditionally behaved (except
perhaps "trusted" linux variants).  But that argument applies to not only setuid
processes, but also to say processes where euid=uid=0; why is a setuid process
more deserving of command-line hiding than a normal root process?  The
kmod-ptrace hole was certainly a very critical bug, but a read-only check seems
to still keep that particular hole closed while making a mimimal change in other
kernel behavior.

Some other Unixes solve this by keeping the globally visible command line in
kernel space, providing a pstat() system call to access/modify it.  But in Linux
you have to actually access the user-space of a process to read it's command
line.  It also seems to me that the /proc fs is not really a ptrace() type
function and probably should have different access rules.  At a minimum though,
if you're root (or have the CAP_SYS_PTRACE capability) you should always be able
to see the command line, and with the current ptrace patch you can not.  This is
a major policy and bahavior change!  Of course it's best left to the kernel
gurus to determine a better fix, but the current patch is breaking software and

Comment 10 James Ralston 2003-04-13 08:54:14 UTC
I agree with Deron in comment #9.

Historically speaking, (regular processes, root processes) have been able to see
the cmdline for (regular, root, euid!=uid) processes.  Over the years, I think
this has been more of an implicit assumption than a conscious decision.

Perhaps, now that this issue has been explicitly examined, a different consensus
might be reached.

Regardless, much software and many users depend on the access to the cmdline
that the ptrace patch took away.  Removing the access is a vast sweeping change,
and it should *not* occur in anything short of the 2.5 kernel, or perhaps even
the 2.7 kernel.  (And even then, I'd argue that the same functionality--a
process being able to make some very small synopsis of its command-line
arguments and/or current state publicly available--should be provided via some
other mechanism.  Linux setproctitle(), anyone?)

Discussion about the ptrace patch seems to have mostly died off on lkml.  The
only patch which came out of the discussion which wasn't obviously wrong was
Tachino Nobuhiro's patch:


It might not be the best solution, but it's *a* solution, and it doesn't seem to
re-open the ptrace vulnerability.

Comment 11 Ivan Martinez 2003-04-29 23:26:47 UTC
Maybe bug  
can be marked as duplicate of this, as this bug has some work for fixing.

Comment 12 Arjan van de Ven 2003-05-14 18:09:45 UTC
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.


Comment 13 James Ralston 2003-05-15 06:56:39 UTC
I tested this out, and everything seems to WORKFORME.

As an added bonus, the root user can now strace (effective id != real id!)
processes.  (That fix wasn't mentioned in the errata, but I know I've seen other
Bugzilla bugs about it.)

Much thanks for fixing this.

Comment 14 Jean-David Beyer 2003-05-15 12:46:10 UTC
This problem is also being addressed in the 87638 thread.

I just installed kernel-smp-2.4.20-13.7 that came out yesterday, and I can
assure everyone that the problem _is not fixed_.

Comment 15 Deron Meranda 2003-05-15 20:07:44 UTC
This is fixed in Red Hat 9, but it is still broken in 8.0 and earlier.  Thus the
errata page RHSA-2003:172-23 describing the new kernel is somewhat misleading. 
In particular the following kernels,

Red Hat 9   - 2.4.20-13.9 - FIXED
Red Hat 8.0 - 2.4.20-13.8 - BROKEN
Red Hat 7.3 - 2.4.20-13.7 - BROKEN

I didn't dive into the source code yet to see what was done, or not done.

Arjan, please reopen this bug for 8.0 (or create a new bug, since technically
this bug was reported against release 9 which is fixed).  Thanks.

Comment 16 Ivan Martinez 2003-05-16 14:06:10 UTC
For me too the problem are solved in 9 and appeared in 7.3
Before upgrade 7.3 was with all cmdlines, and after kernel upgrade many
processes now appear as swapped and they are not swapped.

Comment 17 Rajiv Manglani 2003-05-18 19:59:25 UTC
just updated to kernel 2.4.20-13.7 [RHSA-2003:172-00] on redhat 7.2 and my
machine now has this problem. cat /proc/*/cmdline is empty for child processes. eg:

root      1016     1  0 02:51 ?        00:00:03 /usr/sbin/httpd -DHAVE_ACCESS
nobody    2723  1016  0 04:02 ?        00:00:02  \_ [httpd]
nobody    2724  1016  0 04:02 ?        00:00:02  \_ [httpd]
nobody    2725  1016  0 04:02 ?        00:00:02  \_ [httpd]

this is on a pentium (x86).

Comment 18 Rajiv Manglani 2003-06-04 18:46:32 UTC
just installed kernel-2.4.20-18.7 [RHSA-2003:187-01] and this appears to be
fixed on my redhat 7.2 box.

Note You need to log in before you can comment on or make changes to this bug.