Bug 88047 - /proc/<pid>/cmdline is empty in RHL8
/proc/<pid>/cmdline is empty in RHL8
Status: CLOSED ERRATA
Product: Red Hat Linux
Classification: Retired
Component: kernel (Show other bugs)
9
athlon Linux
high Severity high
: ---
: ---
Assigned To: Arjan van de Ven
Brian Brock
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2003-04-04 16:08 EST by Jordan Russell
Modified: 2005-10-31 17:00 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2003-06-08 10:51:48 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jordan Russell 2003-04-04 16:08:53 EST
Description of problem:
Inexplicably, with the new 2.4.20 kernel, some processes (usually daemons) have 
an empty (zero-byte) /proc/<pid>/cmdline file. Consequently, "ps ax" will not 
show the processes' command lines.

I read that this can happen if processes are swapped out of memory, so I tried 
disabling swap (swapoff -a) and restarting the processes, but the problem was 
still there.

Version-Release number of selected component (if applicable):
kernel-2.4.20-8.athlon.rpm

How reproducible:
Always

Steps to Reproduce:
# ps ax | grep sendmail
 1634 ?        S      0:00 [sendmail]
 1643 ?        S      0:00 [sendmail]
# cat /proc/1634/cmdline
# cat /proc/1643/cmdline

Expected Results:  "ps" and "cat" should show the full command lines, as was 
the case in Red Hat Linux 8 and earlier.
Comment 1 Jordan Russell 2003-04-04 16:23:37 EST
The problem is also present in Red Hat 8.0's 2.4.18-27.8.0 errata kernel.
The stock 2.4.18-14 kernel included with Red Hat 8.0 does not have this problem.
Comment 2 James Ralston 2003-04-04 17:06:16 EST
JR, could you please raise the priority of this bug to "High"?

Not only does this bug make it impossible to see status information from
affected processes (e.g., sendmail), but init.d scripts fail because long
program names are truncated:

 1083 ?        S      0:00 [mimedefang-mult]
 1112 ?        S      0:15  \_ /usr/bin/perl -w /usr/bin/mimedefang.pl -server
 1989 ?        S      0:01  \_ /usr/bin/perl -w /usr/bin/mimedefang.pl -server
 1101 ?        S      0:00 [mimedefang]
 1109 ?        S      0:00  \_ [mimedefang]
 1110 ?        S      0:00      \_ [mimedefang]
 1127 ?        S      0:00 [sendmail]
 1136 ?        S      0:00 [sendmail]

(The "[mimedefang-mult]" process is mimedefang-multiplexor.  Because the
/etc/init.d/mimedefang script (understandable) calls "killproc
mimedefang-multiplexor", running "service mimedefang stop" always fails.)

I'm not sure what the common element here is.  (It had looked like only
processes that attempted to modify argv[0] were broken, but a test program I
wrote worked just fine, so that's not the case.)
Comment 3 Jordan Russell 2003-04-04 17:18:08 EST
Ouch, you're right! Changing the priority...

BTW, I haven't verified it but I *suspect* this bug has something to do with 
the recent fix for the ptrace security hole. See for example:

http://groups.google.com/groups?selm=20030323104008%24737c%40gated-at.bofh.it
Comment 4 James Ralston 2003-04-04 19:02:16 EST
Yeah, that's the problem:

https://listman.redhat.com/pipermail/shrike-list/2003-April/000694.html

From your link, as of 2003-04-04T19:00-0500, it looks like people are still
arguing on lkml about how to properly fix the problem.

If you see a patch float by that Alan, Arjan, et. al. accept, please attach it
here.  (I'll be watching as well.)
Comment 5 James Ralston 2003-04-04 19:12:11 EST
Actually, from skimming through the past few weeks of the lkml archive, I think
it's pretty clear that there's going to be another version of the ptrace patch,
as the current one is broken:

http://www.ussg.iu.edu/hypermail/linux/kernel/0304.0/0137.html

Hopefully the /proc/*/cmdline issues can be addressed in the second patch...
Comment 6 Deron Meranda 2003-04-07 01:58:32 EDT
Yes, this was introduced in the ptrace patch.  Both the /proc/pid/cmdline and
/proc/pid/environ pseudo-files (at least) are affected, and return an empty
string, and also indirectly affects the output of ps(1).  This happens whenever
the process is considered non-dumpable, and occurs for daemons (like sendmail)
whenever the process euid != uid.  This is unrelated to a process changing its
argv or environ arrays.

The problem was introduced in the access_process_vm() function in
linux/kernel/ptrace.c, which now returns -EPERM when the process is not
dumpable.  And that function is used by both proc_pid_cmdline() and
proc_pid_environ() functions in the fs/proc/base.c file for the proc filesystem.

I believe this is not platform specific, should this change in this bug?
Comment 7 Deron Meranda 2003-04-07 02:30:50 EDT
A purported patch was just made by Tachino Nobuhiro on the linux.kernel mailing
list.  Looks reasonable to me, here's the thread to follow...

http://groups.google.com/groups?selm=20030407030016%245069%40gated-at.bofh.it
Comment 8 Arjan van de Ven 2003-04-07 04:55:59 EDT
I'm not at all convinced this is safe.
The thing is, euid!=uid programs are "special" security wise and ANY access to
them needs to be very very careful. Including seeing the arguments as user,
since that sort of is privileged already.
Comment 9 Deron Meranda 2003-04-07 13:38:56 EDT
Yes, there is certainly the argument that even read-only access is not valid,
but that would be a change in the way Linux has traditionally behaved (except
perhaps "trusted" linux variants).  But that argument applies to not only setuid
processes, but also to say processes where euid=uid=0; why is a setuid process
more deserving of command-line hiding than a normal root process?  The
kmod-ptrace hole was certainly a very critical bug, but a read-only check seems
to still keep that particular hole closed while making a mimimal change in other
kernel behavior.

Some other Unixes solve this by keeping the globally visible command line in
kernel space, providing a pstat() system call to access/modify it.  But in Linux
you have to actually access the user-space of a process to read it's command
line.  It also seems to me that the /proc fs is not really a ptrace() type
function and probably should have different access rules.  At a minimum though,
if you're root (or have the CAP_SYS_PTRACE capability) you should always be able
to see the command line, and with the current ptrace patch you can not.  This is
a major policy and bahavior change!  Of course it's best left to the kernel
gurus to determine a better fix, but the current patch is breaking software and
expectations.
Comment 10 James Ralston 2003-04-13 04:54:14 EDT
I agree with Deron in comment #9.

Historically speaking, (regular processes, root processes) have been able to see
the cmdline for (regular, root, euid!=uid) processes.  Over the years, I think
this has been more of an implicit assumption than a conscious decision.

Perhaps, now that this issue has been explicitly examined, a different consensus
might be reached.

Regardless, much software and many users depend on the access to the cmdline
that the ptrace patch took away.  Removing the access is a vast sweeping change,
and it should *not* occur in anything short of the 2.5 kernel, or perhaps even
the 2.7 kernel.  (And even then, I'd argue that the same functionality--a
process being able to make some very small synopsis of its command-line
arguments and/or current state publicly available--should be provided via some
other mechanism.  Linux setproctitle(), anyone?)

Discussion about the ptrace patch seems to have mostly died off on lkml.  The
only patch which came out of the discussion which wasn't obviously wrong was
Tachino Nobuhiro's patch:

http://www.ussg.iu.edu/hypermail/linux/kernel/0304.0/1394.html

It might not be the best solution, but it's *a* solution, and it doesn't seem to
re-open the ptrace vulnerability.
Comment 11 Ivan Martinez 2003-04-29 19:26:47 EDT
Maybe bug  
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=87638
can be marked as duplicate of this, as this bug has some work for fixing.
Comment 12 Arjan van de Ven 2003-05-14 14:09:45 EDT
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2003-172.html
Comment 13 James Ralston 2003-05-15 02:56:39 EDT
I tested this out, and everything seems to WORKFORME.

As an added bonus, the root user can now strace (effective id != real id!)
processes.  (That fix wasn't mentioned in the errata, but I know I've seen other
Bugzilla bugs about it.)

Much thanks for fixing this.
Comment 14 Jean-David Beyer 2003-05-15 08:46:10 EDT
This problem is also being addressed in the 87638 thread.

I just installed kernel-smp-2.4.20-13.7 that came out yesterday, and I can
assure everyone that the problem _is not fixed_.
Comment 15 Deron Meranda 2003-05-15 16:07:44 EDT
This is fixed in Red Hat 9, but it is still broken in 8.0 and earlier.  Thus the
errata page RHSA-2003:172-23 describing the new kernel is somewhat misleading. 
In particular the following kernels,

Red Hat 9   - 2.4.20-13.9 - FIXED
Red Hat 8.0 - 2.4.20-13.8 - BROKEN
Red Hat 7.3 - 2.4.20-13.7 - BROKEN

I didn't dive into the source code yet to see what was done, or not done.

Arjan, please reopen this bug for 8.0 (or create a new bug, since technically
this bug was reported against release 9 which is fixed).  Thanks.
Comment 16 Ivan Martinez 2003-05-16 10:06:10 EDT
For me too the problem are solved in 9 and appeared in 7.3
Before upgrade 7.3 was with all cmdlines, and after kernel upgrade many
processes now appear as swapped and they are not swapped.
Comment 17 Rajiv Manglani 2003-05-18 15:59:25 EDT
just updated to kernel 2.4.20-13.7 [RHSA-2003:172-00] on redhat 7.2 and my
machine now has this problem. cat /proc/*/cmdline is empty for child processes. eg:

root      1016     1  0 02:51 ?        00:00:03 /usr/sbin/httpd -DHAVE_ACCESS
-DHAVE_PERL -DHAVE_PRO
nobody    2723  1016  0 04:02 ?        00:00:02  \_ [httpd]
nobody    2724  1016  0 04:02 ?        00:00:02  \_ [httpd]
nobody    2725  1016  0 04:02 ?        00:00:02  \_ [httpd]

this is on a pentium (x86).
Comment 18 Rajiv Manglani 2003-06-04 14:46:32 EDT
just installed kernel-2.4.20-18.7 [RHSA-2003:187-01] and this appears to be
fixed on my redhat 7.2 box.

Note You need to log in before you can comment on or make changes to this bug.