Bug 467122 - [utrace] restore original semantics to PTRACE_O_TRACEVFORK
[utrace] restore original semantics to PTRACE_O_TRACEVFORK
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Roland McGrath
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2008-10-15 16:25 EDT by Tom Horsley
Modified: 2008-10-17 15:18 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2008-10-17 15:18:10 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Tom Horsley 2008-10-15 16:25:42 EDT
Description of problem:

Bug 464520 describes a problem with the ptrace setoptions behavior on vfork.
That bug is fixed in latest 2.6.27 kernels, but it reveals a different problem.
The new behavior when ptraceing a vfork()ed child now retains the vfork()
shared page semantics (semantics which POSIX says no one should expect :-).

This is a non backward compatible change, the previous behavior of
ptraceing across a vfork resulted in the vfork being treated as a fork,
with the parent and child getting separate copies of memory pages.

In a multi-process debugger (such as the one I work on all the time), this
presents fantastic bookkeeping problems. Out of the blue, modifying
one process will (behind the debugger's back) modify a different process
at the same time.

I concede the desirability of being able to ptrace across a vfork without
changing the semantics of a non-ptraced vfork, but there are advantages the
other way as well, and since it has always worked the other way, I maintain
it should continue to work the other way (i.e. as through it were a fork,
not a vfork).

If you want to support the new semantics, it seem like the proper way to
do it would be by adding something like PTRACE_O_VFORKSHARESPAGES so
people needing the new semantics can explicitly request it rather than
having it unexpectedly thrust upon them.

Version-Release number of selected component (if applicable):

How reproducible:
every time

Steps to Reproduce:
1. I can produce a test program if anyone really needs one.
Actual results:
shared memory between parent and child across a ptrace

Expected results:
original behavior of vfork being treated as fork

Additional info:
Comment 1 Roland McGrath 2008-10-16 20:52:22 EDT
The description is inaccurate AFAIK.  Please produce a test program that demonstrates on some past vanilla kernel where PTRACE_O_TRACEVFORK changed the meaning of vfork.
Comment 2 Tom Horsley 2008-10-17 10:00:44 EDT
Yikes! Sorry about the noise. I did write a test program, but it
always says memory is indeed shared on all the fedora and rhel kernels
I've tried. I've definitely seen the non-shared behavior, but I
guess that was only with some custom kernels, so turning vfork
into fork must be something we did either accidentally or deliberately.
I guess I need to poke around some more and clear up my confusion
about which kernels this happens on.
Comment 3 Roland McGrath 2008-10-17 15:18:10 EDT
The strace utility does tweak vfork syscalls to be fork syscalls.
Maybe that's what you were thinking of.

Note You need to log in before you can comment on or make changes to this bug.