Bug 1186922 - unable to attach ltrace/strace to certain processes with --spc
Summary: unable to attach ltrace/strace to certain processes with --spc
Status: NEW
Alias: None
Product: Red Hat Developer Toolset
Classification: Red Hat
Component: ltrace   
(Show other bugs)
Version: DTS 9.0 RHEL 7
Hardware: Unspecified
OS: Unspecified
low
high
Target Milestone: ---
: 9.0
Assignee: DJ Delorie
QA Contact: Edjunior Barbosa Machado
URL:
Whiteboard:
Keywords:
: 1249835 (view as bug list)
Depends On: 1186918 1316539
Blocks: 1186913 1223489 1249835
TreeView+ depends on / blocked
 
Reported: 2015-01-28 21:21 UTC by Chris Pelland
Modified: 2018-12-06 17:46 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
If the ltrace command is executed inside a Super-Privileged Container (SPC) to trace a process that is running on Red Hat Enterprise Linux Atomic Host, the ltrace command is unable to locate the binary images of the shared libraries that are attached to the process to be traced. As a consequence, ltrace displays a series of error messages, similar to the following example: Can't open /lib64/libwrap.so.0: No such file or directory Couldn't determine base address of /lib64/libwrap.so.0 ltrace: ltrace-elf.c:426: ltelf_destroy: Assertion `(&lte->plt_relocs)->elt_size == sizeof(GElf_Rela)' failed.
Story Points: ---
Clone Of:
: 1249835 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Chris Pelland 2015-01-28 21:21:31 UTC
Filing this bz based on GSS Test Day findings for RHEL Atomic:


    not sure if its a duplicate of the gdb attach failure BZ (https://bugzilla.redhat.com/show_bug.cgi?id=1186918) , they are similar so it may be, but better to let eng determine that

    CONTAINER IMAGE ID: ad763979eaaa [666.8 MB]

    ATOMIC HOST ID: 7.1.115

    ATOMIC RUN COMMAND: atomic run -n test --spc docker-registry.usersys.redhat.com/atomicga/rheltools:latest bash

    Worked when attaching to a command

    Failed when attempting to attach to an existing process:

    bash-4.2# ps aux | grep 4143

    root       4143  0.0  0.1 137272  4856 ?        Ss   10:45   0:00 sshd: root@pts/2

    root      14803  0.0  0.0   9032   668 ?        R+   11:49   0:00 grep 4143

    bash-4.2# ltrace -p 4143

    Can't open /lib64/libwrap.so.0: No such file or directory

    Couldn't determine base address of /lib64/libwrap.so.0

    ltrace: ltrace-elf.c:426: ltelf_destroy: Assertion `(&lte->plt_relocs)->elt_size == sizeof(GElf_Rela)' failed.

    Aborted

Comment 1 Daniel Walsh 2015-02-03 21:45:54 UTC
This is an interesting problem since this library is on the host but mounted under /host/lib64/libwrap.so.0

I wonder if we could fix this by adding something like

export LD_LIBRARY_PATH=/usr/lib64:/host/usr/lib64

Then executing the command?

Comment 3 Daniel Walsh 2015-02-12 19:50:39 UTC
Why would ltrace inside the container not be able to ltrace processes running within the container?  It has all the shared libraries necessary.

The container is not going to know the ID of itself. at lest for now.

Did you try with the LD_LIBRARY_PATH suggested above?

Comment 17 Gary Benson 2015-10-29 10:30:58 UTC
It sounds like ltrace needs updating to flip into the container's mount namespace before opening the files.  If ltrace is single-threaded then this is easy:

Setup:
  struct stat sb;

  ltrace_mnt_ns_fd = open("/proc/%d/ns/mnt" % getpid(), O_RDONLY); // this can persist until ltrace exits
  traced_mnt_ns_fd = open("/proc/%d/ns/mnt" % PID_OF_TRACED_PROCESS, O_RDONLY);  // this can persist until traced process exits

  fstat (ltrace_mnt_ns_fd, &sb)
  ltrace_mnt_ns_id = sb.st_ino

  fstat (traced_mnt_ns_fd, &sb)
  traced_mnt_ns_id = sb.st_ino

  setns_needed = (ltrace_mnt_ns_id != traced_mnt_ns_id)


To open a file:
  if (setns_needed)
    setns (traced_mnt_ns_fd, 0);
  fd = open ("/lib64/libwrap.so.0", ...)
  if (setns_needed)
    setns (ltrace_mnt_ns_fd);

Obviously you have to check for errors.  You don't need to flip for read, write, fstat, or anything else that takes a file descriptor argument.

setns requires some capabilities, which is why it's wrapped: if they're in the same namespace you don't need the setns and thus don't need the capabilities.  If they're in different namespaces then setns will fail with EPERM or some such thing.  (i.e.,

You could just stay in the traced process's namespace but this causes problems if the parent process hasn't completely loaded all its own libraries--it has no access to its own filesystem.  I don't believe there's any way to check this so I think it's safer to dip in and out of the child's mount namespace as necessary.

If ltrace is multithreaded then you will have to spawn a single-threaded helper to handle the namespace flips.  GDB does this, it's all in 
nat/linux-namespaces.c.  It's harder but doable.

If ltrace respects LD_LIBRARY_PATH you *might* be able to work around this issue with LD_LIBRARY_PATH=/proc/$PID_OF_TRACED_PROCESS/root but if anything tries to process symlinks it will break.  There's no way around that that isn't racy and insecure, so this is at best a workaround; the only robust solution is entering the namespace.

Comment 24 James W. Mills 2017-08-15 15:20:35 UTC
*** Bug 1249835 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.