Bug 463225 - [LTC 6.0 FEAT] 200966:Per-thread debugging utility
[LTC 6.0 FEAT] 200966:Per-thread debugging utility
Status: CLOSED DEFERRED
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: systemtap (Show other bugs)
6.1
All All
medium Severity medium
: alpha
: 6.1
Assigned To: Frank Ch. Eigler
Petr Muller
: FutureFeature
Depends On:
Blocks: 538808 580566
  Show dependency treegraph
 
Reported: 2008-09-22 12:50 EDT by IBM Bug Proxy
Modified: 2016-09-19 22:04 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-02-11 17:32:04 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description IBM Bug Proxy 2008-09-22 12:50:54 EDT
=Comment: #0=================================================
Emily J. Ratliff <emilyr@us.ibm.com> - 2008-09-16 18:24 EDT
1. Feature Overview:
Feature Id:	[200966]
a. Name of Feature:	Per-thread debugging utility
b. Feature Description
The endeavor is to provide a tool that enables users to track the activities of a single user
thread. These activities include (but are not limited to) system call patterns, signals
sent/received, threads spawned, etc.

2. Feature Details:
Sponsor:	Linux Systems Tech
Architectures:
x86
x86_64
ppc64

Arch Specificity: Purely Common Code
Affects Core Kernel: Yes
Affects Kernel Modules: Yes
Delivery Mechanism: 
Category:	RAS
Request Type:	Package - Feature from IBM
d. Upstream Acceptance:	In Progress
Sponsor Priority	2
f. Severity: Medium
IBM Confidential:	no
Code Contribution:	IBM code
g. Component Version Target:	Upstream Kernel 2.6.27

3. Business Case
As DB/2 on Linux moves to adopting the threaded model for their next release, the DB/2 development
and service engineers want a tool that'd enable them to trace just at a thread level. They'd not
want other threads to pay a penalty when its certain to them what thread they'd  want to target
their debugging for. ptrace, strace, and other tools fall  short when it comes to debugging in
multithreaded scenarios. We need something new here, something we could easily build on the utrace
infrastructure.

Benefits
Value for SUSE customers:  SUSE customers who've been using tools such as ptrace, strace, etc.,
would find this tool increadibly useful for their debugging purposes. SUSE customers who run IBM SWG
products such as DB/2 will potentially have faster turnaround times for their support issues
involving complex  multithreaded applications.

4. Primary contact at Red Hat: 
John Jarvis
jjarvis@redhat.com

5. Primary contacts at Partner:
Project Management Contact:
Mounir Bsaibes, bsaibes@us.ibm.com, 512-838-1301

Technical contact(s):
Maneesh Soni, smaneesh@in.ibm.com
Ananth N Mavinakayanahall, mananth@in.ibm.com

IBM Manager:
Naren A Devaiah, naren.devaiah@in.ibm.com
Comment 1 Frank Ch. Eigler 2008-09-22 13:09:05 EDT
Can you specify how the target subset of threads may be identified by
the instrumentation?
Comment 2 IBM Bug Proxy 2008-09-25 08:30:36 EDT
(In reply to comment #4)
> ------- Comment From fche@redhat.com 2008-09-22 13:09:05 EDT-------
> Can you specify how the target subset of threads may be identified by
> the instrumentation?

The user specifies the thread-id of interest.
Since utrace provides a per thread engine with callbacks to syscalls, signals,
there would be minimal penalty on the other threads.
What is targeted is a strace like utility (or a systemtap script) that builds on
utrace implementation to list systemcalls, signals.
Comment 3 Frank Ch. Eigler 2008-09-25 09:28:38 EDT
OK, it sounds like all you want is that the syntax

  probe process(PID).* { }

allow PIDs to match kernel task->pid (thread-id) not just task->tgid (process-id).
It would not be hard to have some syntax for that:

  probe thread(TID).{begin,end,syscall,...} { }

though uprobes style probes like .function/.statement are per-process,
so would have to be emulated per-thread if they were at all desirable.
Comment 4 IBM Bug Proxy 2008-09-25 10:40:59 EDT
(In reply to comment #6)
> ------- Comment From fche@redhat.com 2008-09-25 09:28:38 EDT-------
> OK, it sounds like all you want is that the syntax
>
> probe process(PID).* { }
>
> allow PIDs to match kernel task->pid (thread-id) not just task->tgid (process-id).
> It would not be hard to have some syntax for that:
>
> probe thread(TID).{begin,end,syscall,...} { }
>
> though uprobes style probes like .function/.statement are per-process,
> so would have to be emulated per-thread if they were at all desirable.

yes the current requirement is for utrace style probes.
uprobes style probes are not a requirement but surely desirable to probe allow
function entry/exit
(more like per-thread-ltrace) but I feel statement probes would make it more heavy.
Comment 5 Frank Ch. Eigler 2009-09-21 14:19:24 EDT
No patch for this feature yet, design also incomplete.
Comment 6 IBM Bug Proxy 2009-10-01 04:20:53 EDT
------- Comment From srikar@linux.vnet.ibm.com 2009-10-01 04:10 EDT-------
(In reply to comment #9)
> No patch for this feature yet, design also incomplete.
>

Some of the features requested by this bug are already implemented in utrace.
Here is the list of requests
1. Request for a thread to be quiesced without having to use SIGNALS.

(We already have support for this in utrace. However there is no easy way for current user space debuggers to use this feature. This could be addresssed if and when utrace has its own user space API)

2. Ability to trace just one thread of a multithreaded process for activity so that there is no penalty for other threads.
(We already have support for this in utrace/uprobes. Though SystemTap is able to exploit utrace/uprobes, It might not be complete with respect to no penalty on other threads. Also there could be user space debuggers who are  interested in exploiting this feature. This could also be addresssed if and when utrace has its own user space API).

As noted earlier, the request is for system call tracing per thread for process. However providing library function tracing per thread of a process would be great.

3. Reliably distinguish between different reasons for SIGTRAP.
(We already have support for this in utrace thro report_signal handler())

Reliable hardware-breakpoints was also part of the thread level debugging requests. However I think its been tracked by a different bug and also there is already considerable work done in this area, This is up for discussion in LKML by K Prasad thro this patches.
Comment 7 Frank Ch. Eigler 2010-01-22 16:22:00 EST
We have been unable to build this in time for rhel6.0.

http://sourceware.org/bugzilla/show_bug.cgi?id=11212 now tracks this.
Comment 8 John Jarvis 2010-01-25 09:13:10 EST
Removing from RHEL 6.0 tracker, adding to 6.1 tracker.
Comment 9 Frank Ch. Eigler 2011-01-28 17:27:00 EST
Unfortunately, the systemtap PR11212 aspect has slipped through the cracks;
perhaps next version.

If there is still as sense that a non-systemtap solution to the above
issues is a priority, we should raise that in other longer-term fora.
How much interest is there at IBM to help define interfaces and build tools?
Comment 10 John Jarvis 2011-02-11 17:32:04 EST
Closing as DEFERRED since this is not in plan for RHEL 6.1.  Please feel free to reopen when the 6.2 feature queue is opened.

Note You need to log in before you can comment on or make changes to this bug.