Bug 773050

Created attachment 551913 [details]
32bit log

Created attachment 551914 [details]
64bit test result

Created attachment 551915 [details]
[PATCH] Check return value of PTRACE_PEEKTEXT for ppc64

Thanks.  I was the person promising to apply the patch, but it appears that ltrace on ppc is still rather broken.  The patch makes the situation less bad, but we still frequently miss return from a function, and sometimes fail to trace any functions at all.  I'm working on fixing it all first.

------- Comment From emachado.ibm.com 2012-02-03 14:34 EDT-------
(In reply to comment #18)
> Thanks.  I was the person promising to apply the patch, but it appears that
> ltrace on ppc is still rather broken.  The patch makes the situation less bad,
> but we still frequently miss return from a function, and sometimes fail to
> trace any functions at all.  I'm working on fixing it all first.

Hi Petr,
any news regarding this issue against ppc? Please let me know if there is something I can do in order to help.
Thanks for your support.

I had a chance to look into this today and yesterday.  The reason your patch was necessary was that we were a bit impatient about doing ptraces over a process that wasn't ready yet.  There's now a safety latch on process start.  There are also several changes in the early process start that will hopefully make the whole scheme easier to understand and extend in future.  This fixes the first problem.  That's on pmachata/startup upstream branch, I'll port it back to RHEL6 when it's all done.

I'll now look into the second problem, viz. missing function returns, that is still present.

Ok, pmachata/startup upstream branch now mostly works, sans a couple failures in parameters.exp that probably have to do with some alignment issues.  I'll skip looking into those for now, as I have some other effort in this area that this would conflict this.

I didn't realize this was filed against Fedora, that reduces the porting effort to close to nil, so I'll be able to include these fixes really soon now.

One additional failing test is vfork on PPC64, that's caused by bug 753130.

I added the patch to F 16, F 17 and rawhide branches.  I don't suppose it's necessary to build this in mainstream Fedora, as the primary arches are not affected by this bug.  I'm going to close this now.  Feel free to reopen if necessary.

------- Comment From emachado.ibm.com 2012-02-08 22:58 EDT-------
Hi Petr,
I noticed your patches were already commited to upstream ltrace today. Thank you very much for working on this.

------- Comment From emachado.com 2012-04-19 20:59 EDT-------
Hi Petr,

I had the chance to rerun the testsuite on Fedora 17 (using ltrace-0.6.0-4.fc17.src.rpm) and still got bad results on ppc64:

# of expected passes            47
# of unexpected failures        50

It seems ltrace is still failing to collect most of the syscalls due to the problem previously mentioned with PTRACE_PEEKTEXT call.

I also checked the results from upstream ltrace and noticed they look much better:

# of expected passes            95
# of unexpected failures        10

Could you please take a look at this?

Thanks for your support.

Actually I spent about the last month fixing tracing on PPC.  The design of PLT tables on PPC doesn't make it easy to actually trace what we need to trace, viz calls to dynamic libraries.  In the past we've been tracing function entry points instead.

That work is on the branch pmachata/libs, because it was rolled into tracing of calls made from DSOs.  I intend to send a note upstream and merge soonish.

As of a couple hours ago, PPC64/PPC64 and PPC32/PPC32 test suite have one failure each, PPC64/PPC32 has one extra failure, and PPC32/PPC32BSS and PPC64/PPC32BSS have additional two failures each.  So there's some work left still, but overall we are in the best shape ever (BSS-style PPC32 PLT in particular has not been supported at all, or maybe had been, but it was lost along the way as we added support for new-style secure PLT).

A taste of tracing xclock on F16 PPC64 (edited to fit):

$ ./ltrace -e* xclock 
[...]
xclock->XCopyArea(0x18315380, 0x260000c, 0x260000b)             = 1
libXt.so.6->XEventsQueued(0x18315380, 1, 0x260000b <unfinished ...>
libX11.so.6->_XEventsQueued(0x18315380, 1, 0x260000b <unfinished ...>
libX11.so.6->xcb_poll_for_event(0x18316810, 1, 0x260000b <unfinished ...>
libxcb.so.1->pthread_mutex_lock(0x18316828, 1, 0x260000b)       = 0
libxcb.so.1->read(4, 0x18316884, 4096)                          = -1
libxcb.so.1->__errno_location(-1, 0x18316884, 4096)             = 0x80440ac7f0
libxcb.so.1->pthread_mutex_unlock(0x18316828, 0x18316884, 4096) = 0
<... xcb_poll_for_event resumed> )                              = 0
libX11.so.6->xcb_connection_has_error(0x18316810, 0, 0)         = 0
<... _XEventsQueued resumed> )                                  = 0
<... XEventsQueued resumed> )                                   = 0
[...]

I intend to update ltrace in F17 time frame with this patch set.  I will consider adding it to F16 as well, the tracing of DSO calls has been long missing.

(In reply to comment #10)
> I had the chance to rerun the testsuite on Fedora 17 (using
> ltrace-0.6.0-4.fc17.src.rpm) and still got bad results on ppc64:

I just noticed this.  That version doesn't have the necessary patches yet.  With recent ltrace, I get:

# of expected passes		90
# of unexpected failures	7

Unfortunately that ltrace version (ltrace-0.6.0-7) has been stuck in Fedora update system because I forgot to request a push.  I did that now.

------- Comment From emachado.com 2012-04-20 01:43 EDT-------
(In reply to comment #28)
> Unfortunately that ltrace version (ltrace-0.6.0-7) has been stuck in Fedora
> update system because I forgot to request a push.  I did that now.

Thanks again for your help, Petr. I'll retest this once the new version becomes available.

Created attachment 582630 [details]
make -k check file for ltrace in 32 bit


------- Comment (attachment only) From sanpatr1.com 2012-05-07 11:42 EDT-------

Created attachment 582631 [details]
make -k check file for ltrace in 64 bit


------- Comment (attachment only) From sanpatr1.com 2012-05-07 11:42 EDT-------

(In reply to comment #14)
(In reply to comment #15)
Those are GDB test runs.

Created attachment 582871 [details]
make -k check file for ltrace in 32 bit


------- Comment (attachment only) From sanpatr1.com 2012-05-08 06:44 EDT-------

Created attachment 582872 [details]
make -k check file for ltrace in 64 bit


------- Comment (attachment only) From sanpatr1.com 2012-05-08 06:44 EDT-------

(In reply to comment #18)
> Created attachment 582872 [details]
> make -k check file for ltrace in 64 bit
> 
> 
> ------- Comment (attachment only) From sanpatr1.com 2012-05-08 06:44
> EDT-------

This is much more broken than it should be.  Please provide configuration details and ltrace version so that I can reproduce and fix the issues.  Thanks.

------- Comment From emachado.com 2012-05-14 11:35 EDT-------
Hi Petr,

sorry for the confusion with the previous comments.

I had the chance to test ltrace-0.6.0-11.fc17 on Power and got the following results:

32-bit:
# of expected passes            94
# of unexpected failures        3

64-bit:
# of expected passes            92

which means the high number of failures was fixed on Fedora 17. Thus, I'm closing this bugzilla for now.

Thanks for your support.