Bug 467651 - opreport doesn't work
opreport doesn't work
Status: CLOSED NEXTRELEASE
Product: Fedora
Classification: Fedora
Component: oprofile (Show other bugs)
10
All Linux
medium Severity medium
: ---
: ---
Assigned To: William Cohen
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 489732
  Show dependency treegraph
 
Reported: 2008-10-19 21:25 EDT by Matthias Clasen
Modified: 2009-04-17 14:03 EDT (History)
15 users (show)

See Also:
Fixed In Version: 0.9.4-6.fc10
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-04-17 11:08:00 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Matthias Clasen 2008-10-19 21:25:40 EDT
I was trying to get Lennart some profile information from pulseaudio, so I followed the instructions, only to have 

opreport -l /usr/bin/pulseaudio 

tell me

bfd_get_section_contents:get_debug:: Bad value

Without producing any useful data. I then tried to install an unstripped pulseaudio binary to avoid problems with the separate debuginfo, and then
opreport just segfaulted.
Comment 1 Will Woods 2008-10-23 09:59:38 EDT
Same results here. Also, as an added bonus, oprofiled segfaulted:

kernel: oprofiled[23354]: segfault at b70e104c ip 08051ac2 sp bfcfd2b0 error 4 in oprofiled[8047000+14000]
Comment 2 Søren Sandmann Pedersen 2008-10-25 15:42:42 EDT
Sysprof also has problems with libbfd.

See bug 468495.
Comment 3 William Cohen 2008-10-27 17:46:46 EDT
Able to reproduce the problem on a kvm image. Need to force oprofile to use the timer interrupt to get some samples on the machine with the following:

modprobe oprofile timer=1

Then can collect samples on the machine with:

opcontrol --start

Let run for a while, then:

opcontrol --shutdown

The following command give same message about "bfd get section...":

opreport -l /lib/libc-2.8.90.so
Comment 4 William Cohen 2008-10-28 14:30:16 EDT
Something has changed between oprofile 0.9.3 and 0.9.4. With same data only thing different oprofile versions.

[wcohen@f10-i386 Download]$ opreport -t 5 -l /lib/libc-2.8.90.so
CPU: CPU with timer interrupt, speed 0 MHz (estimated)
Profiling through timer interrupt
samples  %        symbol name
3342     38.2205  memcpy
2420     27.6761  re_search_internal
[wcohen@f10-i386 Download]$ rpm -q oprofile
oprofile-0.9.3-18.fc10.i386

[wcohen@f10-i386 Download]$ opreport -t 5 -l /lib/libc-2.8.90.so
CPU: CPU with timer interrupt, speed 0 MHz (estimated)
Profiling through timer interrupt
bfd_get_section_contents:get_debug:: Bad value
[wcohen@f10-i386 Download]$ rpm -q oprofile
oprofile-0.9.4-3.fc10.i386
Comment 5 Bug Zapper 2008-11-25 23:01:15 EST
This bug appears to have been reported against 'rawhide' during the Fedora 10 development cycle.
Changing version to '10'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 6 Karl Pickett 2009-01-14 11:52:48 EST
I have this problem as well.  Opreport doesn't work at all with -l.  According to strace it's crashing while reading the binary.

When no symbols in the binary: (although I have a debuginfo rpm installed that doesn't seem to be being consulted)
[root@webfilter bin]# opreport -l khttpd 
CPU: P4 / Xeon with 2 hyper-threads, speed 2392.28 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
bfd_get_section_contents:get_debug:: Bad value

And now when symbols are in the binary:
[root@wrjo8e1i bin]# opreport -l proxy
CPU: P4 / Xeon, speed 3191.99 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
Segmentation fault

strace:
...
open("/usr/local/flanker/bin/proxy", O_RDONLY) = 4
fstat64(4, {st_mode=S_IFREG|0755, st_size=1280233, ...}) = 0
fcntl64(4, F_GETFL)                     = 0 (flags O_RDONLY)
fcntl64(4, F_GETFL)                     = 0 (flags O_RDONLY)
fstat64(4, {st_mode=S_IFREG|0755, st_size=1280233, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb807d000
_llseek(4, 0, [0], SEEK_CUR)            = 0
read(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\2\0\3\0\1\0\0\0\200\253\4\0104\0\0\0h"..., 4096) = 4096
_llseek(4, 1228800, [1228800], SEEK_SET) = 0
read(4, "\20\7\10\214\20\7\10.\23\7\10\200\23\7\10\242\20\7\10\205\21\7\10\0\0\0\0\0\0\0\0\242\20"..., 4096) = 4096
_llseek(4, 1232896, [1232896], SEEK_SET) = 0
_llseek(4, 1232896, [1232896], SEEK_SET) = 0
_llseek(4, 0, [0], SEEK_SET)            = 0
read(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\2\0\3\0\1\0\0\0\200\253\4\0104\0\0\0h"..., 4096) = 4096
_llseek(4, 1228800, [1228800], SEEK_SET) = 0
read(4, "\20\7\10\214\20\7\10.\23\7\10\200\23\7\10\242\20\7\10\205\21\7\10\0\0\0\0\0\0\0\0\242\20"..., 4096) = 4096
_llseek(4, 0, [0], SEEK_SET)            = 0
read(4, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\2\0\3\0\1\0\0\0\200\253\4\0104\0\0\0h"..., 4096) = 4096
_llseek(4, 1228800, [1228800], SEEK_SET) = 0
read(4, "\20\7\10\214\20\7\10.\23\7\10\200\23\7\10\242\20\7\10\205\21\7\10\0\0\0\0\0\0\0\0\242\20"..., 4096) = 4096
read(4, "\7\0\0\0\1\0\17\0{\1\0\0%\27\7\10\22\0\0\0\1\0\17\0J\1\0\0007\27\7\10\v"..., 12288) = 12288
read(4, "'\2\0\0\22\0\r\0\247b\0\0\260\5\7\10\312\0\0\0\"\0\r\0\346b\0\0\0\0\0\0\0"..., 4096) = 4096
brk(0x8bed000)                          = 0x8bed000
read(4, "3BlockIteratorS0_iE12__FUNCTION__"..., 28672) = 28672
read(4, "0\0_ZNK13BlockIterator17trailing_d"..., 4096) = 2281
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++


oprofile-0.9.4-3.fc10.i386
binutils-2.18.50.0.9-7.fc10.i386
Comment 7 Karl Pickett 2009-01-15 12:57:28 EST
What's strange is rebuilding the src rpm manually (not installing all dependencies, just binutils and popt devel) creates an opreport that doesn't crash.

I did rpmbuild -bp --nodeps oprofile.spec, then ./configure --with-linux=/usr/src/kernels/2.6.27.9-159.ns2.fc10.i686/ --prefix=/usr && make
Comment 8 William Cohen 2009-01-20 09:58:30 EST
(In reply to comment #7)
> What's strange is rebuilding the src rpm manually (not installing all
> dependencies, just binutils and popt devel) creates an opreport that doesn't
> crash.
> 
> I did rpmbuild -bp --nodeps oprofile.spec, then ./configure
> --with-linux=/usr/src/kernels/2.6.27.9-159.ns2.fc10.i686/ --prefix=/usr && make

You verified that just building it with all the buildrequires installed generated a opreport that didn't work? If that is the case, then there is something related to the configuration. Here are all the build requires in oprofile.spec:

BuildRequires: qt3-devel
BuildRequires: libxslt
BuildRequires: docbook-style-xsl
BuildRequires: docbook-utils
BuildRequires: elinks
BuildRequires: gtk2-devel
BuildRequires: automake
BuildRequires: libtool
BuildRequires: binutils-devel
BuildRequires: popt-devel
BuildRequires: java-devel
BuildRequires: jpackage-utils
BuildRequires: java-1.6.0-openjdk-devel

Which of these are these buildrequires are not installed on the machine for the successful version? Should be able to manually disable features and find out which one is causing the problem.
Comment 9 William Cohen 2009-01-20 12:43:08 EST
I noticed that I am not seeing the problem on a "yum update"d f10 or rawhide machine. Could you check whether the regular oprofile rpm works in that environment? If oprofile does work on an "yum update" machine, that would point to an issue with one of the shared libraries.
Comment 10 Pierre Ossman 2009-01-30 11:42:47 EST
I'm seeing this bug here on a fully updated machine:

Starting program: /usr/bin/opreport -d
warning: [vdso] (tgid:14255 range:0x110000-0x111000) could not be found.
CPU: P4 / Xeon with 2 hyper-threads, speed 2792.94 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000

Program received signal SIGSEGV, Segmentation fault.
0x080c99f8 in interesting_symbol (sym=0x81ec288) at bfd_support.cpp:343
343		if (!(sym->section->flags & SEC_CODE))
(gdb) bt
#0  0x080c99f8 in interesting_symbol (sym=0x81ec288) at bfd_support.cpp:343
#1  0x080c4bad in op_bfd::get_symbols (this=0xbffff3d4, symbols=@0xbffff368) at op_bfd.cpp:217
#2  0x080c5c58 in op_bfd (this=0xbffff3d4, fname=@0x81d0ee0, symbol_filter=@0x81bce64, extra_images=@0xbffff540, ok=@0xbffff4c6) at op_bfd.cpp:171
#3  0x0809400a in populate_for_image (samples=@0xbffff51c, ip=@0x81d0ee0, symbol_filter=@0x81bce64, has_debug_info=0x0) at populate.cpp:70
#4  0x08055c16 in opreport (spec=@0xbffff644) at opreport.cpp:576
#5  0x08063611 in run_pp_tool (argc=2, argv=0xbffff734, fct=0x8055673 <opreport>) at common_option.cpp:210
#6  0x08053d18 in main (argc=2, argv=0xbffff734) at opreport.cpp:590

This seems vaguely similar:

http://sourceforge.net/tracker/index.php?func=detail&aid=1921984&group_id=16191&atid=116191
Comment 11 Arkadi Shishlov 2009-02-21 05:33:40 EST
I can confirm that after rebuilding OProfile as described in Karl Pickett comment #c7 opreport works. But there is another problem - no callgraphs. -c2 is passed to opcontrol, but still every function is reported being caller if itself. KGDB can backtrace the kernel, so frame pointers should be there.
The system is current Rawhide.
Comment 12 Jochen Roth 2009-03-11 11:54:17 EDT
I can't confirm that it works with F11 see bug #489732 
The error message (bfd_get_section_contents:get_debug:: Bad value
) disappears but now opreport segfaults.
Comment 13 Clemens Eisserer 2009-03-13 16:46:00 EDT
I don't get a segfault with oprofile-0.9.4-3.fc10.i386 but the following message:
bfd_get_section_contents:get_debug:: Bad value

I am running rawhide with latest updates.
Comment 14 Jochen Roth 2009-03-16 08:08:21 EDT
Please try to follow the steps suggested in comment #7
Comment 15 Eduardo Habkost 2009-04-05 04:09:30 EDT
I was getting the "bfd_get_section_contents:get_debug:: Bad value" error with oprofile-0.9.4-3.fc10.i386. I've rebuilt oprofile as a scratch build on koji (http://koji.fedoraproject.org/koji/taskinfo?taskID=1277734) and the problem was solved.

Just rebuilding oprofile on dist-f10 should solve the issue, but investigating if there is a broken-ABI issue on some library would be interesting.
Comment 16 William Cohen 2009-04-09 12:20:04 EDT
This problem appears to be caused by a problem in a header file, <bfd.h>, that was available when the package was previously built. Oprofile has been rebuilt with a newer version of binutils-devel that has the corrected file. This seems to only affect i386 machines. Could someone with a f10 on a i386 check the problem is resolved with the new oprofile build available from:

http://koji.fedoraproject.org/koji/buildinfo?buildID=97193
Comment 17 Fedora Update System 2009-04-09 14:44:23 EDT
oprofile-0.9.4-6.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/oprofile-0.9.4-6.fc10
Comment 18 Fedora Update System 2009-04-13 15:39:26 EDT
oprofile-0.9.4-6.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update oprofile'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-3558
Comment 19 Karl Pickett 2009-04-13 16:10:25 EDT
The build in comment 16 works on my test box.  Yay!
Comment 20 Fedora Update System 2009-04-17 14:03:32 EDT
oprofile-0.9.4-6.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.