748789 – oprofile does not see samples when using opreport/opannotate

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 748789 - oprofile does not see samples when using opreport/opannotate

Summary: oprofile does not see samples when using opreport/opannotate

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	oprofile
Sub Component:
Version:	6.2
Hardware:	Unspecified
OS:	Unspecified
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	William Cohen
QA Contact:	qe-baseos-tools-bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2011-10-25 11:36 UTC by Petr Muller
Modified:	2016-09-20 02:08 UTC (History)
CC List:	2 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2012-06-20 15:09:58 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2012:0966	0	normal	SHIPPED_LIVE	oprofile bug fix and enhancement update	2012-06-19 21:12:02 UTC

Description Petr Muller 2011-10-25 11:36:41 UTC

Description of problem:
Some of our tests for oprofile regularly fail on certain HW, so I start reporting these so either our tests can be fixed (if they are wrong), or the genuine bug in oprofile is confirmed. 

Some of our tests work try to collect some data in oprofile and then run some commands working with data dumped. This usually works, but sometimes this fails, and it is usually bound to a certain HW or certain box.

The test which fails follows a oprofile tutorial for Fedora 12, on http://people.redhat.com/wcohen/Fedora12OprofileTutorial.txt

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Oprofile configuration
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   PASS   ] :: Running 'opcontrol --init'
:: [07:21:48] ::  x86-64/family10
:: [   PASS   ] :: Running 'opcontrol --setup --no-vmlinux'
:: [   PASS   ] :: Running 'ophelp --get-default-event'
:: [07:21:48] ::  Using CPU_CLK_UNHALTED:100000:0:1:1 as default event
:: [   PASS   ] :: Running 'opcontrol --setup --no-vmlinux --event=CPU_CLK_UNHALTED:100000:0:1:1'
:: [   PASS   ] :: Running 'opcontrol --setup --vmlinux=/usr/lib/debug/lib/modules/2.6.32-211.el6.i686/vmlinux --event=CPU_CLK_UNHALTED:100000:0:1:1'
Using 2.6+ OProfile kernel interface.
Reading module info.
Using log file /var/lib/oprofile/samples/oprofiled.log
Daemon started.
Profiler running.
:: [   PASS   ] :: Running 'opcontrol --start'
:: [   PASS   ] :: Running 'ps aux | grep oprofiled'
:: [   PASS   ] :: Running 'opcontrol --shutdown'

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Collecting and analysing data
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   PASS   ] :: Running 'opcontrol --start'
:: [07:21:56] ::  Sleeping for a while
:: [   PASS   ] :: Running 'opcontrol --dump'
:: [   PASS   ] :: File /var/lib/oprofile/samples/current should exist
:: [   PASS   ] :: Running 'opcontrol --shutdown'
:: [   PASS   ] :: Running 'opcontrol --save=junk'
:: [   PASS   ] :: Running 'opcontrol --start'
:: [07:22:08] ::  Executing the benchmark
<output snipped>
:: [07:22:10] ::  Benchmark finished
:: [   PASS   ] :: Running 'opcontrol --dump'
:: [07:22:11] ::  Sleeping for a while
opreport error: No sample file found: try running opcontrol --dump
or specify a session containing sample files

:: [   FAIL   ] :: Running 'opreport --long-filenames --threshold 1' (Expected 0, got 1)
<same error>
:: [   FAIL   ] :: Running 'opreport --long-filenames --threshold 1 | grep linpack1000d' (Expected 0, got 1)
<same error>
:: [   FAIL   ] :: Running 'opreport -l image:/mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d --threshold .1' (Expected 0, got 1)
<same error>
:: [   FAIL   ] :: Running 'opreport -l image:/mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d --threshold .1 | grep daxpy' (Expected 0, got 1)
<same error>
:: [   FAIL   ] :: Running 'opreport -l image:/mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d --include-symbols=daxpy_ --details' (Expected 0, got 1)
<same error>
:: [   FAIL   ] :: Running 'opannotate --source /mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d -o session_annote' (Expected 0, got 1)
:: [   FAIL   ] :: File session_annote should exist 
:: [   PASS   ] :: Running 'opcontrol --shutdown'

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Saving profiling data
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   PASS   ] :: Running 'opcontrol --start'
:: [07:22:28] ::  Executing the benchmark
<output snipped>
:: [07:22:30] ::  Benchmark finished
:: [   PASS   ] :: Running 'opcontrol --save=linpack'
:: [   PASS   ] :: Running 'opcontrol --dump'
:: [07:22:33] ::  Sleeping for a while
error: no sample files found: profile specification too strict ?
:: [   FAIL   ] :: Running 'opreport session:linpack --long-filenames --threshold 1' (Expected 0, got 1)
error: no sample files found: profile specification too strict ?
:: [   FAIL   ] :: Running 'opreport session:linpack --long-filenames --threshold 1 | grep linpack1000d' (Expected 0, got 1)
error: no sample files found: profile specification too strict ?
:: [   FAIL   ] :: Running 'opreport session:linpack -l image:/mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d --threshold .1' (Expected 0, got 1)
error: no sample files found: profile specification too strict ?
:: [   FAIL   ] :: Running 'opreport session:linpack -l image:/mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d --threshold .1 | grep daxpy' (Expected 0, got 1)
error: no sample files found: profile specification too strict ?
:: [   FAIL   ] :: Running 'opreport session:linpack -l image:/mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d --include-symbols=daxpy_ --details' (Expected 0, got 1)
error: no sample files found: profile specification too strict ?
:: [   FAIL   ] :: Running 'opannotate session:linpack --source /mnt/testarea/tests/oprofile/Sanity/fedora12-tutorial/linpack1000d -o linpack_annote' (Expected 0, got 1)
:: [   FAIL   ] :: File linpack_annote should exist

Version-Release number of selected component (if applicable):
oprofile-0.9.6-21.el6.i686

How reproducible:
always on hp-bl495cg5-01.rhts.bos.redhat.com (4cores, family 16=10h, model 2, arch i686)

Steps to Reproduce:
1. see above
  
Actual results:
opreport and opannotate do not show anything after opcontrol --dump

Expected results:
something useful

Additional info:
This is very environment specific. The identical test PASSed od different box with same CPU family/model running x86_64. I was not able to test if the real factor is the box itself, or x86-64ness.

This is not a regression.

Comment 1 Petr Muller 2011-11-02 16:14:55 UTC

Hm, I think I just seen the similar issue on 8-core ppc :/ Removing the arch specific attributes.

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Environment and preparing samples
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

Unloading oprofile module
:: [   PASS   ] :: Compiling the testcase
:: [   PASS   ] :: Running 'opcontrol --init
:: [20:41:17] ::  Shutting oprofile daemon for the case it is running
Daemon not running
:: [   PASS   ] :: Running 'opcontrol --shutdown'
:: [   PASS   ] :: Running 'opcontrol --no-vmlinux'
Using default event: CYCLES:100000:0:1:1
Daemon started.
Profiler running.
:: [   PASS   ] :: Starting oprofile daemon
Signalling daemon... done
:: [   PASS   ] :: Resetting to abstract away previous tests
:: [20:41:25] ::  Running something for oprofile to annotate 
source_address:  fffe78a4003
dest_address:  fffe78a6007
 1000000000 interations of memcpy(d+7.s+3,65) requires       42.745 seconds
:: [   PASS   ] :: Dumping samples from oprofile

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: Actual testing
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

opannotate error: No sample file found: try running opcontrol --dump
or specify a session containing sample files

:: [   FAIL   ] :: Running opannotate (Expected 0, got 1)

Comment 2 Peng Tao 2011-11-10 05:39:03 UTC

oprofile fails the same way on SMP X86_64. And I am running upstream 3.2-rc1 kernel.

$rpm -q oprofile
oprofile-0.9.6-21.fc15.x86_64

$uname -a
Linux Fedora15 3.2.0-rc1 #21 SMP Wed Nov 9 02:24:59 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

$cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping        : 5
microcode       : 0x11
cpu MHz         : 2660.301
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable aperfmperf pni ssse3 cx16 sse4_1 sse4_2 popcnt hypervisor lahf_lm ida dts
bogomips        : 5320.60
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 26
model name      : Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping        : 5
microcode       : 0x11
cpu MHz         : 2660.301
cache size      : 8192 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 11
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable aperfmperf pni ssse3 cx16 sse4_1 sse4_2 popcnt hypervisor lahf_lm ida dts
bogomips        : 5320.60
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Comment 3 Peng Tao 2011-11-10 06:12:05 UTC

upstream oprofile at
git://oprofile.git.sourceforge.net/gitroot/oprofile/oprofile
works well.

Maybe you should just merge upstream code into rhel/fedora rpms...

Comment 6 errata-xmlrpc 2012-06-20 15:09:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0966.html

Note You need to log in before you can comment on or make changes to this bug.