Bug 509151

Summary: [RHEL5.4 Xen]: oprofile no longer works
Product: Red Hat Enterprise Linux 5 Reporter: Chris Lalancette <clalance>
Component: oprofileAssignee: William Cohen <wcohen>
Status: CLOSED NOTABUG QA Contact: BaseOS QE <qe-baseos-auto>
Severity: medium Docs Contact:
Priority: low    
Version: 5.4CC: benl, pmuller, xen-maint
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-07-10 07:37:58 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chris Lalancette 2009-07-01 15:03:21 UTC
Description of problem:
In RHEL-5.3, running the Xen kernel kernel-xen-2.6.18-128.el5 and the oprofile package oprofile-0.9.3-18.el5.x86_64.rpm, I am successfully able to use xenoprof to collect profile data from the dom0 and hypervisor using this script:

#!/bin/bash

opcontrol --reset
opcontrol --start-daemon --xen=/usr/lib/debug/boot/xen-syms-2.6.18-128.el5.debug --vmlinux=/usr/lib/debug/lib/modules/2.6.18-128.el5xen/vmlinux
opcontrol --start
sleep 10
opcontrol --stop
opcontrol --shutdown

opreport -l

However, when running RHEL-5.4 with kernel-xen-2.6.18-156.el5 and oprofile-0.9.4-10.el5, I can no longer get profile data, and the last step (opreport -l) reports:

opreport error: No sample file found: try running opcontrol --dump
or specify a session containing sample files

Also, running "opcontrol -l" on RHEL-5.3 shows the CPU_CLK_UNHALTED event available, while "opcontrol -l" on RHEL-5.4 doesn't show that event available anymore.  If I downgrade *just* the oprofile package to the 5.3 version, reboot (which seems to be necessary, for some reason), and run the test again, it runs successfully.

Comment 3 William Cohen 2009-07-03 17:38:10 UTC
It seems odd that the machine needed to be rebooted to get things working again. I tried to replicate this problem on 2.6.18-156.el5xen kernel and was unable to replicate it.

# uname -a; rpm -q xen oprofile
Linux calfee.devel.redhat.com 2.6.18-156.el5xen #1 SMP Mon Jun 29 18:24:43 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
xen-3.0.3-80.el5
oprofile-0.9.4-10.el5

I corrected a the script with line continuations, print out a bit less info (opreport -t1) and proper --xen and --vmlinux:

#!/bin/bash

opcontrol --reset
opcontrol --start-daemon \
--xen=/usr/lib/debug/boot/xen-syms-2.6.18-156.el5.debug \
--vmlinux=/usr/lib/debug/lib/modules/`uname -r`/vmlinux
opcontrol --start
sleep 10
opcontrol --stop
opcontrol --shutdown

opreport -l -t1


I definitely obtained data with oprofile-0.9.4-10.

# opcontrol --version
opcontrol: oprofile 0.9.4 compiled on Jun 21 2009 23:09:40
# opreport -l -t1

....

CPU: P4 / Xeon with 2 hyper-threads, speed 2793.1 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
samples  %        image name               app name                 symbol name
1403     15.3652  vmlinux-unknown          vmlinux-unknown          /vmlinux-unknown
327       3.5812  bash                     bash                     /bin/bash
245       2.6832  xen-syms-2.6.18-152.el5.debug xen-syms-2.6.18-152.el5.debug toggle_guest_mode
239       2.6175  ext3                     ext3                     /ext3

Comment 4 Chris Lalancette 2009-07-09 09:35:51 UTC
I just tried it out on two different machines here, and it worked just fine.  Unfortunately the original machine I was having a problem with is in RHTS, so I'll have to go back on that machine and re-reserve it.  I'll put this back to waiting on myself until I can get back on the machine and see if I can reproduce.

Thanks for taking a look,
Chris Lalancette

Comment 7 Chris Lalancette 2009-07-10 07:37:58 UTC
OK.  This must have been user cockpit error, since this now works for me on the original problem machine with kernel-xen 2.6.18-157.el5 and oprofile 0.9.4-10.  Sorry for the noise, I'll close this as NOTABUG.

Chris Lalancette

Comment 8 Chris Lalancette 2010-07-19 13:24:54 UTC
Clearing out bogus flags for reporting purposes.

Chris Lalancette