Bug 1180513

Summary: Unable to profile jit'd JVM code when using static huge pages
Product: Red Hat Enterprise Linux 6 Reporter: John O'Hara <johara>
Component: oprofileAssignee: William Cohen <wcohen>
Status: CLOSED ERRATA QA Contact: Michael Petlan <mpetlan>
Severity: high Docs Contact: Dayle Parker <dayleparker>
Priority: urgent    
Version: 6.7CC: anmiller, mbenitez, mcermak, mpetlan, rbiba, tlavigne
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: oprofile-0.9.9-7.el6 Doc Type: Bug Fix
Doc Text:
OProfile memory mapping improved for huge pages using Java JIT code Previously, when profiling performance of Java just-in-time (JIT) compiled code with static huge pages enabled, OProfile's operf command recorded a large number of event samples to anonymous memory (in anon_hugepage) instead of to the appropriate Java method. Operf now recognizes the static huge page identifiers and correctly maps samples to Java methods when using statically allocated huge pages.
 Story Points: ---
Clone Of:
: 1214077 (view as bug list) Environment:
Last Closed: 2015-07-22 07:05:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1214077    
Attachments:
Description Flags
recognize the perf mmap anon_hugepage information none

Description John O'Hara 2015-01-09 10:38:11 UTC 
Description of problem:

When oprofile is used to profile JIT'd code where the java process has been started with huge page memory enabled for the process a large percentage of time is reported in anon_hugepage. Oprofile does not report to see JIT'd method calls in the perf report.

If I do not start the process with hugespages, we get the correct reporting of jvm method calls. 


How reproducible:


Steps to Reproduce:
1. Using the DaCapo benchmark: http://www.dacapobench.org/
2. profile jvm using: 'operf java -XX:+UseLargePages -agentlib:jvmti_oprofile -jar dacapo-9.12-bach.jar sunflow'
3. view captured data: 'opreport --symbols --threshold 1.0'

Actual results:

samples  %        image name               symbol name
582252   84.4164  anon_hugepage (deleted)  /anon_hugepage (deleted)
68864     9.9841  libjvm.so                /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71-2.5.3.1.el7_0.x86_64/jre/lib/amd64/server/libjvm.so
23543     3.4133  no-vmlinux               /no-vmlinux
11087     1.6074  libjava.so               /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71-2.5.3.1.el7_0.x86_64/jre/lib/amd64/libjava.so

Expected results:

samples  %        image name               symbol name
157817   21.8364  2838.jo                  void org.sunflow.core.accel.KDTree.intersect(org.sunflow.core.Ray, org.sunflow.core.IntersectionState)
147963   20.4729  2838.jo                  Interpreter
131928   18.2542  2838.jo                  void org.sunflow.core.accel.BoundingIntervalHierarchy.intersect(org.sunflow.core.Ray, org.sunflow.core.IntersectionState)~1
104371   14.4413  2838.jo                  void org.sunflow.core.primitive.TriangleMesh$WaldTriangle.intersect(org.sunflow.core.Ray, int, org.sunflow.core.IntersectionState)
79441    10.9919  libjvm.so                /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71-2.5.3.1.el7_0.x86_64/jre/lib/amd64/server/libjvm.so
37384     5.1726  2838.jo                  org.sunflow.image.Color org.sunflow.core.gi.InstantGI.getIrradiance(org.sunflow.core.ShadingState, org.sunflow.image.Color)~1
23981     3.3181  no-vmlinux               /no-vmlinux
11256     1.5574  libjava.so               /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.71-2.5.3.1.el7_0.x86_64/jre/lib/amd64/libjava.so
9761      1.3506  2838.jo                  void org.sunflow.core.light.TriangleMeshLight$TriangleLight.getSamples(org.sunflow.core.ShadingState)
Comment 3 William Cohen 2015-01-09 22:17:26 UTC 
Created attachment 978413 [details]
recognize the perf mmap anon_hugepage information

Something like the attached patch should recognize the slightly different information for the anon_hugepage mmap information from perf.  However, this has not been tested on rhel6.
Comment 9 Michael Petlan 2015-03-06 17:47:23 UTC 
Successfully reproduced with oprofile-0.9.9-6.el6.
When tested with oprofile-0.9.9-7.el6 no issue occured.

The value 128 has been used for nr_hugepages.
Comment 14 errata-xmlrpc 2015-07-22 07:05:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1367.html