Bug 604732
Summary: | "trace-cmd report" fails to generate useful output | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | William Cohen <wcohen> | ||||||
Component: | trace-cmd | Assignee: | John Kacur <jkacur> | ||||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | Red Hat Kernel QE team <kernel-qe> | ||||||
Severity: | medium | Docs Contact: | |||||||
Priority: | low | ||||||||
Version: | 6.0 | CC: | srostedt | ||||||
Target Milestone: | rc | ||||||||
Target Release: | --- | ||||||||
Hardware: | All | ||||||||
OS: | Linux | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | trace-cmd-1.0.2-6.el6 | Doc Type: | Bug Fix | ||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2011-08-12 20:00:58 UTC | Type: | --- | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Attachments: |
|
Description
William Cohen
2010-06-16 15:48:20 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux major release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Major release. This request is not yet committed for inclusion. The error messages are normal, the problem is if you don't have any useful information. Can you repeat the test after doing echo 1 > /debug/tracing/tracing_enabled it is possible that certain programs like latencytop are resetting that to 0 if this solves the problem, we'll see if we can set this to 1 from trace_cmd thanks There appears to be data in the trace.dat, # ls -l trace.dat -rw-r--r--. 1 root root 6311936 Jun 16 17:35 trace.dat However "trace-cmd report trace.dat" doesn't give any output for the trace.dat file. Using the upstream git version provides all sorts of output. repeated the experiment preceeded with: echo 1 > /sys/kernel/debug/tracing/tracing_enabled Get the same results. I don't think there are any other programs running on the system using the tracing subsystem. William, Can you attach a bzip version of the trace.dat file. Basically, what you are saying is that the stable release of trace-cmd produces no output from the file, where as the mainline version does? Created attachment 424832 [details]
compressed trace.dat file that demonstrates problem
This is an example trace.dat file generated on rhel-6 machine. The kernel used to generate this was:
$ uname -a
Linux cannondale 2.6.32-33.el6.x86_64 #1 SMP Thu Jun 3 13:00:03 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
There is plenty of output with git version of trace-cmd, but no output with the rhel-6 trace-cmd-1.0.1-5.el6.x86_64.
Created attachment 424889 [details]
patch to not crash on FORMAT TOO BIG error from kernel
The trace.dat file seems to be a trace that is not from the RHEL6 kernel. It includes tracepoints that were added for 2.6.35.
These tracepoints are from the SCSI system that made the output format bigger than a page. The current code in the kernel failed to output anything when this happened and simply had the format file be "FORMAT TOO BIG".
Trace-cmd could not handle this input and would simply crash. The given patch fixes trace-cmd to prevent it from crashing. Also, these events are simply ignored now.
I already fixed the code in the kernel so these events will now be printed properly. But it was a rewrite of the output code and was too big to try to push it past Linus's strictness in this release. Thus, trace-cmd will not be able to read these events in the 2.6.35 kernel. Neither will perf.
These events are fine with the debugfs/tracing/trace output, because the format file is not needed for the in-kernel pretty printing.
The attached patch is already in stable release trace-cmd-v1.0.3.
Tried trace-cmd-1.0.2-6.el6.x86_64.rpm. The problem appears to be resolved in this most recent build. This issue has been proposed when we are only considering blocker issues in the current Red Hat Enterprise Linux release. It has been denied for the current Red Hat Enterprise Linux release. ** If you would still like this issue considered for the current release, ask your support representative to file as a blocker on your behalf. Otherwise ask that it be considered for the next Red Hat Enterprise Linux release. ** |