Bug 250104 - RHEL5 Kernel patches for blktap statistics
RHEL5 Kernel patches for blktap statistics
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen (Show other bugs)
5.1
All Linux
low Severity low
: ---
: ---
Assigned To: Chris Lalancette
Gurhan Ozen
:
Depends On:
Blocks: 306931 444642 448753 RHEL5u3_relnotes
  Show dependency treegraph
 
Reported: 2007-07-30 10:50 EDT by Chris Lalancette
Modified: 2013-11-03 20:33 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
The blktap (blocktap) userspace toolkit has been updated, providing the functionality to monitor the transfer statistics of blktap backed virtualized guests.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-01-20 15:21:36 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Patch 1 for blktap statistics (11.41 KB, patch)
2008-02-04 14:59 EST, Chris Lalancette
no flags Details | Diff
Patch 2 for blktap statistics (1.75 KB, patch)
2008-02-04 15:00 EST, Chris Lalancette
no flags Details | Diff
Patch 3 for blktap statistics (3.37 KB, patch)
2008-02-04 15:00 EST, Chris Lalancette
no flags Details | Diff
Patch 4 for blktap statistics (7.45 KB, patch)
2008-02-04 15:01 EST, Chris Lalancette
no flags Details | Diff
Patch 5 for blktap statistics (1.07 KB, patch)
2008-02-04 15:01 EST, Chris Lalancette
no flags Details | Diff
Patch 6 for blktap statistics (1.06 KB, patch)
2008-02-04 15:01 EST, Chris Lalancette
no flags Details | Diff
Patch 7 for blktap statistics (1.65 KB, patch)
2008-02-04 15:02 EST, Chris Lalancette
no flags Details | Diff
Patch 8 for blktap statistics (1011 bytes, patch)
2008-02-04 15:02 EST, Chris Lalancette
no flags Details | Diff
Patch 9 for blktap statistics (1.33 KB, patch)
2008-02-04 15:02 EST, Chris Lalancette
no flags Details | Diff
Patch 1 for blktap statistics v2 (3.16 KB, patch)
2008-07-23 05:54 EDT, Chris Lalancette
no flags Details | Diff
Patch 2 for blktap statistics v2 (7.34 KB, patch)
2008-07-23 05:55 EDT, Chris Lalancette
no flags Details | Diff
Patch 3 for blktap statistics v2 (1.09 KB, patch)
2008-07-23 05:56 EDT, Chris Lalancette
no flags Details | Diff
Patch 4 for blktap statistics v2 (1.07 KB, patch)
2008-07-23 05:56 EDT, Chris Lalancette
no flags Details | Diff
Patch 5 for blktap statistics v2 (1.65 KB, patch)
2008-07-23 05:56 EDT, Chris Lalancette
no flags Details | Diff
Patch 6 for blktap statistics v2 (649 bytes, patch)
2008-07-23 05:57 EDT, Chris Lalancette
no flags Details | Diff
Patch 7 for blktap statistics v2 (1.56 KB, patch)
2008-07-23 05:57 EDT, Chris Lalancette
no flags Details | Diff

  None (edit)
Description Chris Lalancette 2007-07-30 10:50:00 EDT
Description of problem:
Currently blktap doesn't export any statistics about reads, writes, oob, etc. 
This makes monitoring of file-backed guests difficult, since you can't really
tell what is going on with the blktap module.  Upstream has added some patches
to enhance blktap to be like the blkback driver and export this information via
sysfs:

http://lists.xensource.com/archives/html/xen-changelog/2007-02/msg00278.html

RHEL-5 should really have this patch so customers can monitor guests more
effectively.
Comment 1 Richard W.M. Jones 2007-07-31 12:02:17 EDT
What tools should I use to see these stats?  vmstat? dstat? ...  (Beyond
just cat-ting sysfs files I mean).
Comment 2 Chris Lalancette 2007-07-31 12:11:26 EDT
Richard,
     Currently, cat'ting the files in /sys is the only way.  I started on a
program called "virtstat" that would actually collect various statistics (block,
network, cpu, memory) on the dom0 about the running domains by looking at these
/sys files for you; however, it is very Xen specific.  I have a feeling this is
this sort of thing we want to include in libvirt; something like
"getBlockStats(domain)" or something.  Additionally, there is no easy way to get
stats out of qemu (either for network or block), so work is needed there.  The
whole thing needs more thought, obviously.

Chris Lalancette
Comment 3 Chris Lalancette 2007-10-08 13:25:57 EDT
I believe we will want xen-3.1-testing changesets 14040, 15078, 15389, 15390,
15402 to fix this.

Chris Lalancette
Comment 4 RHEL Product and Program Management 2008-02-01 17:41:22 EST
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".
Comment 5 Richard W.M. Jones 2008-02-04 08:41:56 EST
Just a note that it's possible to test this using the 'virsh' command
(assuming you have libvirt 0.3.3, see bug 306931).

# virsh dumpxml 1 | grep 'target'
      <target dev='vif1.0'/>
      <target dev='hda'/>

# virsh domifstat 1 vif1.0
vif1.0 rx_bytes 0
vif1.0 rx_packets 0
vif1.0 rx_errs 0
vif1.0 rx_drop 103
vif1.0 tx_bytes 0
vif1.0 tx_packets 0
vif1.0 tx_errs 0
vif1.0 tx_drop 0

# virsh domblkstat 1 hda

Network stats should work always.

Block device (disk) stats are more problematic.  They are most likely
to work with a paravirt, partition-backed domain.  They probably *don't*
work for the following:
(a) Fullvirt domains without PV drivers.
(b) File-backed domains (because of lack of Xen support).
(c) QEMU (because of lack of libvirt support).
Comment 6 Chris Lalancette 2008-02-04 14:58:45 EST
In addition to the above mentioned c/s, we need 12333, 12334, 13325, and 14044.
 So, the full list is:

12333
12334
13325
14040
14044
15078
15389
15390
15402

With all of the above changesets in place, I was able to generate a decent
patchset for the RHEL-5 kernel.  I will attach the various patches that apply
cleanly to a 2.6.18-77.el5 kernel.  Note that this has only been compile tested
so far on i686; it will obviously need compile and run-time testing on all
arches before we can put it in.

Chris Lalancette
Comment 7 Chris Lalancette 2008-02-04 14:59:59 EST
Created attachment 293926 [details]
Patch 1 for blktap statistics
Comment 8 Chris Lalancette 2008-02-04 15:00:32 EST
Created attachment 293927 [details]
Patch 2 for blktap statistics
Comment 9 Chris Lalancette 2008-02-04 15:00:53 EST
Created attachment 293928 [details]
Patch 3 for blktap statistics
Comment 10 Chris Lalancette 2008-02-04 15:01:13 EST
Created attachment 293929 [details]
Patch 4 for blktap statistics
Comment 11 Chris Lalancette 2008-02-04 15:01:32 EST
Created attachment 293930 [details]
Patch 5 for blktap statistics
Comment 12 Chris Lalancette 2008-02-04 15:01:54 EST
Created attachment 293931 [details]
Patch 6 for blktap statistics
Comment 13 Chris Lalancette 2008-02-04 15:02:17 EST
Created attachment 293932 [details]
Patch 7 for blktap statistics
Comment 14 Chris Lalancette 2008-02-04 15:02:36 EST
Created attachment 293933 [details]
Patch 8 for blktap statistics
Comment 15 Chris Lalancette 2008-02-04 15:02:57 EST
Created attachment 293934 [details]
Patch 9 for blktap statistics
Comment 16 RHEL Product and Program Management 2008-06-09 18:01:23 EDT
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 18 Chris Lalancette 2008-07-23 05:54:09 EDT
The first 2 patches in this series to add barrier requests turned out to be
controversial.  Because they weren't actually integral to the patchset, I've
removed these two and re-diffed the patches.  I'll attach the 7 patches in this
series here.

Chris Lalancette
Comment 19 Chris Lalancette 2008-07-23 05:54:48 EDT
Created attachment 312441 [details]
Patch 1 for blktap statistics v2
Comment 20 Chris Lalancette 2008-07-23 05:55:27 EDT
Created attachment 312442 [details]
Patch 2 for blktap statistics v2
Comment 21 Chris Lalancette 2008-07-23 05:56:02 EDT
Created attachment 312443 [details]
Patch 3 for blktap statistics v2
Comment 22 Chris Lalancette 2008-07-23 05:56:26 EDT
Created attachment 312444 [details]
Patch 4 for blktap statistics v2
Comment 23 Chris Lalancette 2008-07-23 05:56:52 EDT
Created attachment 312445 [details]
Patch 5 for blktap statistics v2
Comment 24 Chris Lalancette 2008-07-23 05:57:15 EDT
Created attachment 312446 [details]
Patch 6 for blktap statistics v2
Comment 25 Chris Lalancette 2008-07-23 05:57:39 EDT
Created attachment 312447 [details]
Patch 7 for blktap statistics v2
Comment 26 Don Zickus 2008-07-25 16:16:56 EDT
in kernel-2.6.18-100.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5
Comment 28 Ryan Lerch 2008-11-06 01:57:08 EST
This bug has been marked for inclusion in the Red Hat Enterprise Linux 5.3
Release Notes.

To aid in the development of relevant and accurate release notes, please fill
out the "Release Notes" field above with the following 4 pieces of information:


Cause:   What actions or circumstances cause this bug to present.

Consequence:  What happens when the bug presents.

Fix:   What was done to fix the bug.

Result:  What now happens when the actions or circumstances above occur. (NB:
this is not the same as 'the bug doesn't present anymore')
Comment 30 Chris Lalancette 2008-11-06 08:40:50 EST
Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
In previous versions of RHEL-5, it was not possible to monitor the transfer statistics of blktap backed Xen guests.  These are guests that use files as their backend, represented by tap:aio in the /etc/xen configuration file.

This set of patches introduces blktap statistics into RHEL-5.  With these patches in place, you can correctly monitor the transfer statistics of blktap backed guests.
Comment 32 Ryan Lerch 2008-11-09 22:16:22 EST
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1,3 +1 @@
-In previous versions of RHEL-5, it was not possible to monitor the transfer statistics of blktap backed Xen guests.  These are guests that use files as their backend, represented by tap:aio in the /etc/xen configuration file.
+The blktap (blocktap) userspace toolkit has been updated providing the functionality to monitor the transfer statistics of blktap (block tap) backed virtualized guests.-
-This set of patches introduces blktap statistics into RHEL-5.  With these patches in place, you can correctly monitor the transfer statistics of blktap backed guests.
Comment 33 Ryan Lerch 2008-11-09 22:16:52 EST
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1 @@
-The blktap (blocktap) userspace toolkit has been updated providing the functionality to monitor the transfer statistics of blktap (block tap) backed virtualized guests.+The blktap (blocktap) userspace toolkit has been updated providing the functionality to monitor the transfer statistics of blktap backed virtualized guests.
Comment 34 Ryan Lerch 2008-11-09 22:17:33 EST
Release note updated. If any revisions are required, please set the 
"requires_release_notes"  flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

Diffed Contents:
@@ -1 +1 @@
-The blktap (blocktap) userspace toolkit has been updated providing the functionality to monitor the transfer statistics of blktap backed virtualized guests.+The blktap (blocktap) userspace toolkit has been updated, providing the functionality to monitor the transfer statistics of blktap backed virtualized guests.
Comment 38 errata-xmlrpc 2009-01-20 15:21:36 EST
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-0225.html
Comment 39 Chris Lalancette 2010-07-19 09:04:27 EDT
Just clearing a needinfo request on me.

Chris Lalancette

Note You need to log in before you can comment on or make changes to this bug.