Bug 665817 - Headers present only for first VM but missing for all other VMs in the output of "virt-top --csv"
Headers present only for first VM but missing for all other VMs in the output...
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: virt-top (Show other bugs)
6.0
Unspecified Unspecified
medium Severity medium
: rc
: ---
Assigned To: Richard W.M. Jones
Virtualization Bugs
:
: 682155 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2010-12-27 04:21 EST by Mark Wu
Modified: 2011-12-06 12:12 EST (History)
7 users (show)

See Also:
Fixed In Version: virt-top-1.0.4-3.10.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-12-06 12:12:19 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
processcsv.py (1.56 KB, text/plain)
2011-01-06 09:25 EST, Richard W.M. Jones
no flags Details

  None (edit)
Comment 2 Mark Wu 2010-12-27 04:25:18 EST
The customer think the missed headers for other domains are important for them to parse the output file.
Comment 4 Richard W.M. Jones 2011-01-06 09:24:41 EST
This is documented, unfortunately only as a comment in the code
and not in the man page :-(

The headers are currently printed for one domain only, eg:

Hostname Time Arch ... DomainID DomainName
|                     ||                  |
 global fields -------  per-domain fields

If there are more (or less) than 1 domain, the extra domains'
fields don't have headers, but they are the same headers as
the first domain.

So far so good - just what the customer observed.

Now, why do we do this?  Why not repeat the per-domain headers
for each domain?  It is because domains can appear and disappear
while virt-top is running.  If a new domainappears, then its
fields would not have headers, unless we printed out the header
line again - but if we did that it would make parsing the CSV
file much harder because you'd have to detect the new header line
and do something with it.

I think they might be able to use csvtool (part of RHEL 6.0)
to extract each domain separately.

For example, to get only the first domain (with headers):

 csvtool col 19-26 - < /tmp/csv > /tmp/csv.domain1

Similarly the second domain is columns 27-34, and in general
the nth domain (counting from n=0) is:

 csvtool col $((19+$n*8))-$((26+$n*8)) - < /tmp/csv > /tmp/csv.domain$n

The attached Python script works along the same lines, splitting
the CSV file into a "global.csv" (global data) and several
"domain<NN>.csv" files for each domain, with proper column headers
for every column.
Comment 5 Richard W.M. Jones 2011-01-06 09:25:52 EST
Created attachment 472065 [details]
processcsv.py

Python script to post-process virt-top CSV output, splitting
it into a global.csv file and several domain<NN>.csv files,
with full headers on every column.
Comment 6 Mark Wu 2011-01-06 20:30:50 EST
Richard,
     Many thanks for the detailed explanation and post-process script. BTW, do you have a plan to add the script into the package virt-top?

     Mark.
Comment 8 Mark Wu 2011-01-09 20:11:11 EST
We're waiting for customer's feedback.
Comment 9 Richard W.M. Jones 2011-03-04 05:34:54 EST
*** Bug 682155 has been marked as a duplicate of this bug. ***
Comment 15 Richard W.M. Jones 2011-08-11 09:44:10 EDT
What I've done here is to add processcsv.py as
a regular binary.  I've also added a note to the
man page discussing how to use it.

http://brewweb.devel.redhat.com/brew/taskinfo?taskID=3548952
Comment 17 Huming Jiang 2011-08-12 01:52:47 EDT
According to comment 15
Verified with the virt-top-1.0.4-3.10.el6.x86_64

Steps:
1 #man virt-top
...
           RHEL 6 provides a short Python script called "processcsv.py" which
           can be used to post-process the CSV output.  Run it like this:

            virt-top --csv data.csv
            processcsv.py < data.csv

           This creates or overwrites the following files in the current
           directory:

            global.csv
            domain<NNN>.csv

           "global.csv" will contain the global data.  One "domain<NNN>.csv"
           file will also be created for each domain with ID "NNN", containing
           the per-domain data.
...
So move the status to Verified.
Comment 18 Huming Jiang 2011-08-12 02:45:36 EDT
According to comment 17,do the following tests to check whether the processcsv.py has taken effect or not:

Steps:
1. #virt-top --csv guestinfo.csv
2. #processcsv.py < guestinfo.csv
3. #ls
domain1.csv                 
domain2.csv                      
domain3.csv                         
global.csv                           
guestinfo.csv
...

4.#cat domain1.csv
Hostname,Time,Domain ID,Domain name,CPU (ns),%CPU,Mem (bytes),%Mem,Block RDRQ,Block WRRQ,Net RXBY,Net TXBY
jhm,14:32:44,1,rh6-copy-clone,0.,0.,0,0,,,,
jhm,14:32:47,1,rh6-copy-clone,0.,0.,729088,9,0,0,104,0
...

5.#cat domain2.csv
Hostname,Time,Domain ID,Domain name,CPU (ns),%CPU,Mem (bytes),%Mem,Block RDRQ,Block WRRQ,Net RXBY,Net TXBY
jhm,14:32:44,2,rh6-copy,0.,0.,0,0,,,,
jhm,14:32:47,2,rh6-copy,10000000.,0.0823686582203,6244352,79,0,0,104,0
...
6.# cat domain3.csv 
Hostname,Time,Domain ID,Domain name,CPU (ns),%CPU,Mem (bytes),%Mem,Block RDRQ,Block WRRQ,Net RXBY,Net TXBY
jhm,14:34:03,3,test,0.,0.,0,0,,,,
jhm,14:34:06,3,test,10000000.,0.082726838736,1048576,13,0,0,122,0
...

7.# cat global.csv 
Hostname,Time,Arch,Physical CPUs,Count,Running,Blocked,Paused,Shutdown,Shutoff,Crashed,Active,Inactive,%CPU,Total hardware memory (KB),Total memory (KB),Total guest memory (KB),Total CPU time (ns)
jhm,14:32:44,x86_64,4,3,2,0,0,0,0,0,2,1,0.0,7892640,6973440,6973440,0
jhm,14:32:47,x86_64,4,3,2,0,0,0,0,0,2,1,0.1,7892640,6973440,6973440,10000000
...


So this bug has been fixed.
Comment 19 errata-xmlrpc 2011-12-06 12:12:19 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1692.html

Note You need to log in before you can comment on or make changes to this bug.