Created attachment 614951 [details] some random sar file sar2pcp (3.6.5-1.el6) gives many warnings during operation: Use of uninitialized value in subroutine entry at /usr/bin/sar2pcp line 171. .... but results in an otherwise serviceable pcp archive. (Note bug #859102 may require one to use a working version of sysstat (sadf) to get this far.)
Created attachment 614952 [details] sadf -x from same
On RHEL5 (with pcp-3.6.5-1.el5) similar (and more) errors occur, and _no_ archive is created. [djuran@rhel5 ~]$ sar2pcp /var/log/sa/sa20 /tmp/pcp/foo Warning: XML tag <processes> not handled Warning: XML tag <context-switch> not handled Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 223. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 224. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 225. Argument "\x{31}\x{31}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 226. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 227. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 228. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 228. Argument "\x{38}\x{35}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 229. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 253. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 254. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 255. Argument "\x{32}\x{36}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 256. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 257. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Argument "\x{37}\x{30}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 259. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 253. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 254. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 255. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 256. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 257. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Argument "\x{39}\x{39}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 259. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 253. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 254. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 255. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 256. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 257. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Argument "\x{39}\x{39}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 259. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 253. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 254. Use of uninitialized value in division (/) at /usr/bin/sar2pcp line 255. Argument "\x{31}\x{37}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 256. Argument "\x{30}\x{2c}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 257. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Use of uninitialized value in addition (+) at /usr/bin/sar2pcp line 258. Argument "\x{37}\x{31}..." isn't numeric in division (/) at /usr/bin/sar2pcp line 259. pmiPutValue: Failed @ 20-Sep-2012 14:35:01: Impossible value or scale conversion attaching output of sadf -x
Created attachment 614953 [details] output of sadf -x on RHEL5
Have a fix for the RHEL6 test case (from fche). Creating a generic little test case so we can keep these problematic sar archives and ensure we don't regress again down the track. pcp/qa/370 already exists, and goes to great lengths to check correctness of a localhost sar-create/pcp-import cycle, this new test will be more aimed at verifying we can do conversions for as many different RHEL/Fedora/other sysstat versions as possible. Will look into David's failing case next...
Hi David, Could you attach your failing /var/log/sa/sa20 file here? (i.e. the binary file too please) thanks!
Oh, ignore that last request David - found it over in 859102.
Created attachment 615301 [details] RHEL5 binary sa file
Nathan, terribly sorry for the confusion, I really should open separate Bz for each issue. The binary sa file in bug 859102 is from Fedora 16, attached here is now the binary file from RHEL5 that caused the error-messages in #2
Thanks! I think I understand the failure on the 859102 binary anyway, it has some interesting characteristics that invalidate an assumption that the sar2pcp script (in particular, it has a network interface that appears mid-way through the log that was not there at the start). Got a fair way through fixing it, but have run out of time for today; will keep going early next week. Thanks for reporting the issues, it's been interesting for me to learn all about how this script works. cheers.
Hi David (and Frank), I have sar2pcp working cleanly now for each of these problem scenarios. I've committed all the fixes into the pcp dev git tree now, which can be found at: git://oss.sgi.com/pcp/pcp David, would you be comfortable cloning that git tree and doing a local build to test it with your own data, on whichever Fedora/RHEL you are using? This would give me additional confidence before doing the next PCP release. Alternatively, I could do a release later this and push into Fedora for further testing if thats easier? (I'd prefer the former if possible, so I don't have to do additional fixup releases later). RPMs can be generated from the git tree using the "./Makepkgs" script if that helps at all. FWIW, the particular problem listed earlier in this bug with RHEL5 sar data was resolved by this commit: commit 11f0e2a371fb05940209b36423bb4731e2f9ea82 Author: Nathan Scott <nathans> Date: Sat Sep 22 21:51:08 2012 +1000 Fix another problem in sar2pcp handling of ancient sar archives Really old versions of sysstat used commas in place of decimal points in the XML floating point numbers. This causes failure whenever perl attempts to convert that string to a float. So, we need to fix up the number (give it a decimal point instead of a comma) before doing any arithmetic or attempting to store the number. This is the cause (if you should ever observe it) of this type of perl warning/failure message: (e.g.) 'Argument "\x{31}\x{2c}..." isn't numeric in addition (+)' Red Hat Bugzilla: 859117 There are a number of other fixes in there too, however, so its worth getting the complete set (via git clone/pull) before testing anything. cheers.
In response to #10, actually I don't think it's the really old version of sadf that is causing the comma signs but rather that I had LC_NUMERIC set to sv_SE.utf8.... Nevertheless, the pacth referred to in #10 solves the problem, thanks for the help
Ah yes, that would do it too - I didn't think of something like that. Good to hear its working anyway.
Something still seems fishy though, running pmval on the converted sar data doesn't give any sensible results, e.g. kernel.all.load is reported to be constantly 1.000. And if I understand this metric correctly, it should be the same as "sar -q" Attaching the results of my conversion, the source file is the one from https://bugzilla.redhat.com/attachment.cgi?id=615301 and everything has been run on RHEL5.
Created attachment 616547 [details] converted data
Thanks - yes, found the bug, recent regression I've introduced with the other changes. I've just committed a fix: Changes committed to git://oss.sgi.com/pcp/pcp.git dev src/pmimport/sar2pcp/sar2pcp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) commit 66a116299f59f14d5da29e81487fe3770a1e86d8 Author: Nathan Scott <nathans> Date: Tue Sep 25 08:37:57 2012 +1000 Fix sar2pcp dovalue() routine - returning incorrect values Using git is going to be quickest, but if it suits you more I'll take Franks suggestion on to patch the Fedora spec (just lemme know). Also, just for reference, another way to check this stuff is to use: "pmdumplog -a <pcplog> | less" ... this is a raw dump of the contents of the PCP archive. It differs to using the higher level tools like pmval in that it doesn't do interpolation or rate-conversion on any counter metrics, etc. It's a handy way to see the original logging frequency too (pmval will do 1sec reporting, or -t <interval>, which is independent to the native logged frequency. But thats not directly relevent to this issue, just for reference. :) Thanks for testing it out! cheers.
Nathan, I've confirmed your fix 66a116299f59f14d5da29e81487fe3770a1e86d8 to doValue() fixes the issue I reported on IRC yesterday, thanks!
Thanks, looks much better (-:
pcp-3.6.9-1.el5 has been submitted as an update for Fedora EPEL 5. https://admin.fedoraproject.org/updates/pcp-3.6.9-1.el5
pcp-3.6.9-1.fc18 has been submitted as an update for Fedora 18. https://admin.fedoraproject.org/updates/pcp-3.6.9-1.fc18
pcp-3.6.9-1.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/pcp-3.6.9-1.fc16
pcp-3.6.9-1.el6 has been submitted as an update for Fedora EPEL 6. https://admin.fedoraproject.org/updates/pcp-3.6.9-1.el6
pcp-3.6.9-1.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/pcp-3.6.9-1.fc17
Package pcp-3.6.9-1.el5: * should fix your issue, * was pushed to the Fedora EPEL 5 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=epel-testing pcp-3.6.9-1.el5' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-EPEL-2012-13283/pcp-3.6.9-1.el5 then log in and leave karma (feedback).
pcp-3.6.9-1.el6 has been pushed to the Fedora EPEL 6 stable repository. If problems still persist, please make note of it in this bug report.