Bug 1249572 - pcp-iostat exception at the end of an archive
pcp-iostat exception at the end of an archive
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: pcp (Show other bugs)
24
Unspecified Unspecified
low Severity low
: ---
: ---
Assigned To: Nathan Scott
Fedora Extras Quality Assurance
: Reopened
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-03 06:40 EDT by Mark Goodwin
Modified: 2016-04-06 10:55 EDT (History)
6 users (show)

See Also:
Fixed In Version: pcp-3.10.9-1.fc23 pcp-3.10.9-1.fc22 pcp-3.11.1-1.el5
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-06 10:55:17 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Mark Goodwin 2015-08-03 06:40:06 EDT
Description of problem: pcp-iostat exception at end of archive

Version-Release number of selected component (if applicable): 3.10.6-1

How reproducible: easily

Steps to Reproduce:
1. systemctl enable pmlogger; systemctl start pmlogger
2. sleep 600  (to wait for some data to be written to archive)
3. pcp -a /var/log/pcp/pmlogger/<hostname>/<archive> -t 10s iostat
   (for some <hostname> and <archive>

Actual results:

python exception (KeyError) on last fetch at the end of the archive :

Traceback (most recent call last):
  File "/usr/libexec/pcp/bin/pcp-iostat", line 202, in <module>
    sts = manager.run()
  File "/usr/lib64/python3.3/site-packages/pcp/pmcc.py", line 623, in run
    self._printer.report(self)
  File "/usr/libexec/pcp/bin/pcp-iostat", line 116, in report
    rrqm = (c_rrqm[inst] - p_rrqm[inst]) / dt
KeyError: 'sda'


Expected results: no exception at end of archive

Additional info: will be a trivial fix, but felt I should report it.
Here's an example :

[root]# pcp -a /var/log/pcp/pmlogger/fletch/20150803.20.10 -t 10 iostat

# Device      rrqm/s  wrqm/s    r/s    w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await %util
...
sda              0.1     2.8    0.1    5.6      1.0     35.8     6.46     0.06    10.9    61.0    10.0   5.5
sdb              0.0     0.0    0.0    0.0      0.0      0.0     0.00     0.00     0.0     0.0     0.0   0.0
...
Traceback (most recent call last):
  File "/usr/libexec/pcp/bin/pcp-iostat", line 202, in <module>
    sts = manager.run()
  File "/usr/lib64/python3.3/site-packages/pcp/pmcc.py", line 623, in run
    self._printer.report(self)
  File "/usr/libexec/pcp/bin/pcp-iostat", line 116, in report
    rrqm = (c_rrqm[inst] - p_rrqm[inst]) / dt
KeyError: 'sda'
Comment 1 Mark Goodwin 2015-09-05 21:27:53 EDT
testing a fix
Comment 2 Nathan Scott 2015-11-04 18:05:56 EST
Mark, can you confirm if this problem still exists?  Working fine for me, so I'm wondering if some other python API / pcp-iostat fix has resolved this one also.

thanks.
Comment 3 Nathan Scott 2015-11-24 22:32:24 EST
(In reply to Nathan Scott from comment #2)
> Mark, can you confirm if this problem still exists?  Working fine for me, so
> I'm wondering if some other python API / pcp-iostat fix has resolved this
> one also.

I think I understand the problem here now - I suspect there's a bit more to it than your initial recipe, Mark, which is why I didn't hit this before.  It looks like this issue would be triggered if a disk (instance) was missing/added between samples ... but not if the set of disks is in steady state for the duration of the archive recording (hence my earlier comment, I still haven't reproduced it so far).

I have a potential fix, but need a reproducer archive now - do you still have that original archive handy, Mark?
Comment 4 Mark Goodwin 2015-11-24 23:13:12 EST
hmm, that archive was from August 3 on a system that has since been reloaded. Sorry ... 

I just ran some experiments on the current archive on a difference machine, whilst adding then removing a disk randomly over a few minutes. pmiostat doesn't fail any more .. and the indom being reported at the end of the archive seems to reflect reality (i.e. that disk is either present or not). So I don't know how to reproduce it now.

I'd be happy to RV your potential fix though.

Cheers
Comment 5 Nathan Scott 2015-11-25 00:14:15 EST
Thanks Mark (and for the RV already) - will test a bit more here & merge soon.
Comment 6 Fedora Update System 2015-12-16 20:54:19 EST
pcp-3.10.9-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d08245c076
Comment 7 Fedora Update System 2015-12-16 20:55:09 EST
pcp-3.10.9-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-2b40815137
Comment 8 Fedora Update System 2015-12-16 21:17:57 EST
pcp-3.10.9-1.el5 has been submitted as an update to Fedora EPEL 5. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-2ac90519bc
Comment 9 Fedora Update System 2015-12-17 05:26:24 EST
pcp-3.10.9-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-2b40815137
Comment 10 Fedora Update System 2015-12-17 05:26:43 EST
pcp-3.10.9-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d08245c076
Comment 11 Fedora Update System 2015-12-17 07:48:48 EST
pcp-3.10.9-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'yum --enablerepo=epel-testing update pcp'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-2ac90519bc
Comment 12 Fedora Update System 2016-01-05 16:57:41 EST
pcp-3.10.9-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 13 Fedora Update System 2016-01-05 17:54:16 EST
pcp-3.10.9-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.
Comment 14 Fedora Update System 2016-02-02 22:56:33 EST
pcp-3.11.0-1.el5 has been submitted as an update to Fedora EPEL 5. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-57b7efb2d7
Comment 15 Fedora Update System 2016-02-03 17:17:10 EST
pcp-3.11.0-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-57b7efb2d7
Comment 16 Jan Kurik 2016-02-24 10:54:31 EST
This bug appears to have been reported against 'rawhide' during the Fedora 24 development cycle.
Changing version to '24'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora24#Rawhide_Rebase
Comment 17 Fedora Update System 2016-03-21 20:17:32 EDT
pcp-3.11.1-1.el5 has been pushed to the Fedora EPEL 5 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2016-5b519318e0
Comment 18 Fedora Update System 2016-04-06 10:53:46 EDT
pcp-3.11.1-1.el5 has been pushed to the Fedora EPEL 5 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.