Bug 1661912 - "dstat -d -f -a" crashes
Summary: "dstat -d -f -a" crashes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: pcp
Version: 29
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ---
Assignee: Nathan Scott
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-24 11:23 UTC by Edgar Hoch
Modified: 2019-03-17 19:35 UTC (History)
4 users (show)

Fixed In Version: pcp-4.3.1-2.fc28 pcp-4.3.1-2.fc29
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-17 07:20:23 UTC


Attachments (Terms of Use)

Description Edgar Hoch 2018-12-24 11:23:16 UTC
Description of problem:

Some dstat arguments create a crash with backtrace.

This works as expected:

# dstat -d -f
--dsk/sda-----dsk/sdb-----dsk/sdc--
 read  writ: read  writ: read  writ
   0     0 :2810k    0 :2770k   74M
   0     0 :2707k 1508k:7374k   37M
   0    44k:3311k    0 :  16M   48M

# dstat -d -n -f 
--dsk/sda-----dsk/sdb-----dsk/sdc-- net/enp1s0f--net/virbr0-net/virbr0--net/enp1s0f----net/lo--
 read  writ: read  writ: read  writ| recv  send: recv  send: recv  send: recv  send: recv  send
   0     0 :4098k    0 :9450k 1002k| 581  1076 :   0     0 :   0     0 :   0     0 
   0     0 :4040k    0 :8612k 3192k|2876k   21k:   0     0 :   0     0 :   0     0 
   0     0 :4534k 2763k:  14M 5686k|7278k   91k:   0     0 :   0     0 :   0     0 


This is strange, but may be ok:

# dstat -d -D -f
No matching instances found.



But this crashes:

# dstat -D -f
You did not select any stats, using -cdngy by default.
Traceback (most recent call last):
  File "/usr/bin/dstat", line 1632, in <module>
    dstat.execute()
  File "/usr/bin/dstat", line 1616, in execute
    scheduler.run()
  File "/usr/lib64/python3.7/sched.py", line 151, in run
    action(*argument, **kwargs)
  File "/usr/bin/dstat", line 1625, in perform
    op.perform(update)
  File "/usr/bin/dstat", line 1509, in perform
    line = line + sep + self.gshow(plugin, results)
  File "/usr/bin/dstat", line 1148, in gshow
    metric = op.metrics[plugin.mgroup[0]]
IndexError: list index out of range

# dstat -d -f -a
Traceback (most recent call last):
  File "/usr/bin/dstat", line 1632, in <module>
    dstat.execute()
  File "/usr/bin/dstat", line 1616, in execute
    scheduler.run()
  File "/usr/lib64/python3.7/sched.py", line 151, in run
    action(*argument, **kwargs)
  File "/usr/bin/dstat", line 1625, in perform
    op.perform(update)
  File "/usr/bin/dstat", line 1509, in perform
    line = line + sep + self.gshow(plugin, results)
  File "/usr/bin/dstat", line 1148, in gshow
    metric = op.metrics[plugin.mgroup[0]]
IndexError: list index out of range


Version-Release number of selected component (if applicable):
pcp-4.3.0-1.fc29

How reproducible:
Always

Comment 1 Nathan Scott 2019-01-10 05:43:31 UTC
OK, I see what's happening here.  The problem is I misinterpreted what the semantics of some options that usually take arguments (-D, -C, ...) but without an argument, in the presence of the -f option.  The semantics of the original dstat are quite odd in this situation - what seems to happen is that these columns (disk for -D, cpu for -C etc) are silently removed from the output when no argument is given.

For your test cases:

| This is strange, but may be ok:
| 
| # dstat -d -D -f
| No matching instances found.

What this says is 'show me the disk stats' (-d) and then 'remove all the disk devices from the output' (-D -f).  This is basically handled correctly - the new dstat reports 'No matching instances found.' as you saw, the old dstat reported 'None of the stats you selected are available.' ... basically, this combination of options makes no sense, so these errors are OK.

| dstat -D -f

This equates to 'remove all disk stats from the output' (-D -f).  The old dstat takes that to mean 'no stats requested, and remove all disks too' - so it first adds in default columns (i.e. the -a option list), and then removes the disk column.  :P  This case is not handled correctly in the new dstat.

# dstat -d -f -a

This is effectively the same error but arrived at via slightly different code paths (note -a includes -d).  This invocation seems to tickle a bug in the original dstat too, where it prints out all the disk device columns twice for some reason.  *shrug*

Anyway - thanks for the report, I'll get these fixed up for the next PCP release.

Comment 2 Nathan Scott 2019-01-10 23:59:14 UTC
These issues are resolved in the upstream PCP commit below.  Will be in the pcp-4.3.1 release in a few weeks time.

commit de1213be95c3551c32ac696d226be5f4e17d1eda
Author: Nathan Scott <nathans@redhat.com>
Date:   Fri Jan 11 10:03:10 2019 +1100

    pcp-dstat: handle complete instance removal with --full mode
    
    If an option (-D, -C, ...) has been used to remove all instances
    of a metric, the expected behaviour (from the original dstat) is
    to drop that column from the report entirely.
    
    Resolves Fedora BZ #1661912.

Comment 3 Fedora Update System 2019-02-27 06:20:17 UTC
pcp-4.3.1-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-05aa494895

Comment 4 Fedora Update System 2019-02-27 06:20:54 UTC
pcp-4.3.1-1.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2019-2f1f39ec7a

Comment 5 Fedora Update System 2019-02-28 18:55:30 UTC
pcp-4.3.1-1.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-2f1f39ec7a

Comment 6 Fedora Update System 2019-02-28 21:26:20 UTC
pcp-4.3.1-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-05aa494895

Comment 7 Fedora Update System 2019-03-07 05:02:11 UTC
pcp-4.3.1-2.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-dc1a78c8d2

Comment 8 Fedora Update System 2019-03-07 05:02:39 UTC
pcp-4.3.1-2.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2019-26d6a351e6

Comment 9 Fedora Update System 2019-03-07 20:32:00 UTC
pcp-4.3.1-2.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-dc1a78c8d2

Comment 10 Fedora Update System 2019-03-07 21:03:47 UTC
pcp-4.3.1-2.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-26d6a351e6

Comment 11 Fedora Update System 2019-03-17 07:20:23 UTC
pcp-4.3.1-2.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Fedora Update System 2019-03-17 19:35:33 UTC
pcp-4.3.1-2.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.