Bug 1794273 - dstat not showing md devices
Summary: dstat not showing md devices
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: pcp
Version: 31
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Nathan Scott
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-23 07:37 UTC by Holger
Modified: 2020-03-16 20:29 UTC (History)
4 users (show)

Fixed In Version: pcp-5.0.3 pcp-5.0.3-2.fc31 pcp-5.0.3-1.fc32
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-11 22:45:58 UTC
Type: Bug


Attachments (Terms of Use)

Description Holger 2020-01-23 07:37:35 UTC
Description of problem:

dstat not showing md devices


Version-Release number of selected component (if applicable):

pcp-dstat 5.0.2

How reproducible:

dstat -D md0

Steps to Reproduce:
1.
2.
3.

Actual results:

You did not select any stats, using -cdngy by default.
----total-usage----  -net/total- ---paging-- ---system--
usr sys idl wai stl|| recv  send|  in   out | int   csw 
  0   0 100   0   0|2303   297 |   0     0 | 422   601 
  0   0 100   0   0|3072   268 |   0     0 | 246   252 
  0   0 100   0   0|2428   236 |   0     0 | 294   238 ^

Expected results:

You did not select any stats, using -cdngy by default.
----total-usage---- --dsk/md0-- -net/total- ---paging-- ---system--
usr sys idl wai stl| read  writ| recv  send|  in   out | int   csw 
  0   0 100   0   0|   0     0 |1868   640 |   0     0 | 191   194 
  0   0 100   0   0|   0     0 |1586   753 |   0     0 | 500   678 
  0   0 100   0   0|   0     0 |2833   268 |   0     0 | 242   230

Additional info:

With version Dstat 0.7.2 this used to work.

Comment 1 Nathan Scott 2020-01-28 23:42:55 UTC
Hmm, looking into it, it turns out it may be difficult to figure out the best path to take here.

Old dstat used the following regex on device names to decide what to classify as a 'disk':
^([hsv]d[a-z]+\d+|cciss/c\d+d\d+p\d+|dm-\d+|md\d+|mmcblk\d+p\d0|VxVM\d+)$

As reported here, this definitely includes md and dm devices as disks.  Somewhat counter intuitively, it excludes actual disks (!) as it considers only partitions as disks in the first part of the regex (this is a bug, partitioning is optional).

One worthy goal here (and I believe this might be why partitions were favoured over disks originally) is to avoid double-accounting - IOW, summing the I/O to both a partition and the device the partitions sits on would make the 'total' column misleading.

In the metrics namespace used by PCP and now dstat, we have 'disk' metrics in separate trees:

$ pminfo disk.dev
(raw disks - this is what dstat nowadays uses, but previously ignored in favour of partitions+DM+MD)

$ pminfo disk.partitions disk.md disk.dm
(the other classes of device, that sit atop disks)

It turns out its not trivial to form a metric which is the union of all four sets, and its not clear we should do that anyway due to the double accounting issue.  We could be bug-for-bug compatible with the original dstat, and use the union of partitions+DM+MD devices as 'disks' ... but that seems to me to be not ideal either (DM and MD devices are being built out of disks and/or partitions).

I think the safest and best approach here will be to add dstat command line options for each class of device.  That way, one knows exactly what they're looking at and we have the best chance of correctly accounting devices in the 'total' columns.  Another smaller advantage of this approach of using disk.dev.* devices for --disk reporting is that this option will (continue to) work on platforms other than Linux.

So to summarise, I'm proposing we leave --disk reporting for actual disks, as it does now, and add plugins/command line options for --part/--partitions, --dm/--device-mapper and --md/--multi-device driver devices.  The 'total' column is then very well-defined for each case.

Comment 2 Nathan Scott 2020-02-12 01:28:24 UTC
commit a88e8cef5f21e1d95b322fbefe8b7e5461ff4cb5
Author: Nathan Scott <nathans@redhat.com>
Date:   Wed Jan 29 16:49:07 2020 +1100

    pcp-dstat: add options for md, dm and disk partition reporting
    
    Keep the --disk/-D options for (actual, physical) disks, and add new
    plugins and options for --dm, --md and --partition based reporting.
    This allows us to report these devices while continuing to report a
    reasonable --disk 'total' column.
    
    Resolves Fedora BZ #1794273

Comment 3 Fedora Update System 2020-02-27 11:26:04 UTC
FEDORA-2020-1948419b2e has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-1948419b2e

Comment 4 Fedora Update System 2020-02-27 18:38:12 UTC
pcp-5.0.3-1.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-1948419b2e

Comment 5 Fedora Update System 2020-02-28 01:27:26 UTC
pcp-5.0.3-1.fc32 has been pushed to the Fedora 32 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-7ac95d60db

Comment 6 Fedora Update System 2020-03-02 05:27:55 UTC
FEDORA-2020-a4236a5dcf has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-a4236a5dcf

Comment 7 Fedora Update System 2020-03-02 19:56:55 UTC
pcp-5.0.3-2.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-a4236a5dcf

Comment 8 Fedora Update System 2020-03-11 22:45:58 UTC
pcp-5.0.3-2.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 9 Fedora Update System 2020-03-16 20:16:52 UTC
pcp-5.0.3-1.fc32 has been pushed to the Fedora 32 stable repository. If problems still persist, please make note of it in this bug report.

Comment 10 Fedora Update System 2020-03-16 20:29:36 UTC
pcp-5.0.3-1.fc32 has been pushed to the Fedora 32 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.