Red Hat Bugzilla – Bug 1145442
Add machine-readable/parseable output option to multipath command
Last modified: 2016-09-14 17:04:16 EDT
Description of problem: Systems administrators often need to pull out informtion about multipath devices and their slave devices. The 'multipath -l' and 'multipath -ll' output formats as they stand make it very difficult to parse useful information out, especially when they differ significantly depending on release and ASCII art to display path topology. Version-Release number of selected component (if applicable): device-mapper-multipath-0.4.7 through 0.4.9 and probably later How reproducible: 100% Steps to Reproduce: 1. multipath -ll Actual results: # RHEL 6 with SVC mpathbz (3600507680c800024c000000000000021) dm-32 IBM,2145 size=1.0T features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 0:0:27:0 sdfp 130:176 active ready running | `- 5:0:24:0 sdwe 69:672 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 0:0:28:0 sdhi 133:128 active ready running `- 5:0:28:0 sdxx 128:624 active ready running # RHEL 5 mpath127 (20017380063ce000d) dm-33 IBM,2810XIV [size=1010G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=12][active] \_ 1:0:22:3 sdaar 132:752 [active][ready] \_ 6:0:9:3 sdayt 68:1296 [active][ready] \_ 6:0:11:3 sdbbh 128:1328 [active][ready] \_ 6:0:12:3 sdbcn 130:1328 [active][ready] \_ 6:0:15:3 sdbgh 8:1616 [active][ready] \_ 6:0:16:3 sdbhn 66:1616 [active][ready] \_ 6:0:19:3 sdbld 128:1584 [active][ready] \_ 1:0:15:3 sdsd 135:272 [active][ready] \_ 1:0:16:3 sdtj 65:528 [active][ready] \_ 1:0:17:3 sdup 67:528 [active][ready] \_ 1:0:18:3 sdvv 69:528 [active][ready] \_ 1:0:19:3 sdxb 71:528 [active][ready] Expected results: Just a suggestion, not saying this is exactly how it should be; obviously this format omits all topology and balancing policy information. # multipath -lm mpathbz 3600507680c800024c000000000000021 dm-32 IBM,2145 0:0:27:0 sdfp 130:176 active ready running mpathbz 3600507680c800024c000000000000021 dm-32 IBM,2145 5:0:24:0 sdwe 69:672 active ready running mpathbz 3600507680c800024c000000000000021 dm-32 IBM,2145 0:0:28:0 sdhi 133:128 active ready running mpathbz 3600507680c800024c000000000000021 dm-32 IBM,2145 5:0:28:0 sdxx 128:624 active ready running mpath127 20017380063ce000d dm-33 IBM,2810XIV 1:0:22:3 sdaar 132:752 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 6:0:9:3 sdayt 68:1296 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 6:0:11:3 sdbbh 128:1328 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 6:0:12:3 sdbcn 130:1328 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 6:0:15:3 sdbgh 8:1616 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 6:0:16:3 sdbhn 66:1616 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 6:0:19:3 sdbld 128:1584 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 1:0:15:3 sdsd 135:272 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 1:0:16:3 sdtj 65:528 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 1:0:17:3 sdup 67:528 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 1:0:18:3 sdvv 69:528 active ready mpath127 20017380063ce000d dm-33 IBM,2810XIV 1:0:19:3 sdxb 71:528 active ready Additional info:
This is an example of someone else who was similarly frustrated by the output format, however this utility doesn't appear to have been maintained and doesn't work on our systems: http://www.tablespace.net/ppath/index.html
This seems mostly handled by the # multipathd show maps format and # multipathd show paths format commands. However these commands print headers, and add spaces for padding that can make parsing harder. I've implemented versions of these commands called # multipathd raw maps format and # multipathd raw paths format that just print the data you ask for in the way you ask for it in the format string. I can backport this to make parsing easier.
Thanks for looking into this. Interesting, I had no idea until now that 'multipathd' had gained some parameters to do anything other than start the daemon. Doesn't this defeat the purpose of the 'multipath' command somewhat? Will that be deprecated? I fail to find the content of the $format parameter documented anywhere in the man pages, am I missing something obvious? The 'raw' option would indeed be useful, however parsing out padding space and headings is relatively trivial - it was the completely variable output formats of 'multipath -ll' that previously presented a real challenge.
(In reply to Scott Rochford from comment #4) > Thanks for looking into this. > > Interesting, I had no idea until now that 'multipathd' had gained some > parameters to do anything other than start the daemon. Doesn't this defeat > the purpose of the 'multipath' command somewhat? Will that be deprecated? Possibly in the future, but there are no immediate plans to do so. > I fail to find the content of the $format parameter documented anywhere in > the man pages, am I missing something obvious? More like something obscure. The format string is just a string using various wildcards to request the information. You can see the supported wildcards with # multipathd show wildcards For instance, you could show all the multipath map information from your example above with: # multipathd show maps format "%n %w %d %s" Unfortunately, RHEL6 has many fewer wildcards than RHEL7, and there isn't one to list what multipath device a path belongs to, so you have to compare wwids, but you can get the path information from your example by running # multipathd show paths format "%w %i %d %t %o %T" > The 'raw' option would indeed be useful, however parsing out padding space > and headings is relatively trivial - it was the completely variable output > formats of 'multipath -ll' that previously presented a real challenge. So, I suppose this bug should be used to both add the raw option, and backport the additional wildcards.
> More like something obscure. The format string is just a string using > various wildcards to request the information. You can see the supported > wildcards with > # multipathd show wildcards Sorry, somehow I read straight past that one on the man page without seeing it. > So, I suppose this bug should be used to both add the raw option, and > backport the additional wildcards. Yes, both would be appreciated, thanks.
The fix for this bug caused a coverity issue. I've fixed the patch to avoid this issue.
1) [root@storageqe-18 ~]# multipathd show paths format "%w %i %d %t %o %T" uuid hcil dev dm_st dev_st chk_st 360a98000324669436c2b45666c567942 4:0:1:0 sdj active running ready 360a98000324669436c2b45666c567942 5:0:1:0 sdz active running ready 360a98000324669436c2b45666c567942 4:0:0:0 sda active running ready 360a98000324669436c2b45666c567942 5:0:0:0 sds active running ready [root@storageqe-18 ~]# multipathd show paths raw format "%w %i %d %t %o %T" 360a98000324669436c2b45666c567942 4:0:1:0 sdj active running ready 360a98000324669436c2b45666c567942 5:0:1:0 sdz active running ready 360a98000324669436c2b45666c567942 4:0:0:0 sda active running ready 360a98000324669436c2b45666c567942 5:0:0:0 sds active running ready 2) [root@storageqe-18 ~]# multipathd show maps format "%n %w %d %s" name uuid sysfs vend/prod/rev mpathc 360a98000324669436c2b45666c567942 dm-0 NETAPP,LUN [root@storageqe-18 ~]# multipathd show maps raw format "%n %w %d %s" mpathc 360a98000324669436c2b45666c567942 dm-0 NETAPP,LUN From step 1 and step 2, # multipathd show paths raw format $FMT and # multipathd show maps raw format $FMT work the same as the non-raw versions, but without headers and extra padding being added to the output. 3) [root@storageqe-18 ~]# multipathd show paths format "%z %m %N %n %R %r %a" serial multipath host WWNN target WWNN host WWPN target WWPN host adapter 2FiCl+EflVyB mpathc 0x20000000c9a02834 0x500a0980891b8dc5 0x10000000c9a02834 0x500a0982991b8dc5 0000:00:06.0 2FiCl+EflVyB mpathc 0x20000000c9a02835 0x500a0980891b8dc5 0x10000000c9a02835 0x500a0981991b8dc5 0000:00:06.0 2FiCl+EflVyB mpathc 0x20000000c9a02834 0x500a0980891b8dc5 0x10000000c9a02834 0x500a0982891b8dc5 0000:00:06.0 2FiCl+EflVyB mpathc 0x20000000c9a02835 0x500a0980891b8dc5 0x10000000c9a02835 0x500a0981891b8dc5 0000:00:06.0 [root@storageqe-18 ~]# multipathd show paths raw format "%z %m %N %n %R %r %a" 2FiCl+EflVyB mpathc 0x20000000c9a02834 0x500a0980891b8dc5 0x10000000c9a02834 0x500a0982991b8dc5 0000:00:06.0 2FiCl+EflVyB mpathc 0x20000000c9a02835 0x500a0980891b8dc5 0x10000000c9a02835 0x500a0981991b8dc5 0000:00:06.0 2FiCl+EflVyB mpathc 0x20000000c9a02834 0x500a0980891b8dc5 0x10000000c9a02834 0x500a0982891b8dc5 0000:00:06.0 2FiCl+EflVyB mpathc 0x20000000c9a02835 0x500a0980891b8dc5 0x10000000c9a02835 0x500a0981891b8dc5 0000:00:06.0 4) [root@storageqe-18 ~]# multipathd show paths raw format %z 2FiCl+EflVyB 2FiCl+EflVyB 2FiCl+EflVyB 2FiCl+EflVyB [root@storageqe-18 ~]# multipathd show paths raw format %m mpathc mpathc mpathc mpathc [root@storageqe-18 ~]# multipathd show paths raw format %N 0x20000000c9a02834 0x20000000c9a02835 0x20000000c9a02834 0x20000000c9a02835 [root@storageqe-18 ~]# multipathd show paths raw format %n 0x500a0980891b8dc5 0x500a0980891b8dc5 0x500a0980891b8dc5 0x500a0980891b8dc5 [root@storageqe-18 ~]# multipathd show paths raw format %R 0x10000000c9a02834 0x10000000c9a02835 0x10000000c9a02834 0x10000000c9a02835 [root@storageqe-18 ~]# multipathd show paths raw format %r 0x500a0982991b8dc5 0x500a0981991b8dc5 0x500a0982891b8dc5 0x500a0981891b8dc5 [root@storageqe-18 ~]# multipathd show paths raw format %a 0000:00:06.0 0000:00:06.0 0000:00:06.0 0000:00:06.0 From step 3 and step 4, the %z, %m, %N, %n, %R, %r, and %a path format wildcards work as expected.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0777.html