Description of problem: I have an MSA1000 with 4 units presented back to the host which has 2 host bus adapters. When I bring up the host I am able to view the luns with multipath -v2 -d options but the multipath devices are not listed with the multipath -ll options. The multipath devices are also not created. I have also uploaded the multipath.conf file which shows the 2 internal disks blacklisted by their wwid. The user_friendly_names has been enabled and aliases have been defined for the MSA luns [root@perf6 ~]# multipath -v2 -d create: oramp1 (3600805f30015987000000000768a0019) COMPAQ,MSA1000 VOLUME [size=68G][features=0][hwhandler=1 hp_sw] \_ round-robin 0 [prio=2][undef] \_ 2:0:0:5 sdc 8:32 [undef][ready] \_ 3:0:0:5 sdg 8:96 [undef][ready] create: oramp2 (3600805f30015987000000000d643001a) COMPAQ,MSA1000 VOLUME [size=68G][features=0][hwhandler=1 hp_sw] \_ round-robin 0 [prio=2][undef] \_ 2:0:0:6 sdd 8:48 [undef][ready] \_ 3:0:0:6 sdh 8:112 [undef][ready] create: mpath4 (3600805f3001598700000000086fc001b) COMPAQ,MSA1000 VOLUME [size=68G][features=0][hwhandler=1 hp_sw] \_ round-robin 0 [prio=2][undef] \_ 2:0:0:7 sde 8:64 [undef][ready] \_ 3:0:0:7 sdi 8:128 [undef][ready] create: mpath5 (3600805f300159870000000000984001c) COMPAQ,MSA1000 VOLUME [size=34G][features=0][hwhandler=1 hp_sw] \_ round-robin 0 [prio=2][undef] \_ 2:0:0:8 sdf 8:80 [undef][ready] \_ 3:0:0:8 sdj 8:144 [undef][ready] [root@perf6 by-id]# multipath -ll [root@perf6 by-id]# The long listing of /dev/disk/by-id also shows the wwid for the 4 luns lrwxrwxrwx 1 root root 9 Feb 1 11:37 ata-LITE-ON_COMBO_SOHC-5236V_2005120800024868 -> ../../hdb lrwxrwxrwx 1 root root 9 Feb 1 11:37 scsi-20010b9fc0813836e -> ../../sda lrwxrwxrwx 1 root root 10 Feb 1 11:37 scsi-20010b9fc0813836e-part1 -> ../../sda1 lrwxrwxrwx 1 root root 10 Feb 1 11:37 scsi-20010b9fc0813836e-part2 -> ../../sda2 lrwxrwxrwx 1 root root 10 Feb 1 11:37 scsi-20010b9fc0813836e-part3 -> ../../sda3 lrwxrwxrwx 1 root root 9 Feb 1 11:37 scsi-20010b9fc08148885 -> ../../sdb lrwxrwxrwx 1 root root 10 Feb 1 11:37 scsi-20010b9fc08148885-part1 -> ../../sdb1 lrwxrwxrwx 1 root root 10 Feb 1 11:37 scsi-20010b9fc08148885-part2 -> ../../sdb2 lrwxrwxrwx 1 root root 10 Feb 1 11:37 scsi-20010b9fc08148885-part3 -> ../../sdb3 lrwxrwxrwx 1 root root 9 Feb 1 11:38 scsi-3600805f300159870000000000984001c -> ../../sdj lrwxrwxrwx 1 root root 9 Feb 1 11:38 scsi-3600805f30015987000000000768a0019 -> ../../sdc lrwxrwxrwx 1 root root 9 Feb 1 11:38 scsi-3600805f3001598700000000086fc001b -> ../../sde lrwxrwxrwx 1 root root 9 Feb 1 11:38 scsi-3600805f30015987000000000d643001a -> ../../sdh Version-Release number of selected component (if applicable): How reproducible: The system is in this state and I am able to reproduce this over and over again. Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Created attachment 147117 [details] This is the multipath configuration file used for this setup.
Can you please run # multipath -v6 and send me a copy of the output?
dm-0: blacklisted dm-1: blacklisted dm-2: blacklisted dm-3: blacklisted fd0: blacklisted hdb: blacklisted md0: blacklisted ram0: blacklisted ram10: blacklisted ram11: blacklisted ram12: blacklisted ram13: blacklisted ram14: blacklisted ram15: blacklisted ram1: blacklisted ram2: blacklisted ram3: blacklisted ram4: blacklisted ram5: blacklisted ram6: blacklisted ram7: blacklisted ram8: blacklisted ram9: blacklisted sda: not found in pathvec sda: mask = 0x1f sda: bus = 1 sda: dev_t = 8:0 sda: size = 142264000 sda: vendor = COMPAQ sda: product = BD07287B4C sda: rev = HPB5 sda: h:b:t:l = 0:0:4:0 sda: serial = D20KRZAK sda: path checker = readsector0 (config file default) sda: state = 2 sda: getprio = NULL (internal default) sda: prio = 1 sda: getuid = /sbin/scsi_id -g -u -s /block/%n (config file default) sda: uid = 20010b9fc0813836e (callout) sdb: not found in pathvec sdb: mask = 0x1f sdb: bus = 1 sdb: dev_t = 8:16 sdb: size = 142264000 sdb: vendor = COMPAQ sdb: product = BD07287B4C sdb: rev = HPB5 sdb: h:b:t:l = 0:0:6:0 sdb: serial = D20LYCWK sdb: path checker = readsector0 (config file default) sdb: state = 2 sdb: getprio = NULL (internal default) sdb: prio = 1 sdb: getuid = /sbin/scsi_id -g -u -s /block/%n (config file default) sdb: uid = 20010b9fc08148885 (callout) sdc: not found in pathvec sdc: mask = 0x1f sdc: bus = 1 sdc: dev_t = 8:32 sdc: size = 142239510 sdc: vendor = COMPAQ sdc: product = MSA1000 VOLUME sdc: rev = 4.32 sdc: h:b:t:l = 2:0:0:5 sdc: tgt_node_name = 0x500805f300159870 sdc: serial = P56350GX3RJ0FJ sdc: path checker = hp_sw (controller setting) sdc: state = 2 sdc: getprio = NULL (internal default) sdc: prio = 1 sdc: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdc: uid = 3600805f30015987000000000768a0019 (callout) sdd: not found in pathvec sdd: mask = 0x1f sdd: bus = 1 sdd: dev_t = 8:48 sdd: size = 142239510 sdd: vendor = COMPAQ sdd: product = MSA1000 VOLUME sdd: rev = 4.32 sdd: h:b:t:l = 2:0:0:6 sdd: tgt_node_name = 0x500805f300159870 sdd: serial = P56350GX3RJ0FJ sdd: path checker = hp_sw (controller setting) sdd: state = 2 sdd: getprio = NULL (internal default) sdd: prio = 1 sdd: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdd: uid = 3600805f30015987000000000d643001a (callout) sde: not found in pathvec sde: mask = 0x1f sde: bus = 1 sde: dev_t = 8:64 sde: size = 142239510 sde: vendor = COMPAQ sde: product = MSA1000 VOLUME sde: rev = 4.32 sde: h:b:t:l = 2:0:0:7 sde: tgt_node_name = 0x500805f300159870 sde: serial = P56350GX3RJ0FJ sde: path checker = hp_sw (controller setting) sde: state = 2 sde: getprio = NULL (internal default) sde: prio = 1 sde: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sde: uid = 3600805f3001598700000000086fc001b (callout) sdf: not found in pathvec sdf: mask = 0x1f sdf: bus = 1 sdf: dev_t = 8:80 sdf: size = 71119755 sdf: vendor = COMPAQ sdf: product = MSA1000 VOLUME sdf: rev = 4.32 sdf: h:b:t:l = 2:0:0:8 sdf: tgt_node_name = 0x500805f300159870 sdf: serial = P56350GX3RJ0FJ sdf: path checker = hp_sw (controller setting) sdf: state = 2 sdf: getprio = NULL (internal default) sdf: prio = 1 sdf: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdf: uid = 3600805f300159870000000000984001c (callout) sdg: not found in pathvec sdg: mask = 0x1f sdg: bus = 1 sdg: dev_t = 8:96 sdg: size = 142239510 sdg: vendor = COMPAQ sdg: product = MSA1000 VOLUME sdg: rev = 4.32 sdg: h:b:t:l = 3:0:0:5 sdg: tgt_node_name = 0x500805f300159870 sdg: serial = P56350GX3RJ0FJ sdg: path checker = hp_sw (controller setting) sdg: state = 2 sdg: getprio = NULL (internal default) sdg: prio = 1 sdg: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdg: uid = 3600805f30015987000000000768a0019 (callout) sdh: not found in pathvec sdh: mask = 0x1f sdh: bus = 1 sdh: dev_t = 8:112 sdh: size = 142239510 sdh: vendor = COMPAQ sdh: product = MSA1000 VOLUME sdh: rev = 4.32 sdh: h:b:t:l = 3:0:0:6 sdh: tgt_node_name = 0x500805f300159870 sdh: serial = P56350GX3RJ0FJ sdh: path checker = hp_sw (controller setting) sdh: state = 2 sdh: getprio = NULL (internal default) sdh: prio = 1 sdh: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdh: uid = 3600805f30015987000000000d643001a (callout) sdi: not found in pathvec sdi: mask = 0x1f sdi: bus = 1 sdi: dev_t = 8:128 sdi: size = 142239510 sdi: vendor = COMPAQ sdi: product = MSA1000 VOLUME sdi: rev = 4.32 sdi: h:b:t:l = 3:0:0:7 sdi: tgt_node_name = 0x500805f300159870 sdi: serial = P56350GX3RJ0FJ sdi: path checker = hp_sw (controller setting) sdi: state = 2 sdi: getprio = NULL (internal default) sdi: prio = 1 sdi: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdi: uid = 3600805f3001598700000000086fc001b (callout) sdj: not found in pathvec sdj: mask = 0x1f sdj: bus = 1 sdj: dev_t = 8:144 sdj: size = 71119755 sdj: vendor = COMPAQ sdj: product = MSA1000 VOLUME sdj: rev = 4.32 sdj: h:b:t:l = 3:0:0:8 sdj: tgt_node_name = 0x500805f300159870 sdj: serial = P56350GX3RJ0FJ sdj: path checker = hp_sw (controller setting) sdj: state = 2 sdj: getprio = NULL (internal default) sdj: prio = 1 sdj: getuid = /sbin/scsi_id -g -u -s /block/%n (controller setting) sdj: uid = 3600805f300159870000000000984001c (callout) ===== paths list ===== uuid hcil dev dev_t pri dm_st chk_st vend/pr 20010b9fc0813836e 0:0:4:0 sda 8:0 1 [undef][ready] COMPAQ, 20010b9fc08148885 0:0:6:0 sdb 8:16 1 [undef][ready] COMPAQ, 3600805f30015987000000000768a0019 2:0:0:5 sdc 8:32 1 [undef][ready] COMPAQ, 3600805f30015987000000000d643001a 2:0:0:6 sdd 8:48 1 [undef][ready] COMPAQ, 3600805f3001598700000000086fc001b 2:0:0:7 sde 8:64 1 [undef][ready] COMPAQ, 3600805f300159870000000000984001c 2:0:0:8 sdf 8:80 1 [undef][ready] COMPAQ, 3600805f30015987000000000768a0019 3:0:0:5 sdg 8:96 1 [undef][ready] COMPAQ, 3600805f30015987000000000d643001a 3:0:0:6 sdh 8:112 1 [undef][ready] COMPAQ, 3600805f3001598700000000086fc001b 3:0:0:7 sdi 8:128 1 [undef][ready] COMPAQ, 3600805f300159870000000000984001c 3:0:0:8 sdj 8:144 1 [undef][ready] COMPAQ, 20010b9fc0813836e: blacklisted 20010b9fc08148885: blacklisted Found matching wwid [3600805f30015987000000000768a0019] in bindings file. Setting alias to mpath2 sdc: ownership set to mpath2 sdc: not found in pathvec sdc: mask = 0xc sdc: state = 2 sdc: prio = 1 sdg: ownership set to mpath2 sdg: not found in pathvec sdg: mask = 0xc sdg: state = 2 sdg: prio = 1 mpath2: pgfailover = -1 (internal default) mpath2: pgpolicy = multibus (controller setting) mpath2: selector = round-robin 0 (controller setting) mpath2: features = 0 (controller setting) mpath2: hwhandler = 1 hp_sw (controller setting) mpath2: rr_weight = 1 (controller setting) mpath2: minio = 1000 (controller setting) mpath2: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath2: set ACT_CREATE (map does not exist) mpath2: domap (0) failure for create/reload map sdc: orphaned sdg: orphaned Found matching wwid [3600805f30015987000000000d643001a] in bindings file. Setting alias to mpath3 sdd: ownership set to mpath3 sdd: not found in pathvec sdd: mask = 0xc sdd: state = 2 sdd: prio = 1 sdh: ownership set to mpath3 sdh: not found in pathvec sdh: mask = 0xc sdh: state = 2 sdh: prio = 1 mpath3: pgfailover = -1 (internal default) mpath3: pgpolicy = multibus (controller setting) mpath3: selector = round-robin 0 (controller setting) mpath3: features = 0 (controller setting) mpath3: hwhandler = 1 hp_sw (controller setting) mpath3: rr_weight = 1 (controller setting) mpath3: minio = 1000 (controller setting) mpath3: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath3: set ACT_CREATE (map does not exist) mpath3: domap (0) failure for create/reload map sdd: orphaned sdh: orphaned Found matching wwid [3600805f3001598700000000086fc001b] in bindings file. Setting alias to mpath4 sde: ownership set to mpath4 sde: not found in pathvec sde: mask = 0xc sde: state = 2 sde: prio = 1 sdi: ownership set to mpath4 sdi: not found in pathvec sdi: mask = 0xc sdi: state = 2 sdi: prio = 1 mpath4: pgfailover = -1 (internal default) mpath4: pgpolicy = multibus (controller setting) mpath4: selector = round-robin 0 (controller setting) mpath4: features = 0 (controller setting) mpath4: hwhandler = 1 hp_sw (controller setting) mpath4: rr_weight = 1 (controller setting) mpath4: minio = 1000 (controller setting) mpath4: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath4: set ACT_CREATE (map does not exist) mpath4: domap (0) failure for create/reload map sde: orphaned sdi: orphaned Found matching wwid [3600805f300159870000000000984001c] in bindings file. Setting alias to mpath5 sdf: ownership set to mpath5 sdf: not found in pathvec sdf: mask = 0xc sdf: state = 2 sdf: prio = 1 sdj: ownership set to mpath5 sdj: not found in pathvec sdj: mask = 0xc sdj: state = 2 sdj: prio = 1 mpath5: pgfailover = -1 (internal default) mpath5: pgpolicy = multibus (controller setting) mpath5: selector = round-robin 0 (controller setting) mpath5: features = 0 (controller setting) mpath5: hwhandler = 1 hp_sw (controller setting) mpath5: rr_weight = 1 (controller setting) mpath5: minio = 1000 (controller setting) mpath5: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath5: set ACT_CREATE (map does not exist) mpath5: domap (0) failure for create/reload map sdf: orphaned sdj: orphaned Found matching wwid [3600805f30015987000000000768a0019] in bindings file. Setting alias to mpath2 sdc: ownership set to mpath2 sdc: not found in pathvec sdc: mask = 0xc sdc: path checker = hp_sw (controller setting) sdc: state = 2 sdc: getprio = NULL (internal default) sdc: prio = 1 sdg: ownership set to mpath2 sdg: not found in pathvec sdg: mask = 0xc sdg: path checker = hp_sw (controller setting) sdg: state = 2 sdg: getprio = NULL (internal default) sdg: prio = 1 mpath2: pgfailover = -1 (internal default) mpath2: pgpolicy = multibus (controller setting) mpath2: selector = round-robin 0 (controller setting) mpath2: features = 0 (controller setting) mpath2: hwhandler = 1 hp_sw (controller setting) mpath2: rr_weight = 1 (controller setting) mpath2: minio = 1000 (controller setting) mpath2: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath2: set ACT_CREATE (map does not exist) mpath2: domap (0) failure for create/reload map sdc: orphaned sdg: orphaned Found matching wwid [3600805f30015987000000000d643001a] in bindings file. Setting alias to mpath3 sdd: ownership set to mpath3 sdd: not found in pathvec sdd: mask = 0xc sdd: path checker = hp_sw (controller setting) sdd: state = 2 sdd: getprio = NULL (internal default) sdd: prio = 1 sdh: ownership set to mpath3 sdh: not found in pathvec sdh: mask = 0xc sdh: path checker = hp_sw (controller setting) sdh: state = 2 sdh: getprio = NULL (internal default) sdh: prio = 1 mpath3: pgfailover = -1 (internal default) mpath3: pgpolicy = multibus (controller setting) mpath3: selector = round-robin 0 (controller setting) mpath3: features = 0 (controller setting) mpath3: hwhandler = 1 hp_sw (controller setting) mpath3: rr_weight = 1 (controller setting) mpath3: minio = 1000 (controller setting) mpath3: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath3: set ACT_CREATE (map does not exist) mpath3: domap (0) failure for create/reload map sdd: orphaned sdh: orphaned Found matching wwid [3600805f3001598700000000086fc001b] in bindings file. Setting alias to mpath4 sde: ownership set to mpath4 sde: not found in pathvec sde: mask = 0xc sde: path checker = hp_sw (controller setting) sde: state = 2 sde: getprio = NULL (internal default) sde: prio = 1 sdi: ownership set to mpath4 sdi: not found in pathvec sdi: mask = 0xc sdi: path checker = hp_sw (controller setting) sdi: state = 2 sdi: getprio = NULL (internal default) sdi: prio = 1 mpath4: pgfailover = -1 (internal default) mpath4: pgpolicy = multibus (controller setting) mpath4: selector = round-robin 0 (controller setting) mpath4: features = 0 (controller setting) mpath4: hwhandler = 1 hp_sw (controller setting) mpath4: rr_weight = 1 (controller setting) mpath4: minio = 1000 (controller setting) mpath4: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath4: set ACT_CREATE (map does not exist) mpath4: domap (0) failure for create/reload map sde: orphaned sdi: orphaned Found matching wwid [3600805f300159870000000000984001c] in bindings file. Setting alias to mpath5 sdf: ownership set to mpath5 sdf: not found in pathvec sdf: mask = 0xc sdf: path checker = hp_sw (controller setting) sdf: state = 2 sdf: getprio = NULL (internal default) sdf: prio = 1 sdj: ownership set to mpath5 sdj: not found in pathvec sdj: mask = 0xc sdj: path checker = hp_sw (controller setting) sdj: state = 2 sdj: getprio = NULL (internal default) sdj: prio = 1 mpath5: pgfailover = -1 (internal default) mpath5: pgpolicy = multibus (controller setting) mpath5: selector = round-robin 0 (controller setting) mpath5: features = 0 (controller setting) mpath5: hwhandler = 1 hp_sw (controller setting) mpath5: rr_weight = 1 (controller setting) mpath5: minio = 1000 (controller setting) mpath5: no_path_retry = NONE (internal default) pg_timeout = NONE (internal default) mpath5: set ACT_CREATE (map does not exist) mpath5: domap (0) failure for create/reload map sdf: orphaned sdj: orphaned
Just to verify, you removed the aliases from you multipath.conf file before this last run, correct? Otherwise, I have no idea why you would be getting lines like this: Found matching wwid [3600805f300159870000000000984001c] in bindings file. Setting alias to mpath5 There appear to be a couple places where multipath could be failing, but most likely, it's failing on the actual device-mapper create call. Can you please run: # rpm -q device-mapper # dmsetup table and copy the output into this bugzilla. I'm going to generate a new device-mapper-multipath package that prints out some debugging information.
I did remove the aliases from the multipath.conf file before the last run. Here is the output [root@perf6 ~]# rpm -q device-mapper device-mapper-1.02.13-1.el5 device-mapper-1.02.13-1.el5 [root@perf6 ~]# dmsetup table VolGroup01-LogVol01: 0 4063232 linear 8:18 137953664 VolGroup01-LogVol00: 0 137953280 linear 8:18 384 VolGroup00-LogVol01: 0 4063232 linear 8:2 137953664 VolGroup00-LogVol00: 0 137953280 linear 8:2 384
The hp_sw hardware handler was originally going to be in RHEL5.0. Now it will be coming out in 5.1, I believe. Unfortunately, the default configuration for some devices wasn't changed to reflect that. This issue effects devices with: vendor "DEC" product "HSG80" and vendor "(COMPAQ|HP)" product "MSA1.*" If /var/log/messages has lines like the following after a failed multipath run: device-mapper: table: 253:4: multipath: unknown hardware handler type device-mapper: ioctl: error adding target to table then the this is the problem you are seeing. To work around this, you must add a devices section for these devices to /etc/multipath.conf. The devices section below will fix this problem for all effected device types. devices { device { vendor "DEC" product "HSG80" path_grouping_policy group_by_serial } device { vendor "(COMPAQ|HP)" product "MSA1.*" path_grouping_policy group_by_serial } }
Adding the lines into the multipath.conf resolved this issue.
Can you let me know either when this gets in, or if it's not going to make it, when you know that it will slip, so that I can either updated the config, or close the bug out?
Ben this did not make 5.1
These devices now use the hp_sw handler which will be in RHEL 5.2
*** Bug 348951 has been marked as a duplicate of this bug. ***
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on the solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHBA-2008-0337.html