Hide Forgot
+++ This bug was initially created as a clone of Bug #1305589 +++ Description of problem: Opening at customer request. Is it possible to get a warning if the dm_multipath module is loaded, multipath devices are present, but multipathd is not set to on? Perhaps when using the 'multipath' command. The idea is that most users never use the multipathd command; they configure multipathing and use the multipath command. The multipathd service is an integral part of multipathing, so we should at least give an informative message if it is not running and the system is using multipath. Why do we not auto-start multipathd if the module is in use? Are there use cases where we would want to have multipath devices, but not use multipathd? I have seen many cases where proper failover/mapping was not acheived because multipathd was not running and no one knew. Could we request an auto load for multipathd instead of the warning? Maybe both? Thanks for your time. Please let me know if I can be of any help. Version-Release number of selected component (if applicable): device-mapper-multipath-0.4.9-87.el6.x86_64 Expected Results: - multipathd service autostart with module load - warning if service is not running when using multipath command --- Additional comment from Ben Marzinski on 2016-02-08 15:04:02 EST --- Loading the module can't autostart the service. There's no hook for anything like that in the module code. Besides, if there are no multipath devices, we don't want to start the service. If the service is enabled in systemd, it will autostart on boot, as long a /etc/multipath.conf exists. Creating the configuration file the recommended way, with # mpathconf --enable does enable the service in systemd. However, it does not autostart the service at that point. This is because users often want to actually edit the configuration file before starting the service the first time. The service will autostart on the next reboot. There are two ways to actually create multipath devices. Either multipathd will automatically create them, or they can be created manually by running multipath. I can make multipath print a warning message if multipathd isn't running. This should solve the case where there are multipath devices but multipathd isn't running. Does this sound reasonable? --- Additional comment from John Pittman on 2016-02-08 15:12:22 EST --- Yes Ben that sounds perfect. Thanks a lot. John --- Additional comment from Milan P. Gandhi on 2016-07-24 09:19 EDT --- With the attached patch, dm-multipath commands e.g. multipath -v2, multipath -ll etc. now checks if there are multipath device maps created, and multipathd service is running or not? If the multipath device maps are created, but multipathd service is not running, then in such case there will be a warning message displayed to inform the user that IO failover/failback may not work as expected without multipathd process running.
Created attachment 1183370 [details] Proposed Patch v1
dm-multipath command outputs before applying the attached patch: [root@testhost ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.2 (Maipo) [root@testhost ~]# uname -a Linux testhost 3.10.0-327.22.2.el7.x86_64 #1 SMP Thu Jun 9 10:09:10 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux [root@testhost ~]# rpm -qa|grep -i multipath device-mapper-multipath-0.4.9-85.el7_2.5.x86_64 device-mapper-multipath-libs-0.4.9-85.el7_2.5.x86_64 [root@testhost ~]# Currently "multipathd" service is running: [root@testhost ~]# systemctl status multipathd ● multipathd.service - Device-Mapper Multipath Device Controller Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2016-07-24 17:06:31 IST; 4min 4s ago Main PID: 517 (multipathd) CGroup: /system.slice/multipathd.service └─517 /sbin/multipathd [...] [root@testhost ~]# [root@testhost ~]# Stop the "multipathd" service: [root@testhost ~]# systemctl stop multipathd [root@testhost ~]# systemctl status multipathd ● multipathd.service - Device-Mapper Multipath Device Controller Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled) Active: inactive (dead) since Sun 2016-07-24 17:10:42 IST; 2s ago Main PID: 517 (code=exited, status=0/SUCCESS) [...] Jul 24 17:10:42 testhost systemd[1]: Stopped Device-Mapper Multipath Device Controller. [root@testhost ~]# With "multipathd" service stopped, we do not get any warnings for "multipath -v2, -ll, -v4" commands: [root@testhost ~]# multipath -ll mpathd (1IET_00050004) dm-5 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:4 sdd 8:48 active ready running mpathc (1IET_00050003) dm-4 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:3 sdc 8:32 active ready running mpathb (1IET_00050002) dm-3 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:2 sdb 8:16 active ready running mpatha (1IET_00050001) dm-2 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:1 sda 8:0 active ready running [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# multipath -v2 [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# multipath -F [root@testhost ~]# multipath -v2 create: mpatha (1IET_00050001) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:1 sda 8:0 undef ready running create: mpathb (1IET_00050002) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:2 sdb 8:16 undef ready running create: mpathc (1IET_00050003) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:3 sdc 8:32 undef ready running create: mpathd (1IET_00050004) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:4 sdd 8:48 undef ready running [root@testhost ~]# [root@testhost ~]#
Using the test dm-multipath packages containing the attached patch: Currently "multipathd" service is running on the server: ======================================================= [root@testhost ~]# [root@testhost ~]# ps aux|grep -i multip root 520 0.0 0.6 252732 6920 ? SLl 18:13 0:00 /sbin/multipathd root 2464 0.0 0.0 112652 980 pts/0 S+ 18:16 0:00 grep --color=auto -i multip [root@testhost ~]# Now stop the "multipathd" service: ================================= [root@testhost ~]# systemctl stop multipathd [root@testhost ~]# systemctl status multipathd ● multipathd.service - Device-Mapper Multipath Device Controller Loaded: loaded (/usr/lib/systemd/system/multipathd.service; enabled; vendor preset: enabled) Active: inactive (dead) since Sun 2016-07-24 18:17:52 IST; 4s ago Process: 516 ExecStart=/sbin/multipathd (code=exited, status=0/SUCCESS) Process: 507 ExecStartPre=/sbin/multipath -A (code=exited, status=0/SUCCESS) Process: 488 ExecStartPre=/sbin/modprobe dm-multipath (code=exited, status=0/SUCCESS) Main PID: 520 (code=exited, status=0/SUCCESS) [...] Jul 24 18:17:52 testhost systemd[1]: Stopped Device-Mapper Multipath Device Controller. [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# ps aux|grep -i multip root 2613 0.0 0.0 112648 980 pts/0 R+ 18:18 0:00 grep --color=auto -i multip [root@testhost ~]# [root@testhost ~]# The "multipath -v2", "multipath -ll" commands now verifies if there is at least one multipath device present on server, if yes, it will check if "multipathd" process is running or not. Since there are multipath device maps present, but "multipathd" service is not running, we get the warning as seen in following command output: ======================================================= [root@testhost ~]# multipath -ll mpathd (1IET_00050004) dm-5 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:4 sdd 8:48 active ready running mpathc (1IET_00050003) dm-4 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:3 sdc 8:32 active ready running mpathb (1IET_00050002) dm-3 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:2 sdb 8:16 active ready running mpatha (1IET_00050001) dm-2 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:1 sda 8:0 active ready running Jul 24 18:18:08 | multipath device maps are present, but 'multipathd' service is not running Jul 24 18:18:08 | IO failover/failback will not work without 'multipathd' service running [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# multipath -v2 Jul 24 18:18:12 | multipath device maps are present, but 'multipathd' service is not running Jul 24 18:18:12 | IO failover/failback will not work without 'multipathd' service running [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# multipath -ll mpathd (1IET_00050004) dm-5 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:4 sdd 8:48 active ready running mpathc (1IET_00050003) dm-4 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:3 sdc 8:32 active ready running mpathb (1IET_00050002) dm-3 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:2 sdb 8:16 active ready running mpatha (1IET_00050001) dm-2 IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 3:0:0:1 sda 8:0 active ready running Jul 24 18:18:16 | multipath device maps are present, but 'multipathd' service is not running Jul 24 18:18:16 | IO failover/failback will not work without 'multipathd' service running [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# [root@testhost ~]# multipath -F [root@testhost ~]# [root@testhost ~]# multipath -v2 create: mpatha (1IET_00050001) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:1 sda 8:0 undef ready running create: mpathb (1IET_00050002) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:2 sdb 8:16 undef ready running create: mpathc (1IET_00050003) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:3 sdc 8:32 undef ready running create: mpathd (1IET_00050004) undef IET ,VIRTUAL-DISK size=512M features='0' hwhandler='0' wp=undef `-+- policy='service-time 0' prio=1 status=undef `- 3:0:0:4 sdd 8:48 undef ready running Jul 24 18:18:23 | 'multipathd' service is currently not running, IO failover/failback will not work [root@testhost ~]#
Adding pm_ack for this.
Patch applied. Thanks.
Ben: Could you check my edit to the doc text description for this feature (just a slight rearrangement for the release note format). Steven
Looks fine.
Adding a "topic" summary sentence to the doc text. I had thought the first paragraph would serve as a fine summary, but it turns out to be too many characters. I didn't change anything in the approved description.
Verified on device-mapper-multipath-0.4.9-111 1. [root@storageqe-84 ~]# rpm -qa | grep multipath device-mapper-multipath-debuginfo-0.4.9-111.el7.x86_64 device-mapper-multipath-libs-0.4.9-111.el7.x86_64 device-mapper-multipath-0.4.9-111.el7.x86_64 device-mapper-multipath-sysvinit-0.4.9-111.el7.x86_64 device-mapper-multipath-devel-0.4.9-111.el7.x86_64 2. [root@storageqe-84 ~]# mpathconf --enable 3. [root@storageqe-84 ~]# service multipathd restart Restarting multipathd (via systemctl): [ OK ] 4. [root@storageqe-84 ~]# multipath -ll mpathb (353333330000007d0) dm-5 Linux ,scsi_debug size=1.0G features='0' hwhandler='0' wp=rw |-+- policy='service-time 0' prio=1 status=active | `- 5:0:0:0 sdc 8:32 active ready running `-+- policy='service-time 0' prio=1 status=enabled `- 6:0:0:0 sdd 8:48 active ready running mpatha (360fff19abdd9552f8a36e5355226ba27) dm-0 EQLOGIC ,100E-00 size=50G features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 4:0:1:0 sdb 8:16 active ready running 5. [root@storageqe-84 ~]# service multipathd stop Stopping multipathd (via systemctl): [ OK ] 6. [root@storageqe-84 ~]# multipath -ll mpathb (353333330000007d0) dm-5 Linux ,scsi_debug size=1.0G features='0' hwhandler='0' wp=rw |-+- policy='service-time 0' prio=1 status=active | `- 5:0:0:0 sdc 8:32 active ready running `-+- policy='service-time 0' prio=1 status=enabled `- 6:0:0:0 sdd 8:48 active ready running mpatha (360fff19abdd9552f8a36e5355226ba27) dm-0 EQLOGIC ,100E-00 size=50G features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=1 status=active `- 4:0:1:0 sdb 8:16 active ready running May 22 13:00:34 | multipath device maps are present, but 'multipathd' service is not running -------------------------> May 22 13:00:34 | IO failover/failback will not work without 'multipathd' service running ------------------------> ----------------->List multipath devices while multipathd is stopped, multipath prints a warning message now.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:1961