Hide Forgot
Created attachment 1127985 [details] logs for multipath and var/log/messages Description of problem: Multipath is unable to add new paths and fails in domap for FC volumes Version-Release number of selected component (if applicable): device-mapper-multipath-libs-0.4.9-87.el6.x86_64 device-mapper-multipath-0.4.9-87.el6.x86_64 How reproducible: Easily reproducible Steps to Reproduce: 1.There are 2 active paths to the volume via 2 different FC interface. At a given time one interface is up. 2. There is a script which brings down a FC interface while the other interface is up. After 30 seconds the interface is brought up while the other interface goes through the same process. This is repeated iterations 3. After few repeated iteration of FC port flapping some of the paths do not recover. 4. Our multipath dev_los_tmo is 150 seconds. There is a udev event for add path, but multipath is unable to add it Actual results: Feb 11 19:34:32 hitdev-rhel67 multipathd: delay_wait_checks = DISABLED (internal default) Feb 11 19:34:32 hitdev-rhel67 multipathd: mpathe: failed in domap for addition of new path sdk Feb 11 19:34:32 hitdev-rhel67 multipathd: mpathe: giving up reload Feb 11 19:34:32 hitdev-rhel67 kernel: device-mapper: table: 253:5: multipath: error getting device Feb 11 19:34:32 hitdev-rhel67 kernel: device-mapper: ioctl: error adding target to table Expected results: The paths should be recovered Additional info: Following are attached : 1. /var/log/messages 2. multipath.conf 3. multipath -ll
Hello Raunak, Because we have no Nimble Storage in our lab, could you help provide test result once the package is available? We will do sanityonly testing. thanks!
Yes sure.
Once this happens, does running # multipath add the path. If not, then you can run # multipath -v4 To see why device-mapper is failing the reload. The output should contain something like: Feb 19 10:07:37 | libdevmapper: ioctl/libdm-iface.c(1786): device-mapper: reload ioctl on mpathb failed: Device or resource busy If not, are you able to reproduce this with verbosity 4 added to the defaults section of multipath.conf? This cause a huge amount of messages to be logged, but you are still looking for a libdevmapper message, like above. If these don't work, I can send you a test package that will cut down on all the extra messages, and just print the important ones, so we can figure out why device mapper can't grab the path device. My assumption is that something has it opened exclusively, and you are going to get a "Device or resource busy" message. The question is whether whatever has it open exclusively keeps it open, or only has it open temporarily, so that a retry could fix this.
Have you been able to try the steps I mentioned in Comment #4?
Hi Ben, We tried to recreate this again with the steps you mentioned, but we did not hit it again.
Have you been able to recreate this at all?
This bug was reopened by mistake. If you would like to reopen it, please provide the information requested in Comment #4.