Bug 1591603

Is this issue easily reproducible ?

Is kernel reporting any failure ?

Is any other 'read' from multipath device working ?

Is there ANY path actually working ??
(since there is set 'queue_if_no_path' for MP device - with no working path there will be hanging processes trying to read/write)

IMHO from light overlook of reported case all seems to work as designed.

So, the likely issue here is that the default configuration for Compellent storage arrays is to queue forever (or until manual intervention happens) if there are no usable paths. These default configurations come directly from the storage vendors. We only change them if they are actually broken.

If want your queued IO to automatically fail back after a timeout, you can override this. In /etc/multipath.conf add a section that looks like this



devices {
        device {
                vendor "COMPELNT"
                product "Compellent Vol"
                no_path_retries <number_of_retries>
        }
}



By default, multipathd retries a failed path every 5 seconds, so the total length of time multipath would queue for, after the last path to your storage had failed, would be 5 * <number_of_retries>.

If you have multiple different types of devices, and you want them all to queue for a standard length of time, you can add a section that looks like this instead


devices {
        device {
                all_devs yes
                no_path_retries <number_of_retries>
        }
}

Let me know if that clears this up.

Wouldn't it be better if we setup a timeout for blkid?

We have a script in OpenStack for ceph-disks that is running re-scan of the disks on regular basis, and I have seen situations where it actually bogged down the server by running too many blkid processes that were hang definitely.

Would there be actually any down-side to having such timeout implemented?

When multipath is set to queue forever, it won't return the IO until it can submit it.  So even if blkid could timeout (which is would probably have to do by using async IO), the IO would still by stuck on the device. It really seems like if you don't want that IO to queue forever, don't have multipath queue the IO forever.  The "all_devs" option above is what RHEV uses to deal with issues like this.