Bug 433498

Summary: [RFE] device-mapper: improve integration between device-mapper-multipath and SCSI layer
Product: Red Hat Enterprise Linux 6 Reporter: Bryn M. Reeves <bmr>
Component: kernelAssignee: LVM and device-mapper development team <lvm-team>
Status: CLOSED CURRENTRELEASE QA Contact: Barry Donahue <bdonahue>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.0CC: bubrown, coughlan, cward, james.brown, jbrier, junichi.nomura, kueda, m-ikeda, mikeda, msnitzer, snagar, tao
Target Milestone: rcKeywords: FutureFeature, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-11 16:05:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 534151    
Attachments:
Description Flags
Headers from the request-based dm patches merged upstream none

Description Bryn M. Reeves 2008-02-19 17:41:26 UTC 
Description of problem:
Currently, the SCSI sense data returned by the host/target/driver layer on an
I/O error is not made available to device-mapper since the latter operates at
the bio layer rather than at the request level.

Having access to the sense buffer would allow dm-mpio to make more informed
decisions when an error occurs, for example, avoiding failing a path entirely if
the sense data indicates a transient error that the host should retry immediately.

Patches have been proposed to implement this but have not been merged upstream.
Further work is ongoing to find a solution that provides this functionality & is
acceptable upstream.

This bugzilla is to track the possible inclusion of such functionality in RHEL6.
Comment 12 Bryn M. Reeves 2010-02-18 11:57:18 UTC 
Hey Piyush,

This is based on the patches from Kiyoshi Ueda and Jun'ichi Nomura from NEC that are already upstream and in Fedora. You can find some discussion of this in the proceedings of the 2007 Ottawa Linux Symposium here:

http://www.linuxinsight.com/ols2007-request-based-device-mapper-multipath-and-dynamic-load-balancing.html

There's also an old LWN article here that contains some of Kiyoshi's posts on the subject:

http://lwn.net/Articles/215058/

There are more recent threads that you can find in the archives, e.g. the posting of the patch series that ended up getting merged.

If you're interested in the upstream commits that introduced these changes the following commits (in Linus' tree, linux-2.6.git) provide the history:

6db4ccd6357f28c9ef7058b3bc48904c4b2ac419
d0bcb8786532b01206f04258eb6b7d4ac858436a
11a68244e16b0c35e122dd55b4e7c595e0fb67a1
9f518b27cf682dd5155a4c1679d52cd4b5be82f2
a77e28c7e1dc1a6a035c7627d4a88ecf3ea09aea
f40c67f0f7e2767f80f7cbcbc1ab86c4113c202e
e6ee8c0b767540f59e20da3ced282601db8aa502
cec47e3d4a861e1d942b3a580d0bbef2700d2bb2

For convenience I'll post the patch headers in a follow-on comment.
Comment 13 Bryn M. Reeves 2010-02-18 11:58:21 UTC 
Created attachment 394892 [details]
Headers from the request-based dm patches merged upstream
Comment 14 Mike Snitzer 2010-02-18 14:54:34 UTC 
Request-based DM multipath does not change multipath-tools at all.  Though the user can choose to make use of 2 new path selectors that were introduced along with request-based DM multipath.

Here are more details on the new path-selectors; from the RHEL6 (alpha2) release notes:

8.  Storage

8.1 Dynamic Load Balancing Added to device-mapper-multipath.
Two new load balancing algorithms have been added to device-mapper-multipath.
One performs path selection based the observed queue depth at each path. The
other selects the path based on the measured service time for previous I/O on
the paths.

The path-selector that is used is configured in /etc/multipath.conf. This is
done in the 'defaults' section (applies to all mpath devices) by setting
"selector" to "queue-length 0" or "service-time 0" or "round-robin 0". For
example:

defaults
 {
      ...
       selector                "queue-length 0"
 }                                                                                                                                                                     

This can also be done for specific multipath devices in the 'device' section
by setting the "path_selector" value. For example:

devices
 {
       device
       {
               ...
               path_selector   "service-time 0"
       }                                                                                                                                                               
 }                                                                                                                                                                     

If no selector is specified, it defaults to "round-robin 0".
Comment 17 Barry Donahue 2010-10-05 15:00:30 UTC 
SanityOnly
Comment 18 releng-rhel@redhat.com 2010-11-11 16:05:52 UTC 
Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.