Bug 484419

Mike, 

Do you suppose you could try again to get this upstream? The we can backport it to RHEL 5. 

Tom

Maybe it's possible/worthwhile to separate out the two attributes (vendor/product) device-mapper-multipath needs from the rest of the sysfs attributes patch?

Is that all that's needed, the vendor and product attributes? Separating these 2 out is probably the easiest way to get it accepted.

Afaik yes - those are the only attributes multipath needs to retrieve from sysfs that are missing for CCISS (it's just to match the config entries in multipath.conf/hwtable.c).

Where do you expect vendor and model to show up in the sysfs tree.  I currently have a patch that has these entries in:

/sys/class/cciss_logical_drive/c#d#

They should appear in files named "vendor" and "product" inside the device's device directory.

E.g., this is how the accessor functions in RHEL-5's multipath-tools look today:

declare_sysfs_get_str(vendor, "%s/block/%s/device/vendor");
declare_sysfs_get_str(model, "%s/block/%s/device/model");

*** Bug 249585 has been marked as a duplicate of this bug. ***

Mike,
this will probably make it into RHEL4, so we now should have it also for RHEL5.
Please post an appropriate patch for this here.
Thanks,
Tomas

Does everything work as expected? I'm getting reports around that the multi-path is not working, i.e., no devices showing up.

(In reply to comment #11)
> Does everything work as expected? I'm getting reports around that the
> multi-path is not working, i.e., no devices showing up.
I'll test it (ask someone what is missing) in near future.

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

(In reply to comment #11)
> Does everything work as expected? I'm getting reports around that the
> multi-path is not working, i.e., no devices showing up.  

It looks like we need not only vendor/product, but also the bus attribute.
Thanks, Tomas

This patch has the bus attributes too (not backported to rhel 4 yet):

http://marc.info/?l=linux-kernel&m=123912749517010&w=2

Mike Miller has a slightly newer version that addresses Andrew Morton's review comments.

> Mike Miller has a slightly newer version that addresses Andrew Morton's review
> comments.  

Mike, please post your version so we can start testing the newest patch.
Thanks,
Tomas

(In reply to comment #15)
> This patch has the bus attributes too (not backported to rhel 4 yet):
> 
> http://marc.info/?l=linux-kernel&m=123912749517010&w=2

Andrew, are you sure the bus attribute is there? I've seen only
model, number, revision and not bus. See the patch in bz484415 https://bugzilla.redhat.com/attachment.cgi?id=342323

The patchset in http://marc.info/?l=linux-kernel&m=123912749517010&w=2 is not in
rhel 4.8.  My hope was that it would make it there, but it seems an earlier patch by Chase Maupin was taken instead with some later modifications.

(In reply to comment #18)
> The patchset in http://marc.info/?l=linux-kernel&m=123912749517010&w=2 is not
> in
> rhel 4.8.  My hope was that it would make it there, but it seems an earlier
> patch by Chase Maupin was taken instead with some later modifications.  

Andrew,please could you provide a patch with vendor/product/bus attribute for multipath ?
Our latest sources are here http://people.redhat.com/dzickus/el5/ - the biggest number.
Thanks.

Mike Miller is going to backport the http://marc.info/?l=linux-kernel&m=123912749517010&w=2 patches.

We are too late, so I'm moving this to RHEL5.5

I'm hitting a dependency I can't resolve. 

[root@duke] rpm -i kernel-2.6.18-150.el5.src.rpm
error: Failed build dependencies:
        hmaccalc is needed by kernel-2.6.18-150.el5.x86_64

Where can I find the package containing hmaccalc?

(In reply to comment #22)

> Where can I find the package containing hmaccalc?  

I just mailed it to you.

I am going to do this backport for Mike.  I need an ia64 version of the hmaccalc
rpm.

Andrew

BTW, Andrew Morton accepted this patch in his kernel tree.

(In reply to comment #24)
> I am going to do this backport for Mike.  I need an ia64 version of the
> hmaccalc
> rpm.
> 
> Andrew  

I think that you get the error message when you start rpmbuild and not rpm right?
If it is so then can add argument "--without fips" to the command line.
example for x86 : rpmbuild -bp --target=x86_64 --without fips kernel-2.6.spec

I'll also look for hmaccalc and send it to you.
Tomas

Moving to RHEL 5.5 since this was not resolved prior to the beta.

Andrew,
It looks like upstream accepted the vendor/product, but we also need the 'bus' attribute for multipath.

Thanks, Tomas 

(In reply to comment #25)
> BTW, Andrew Morton accepted this patch in his kernel tree.

Bryan,
most of the attributes - model,vendor,id,rev already is in RHEl5. Please let me know if this is enough or if we really need some additional fields.
Thanks, Tomas

Moving it to RHEL5.6 it will be solved there when we know what else is needed.

(In reply to comment #31)
> Moving it to RHEL5.6 it will be solved there when we know what else is needed.    

HP agrees to defer this to RHEL 5.6.

I'm not able to test myself (no hardware) and this BZ appears to have lost its IT tickets - we'll likely need to rely on HP to test this change in 5.6.

*** Bug 475177 has been marked as a duplicate of this bug. ***

Sandy,
this might be resolved in between, it probably needs only some testing with hw I don't have. Please verify that and if a patch is still needed for this post it.

I have compatible hardware that could be used to test this. Please let me know what I need to do.

(In reply to comment #36)
> I have compatible hardware that could be used to test this. Please let me know
> what I need to do.    

Install RHEL5.5 and let us know if multipath works for you. I think the id's are all in so it should work just fine. 
Here should be some description http://www.redhat.com/docs/manuals/enterprise/RHEL-5-manual/en-US/RHEL510/pdf/DM_Multipath.pdf

I've just installed RHEL5.5 and it seems to have the same problems I have already seen:
 * scsi_id has undocumented requirement of the -s parameter
 * scsi_id -s parameter doesn't seem to work properly anyway
 * even when overriding the getuid_callout with a script that returns the correct device UIDs to use with multipath, output of `multipath -ll -v3` still ignores the actual devices with output like the following

.
.
.
cciss!c0d0: exception-listed
cciss!c0d0: not found in pathvec
cciss!c0d0: mask = 0x5
.
.
.

For all of the paths.

Oliver,
thanks for the report.

Ben,
could you please look into this, let us know what else should be added to the cciss driver to make it work with multipath.

RHEL5.5 has the cciss_id tool which should be used instead for CCISS controllers:

# which cciss_id
/sbin/cciss_id
# rpm -qf $(which cciss_id)
device-mapper-multipath-0.4.7-34.el5_5.1

cciss!c0d0: exception-listed

Shows that this is matching a provided blacklist exceptions regex.

Can you post the multipath.conf that you are using and the full output from "multipath -v4"?

Certainly, /sbin/cciss_id fixes the issue with getting a UID which the current version of scsi_id cannot do:

[root@localhost ~]# cciss_id /dev/cciss/c0d0
3600c0ff000da5e68632a124c01000000

However even when using this in the multipath.conf getuid_callout there is still a problem actually configuring the paths, which seems to point to a problem in multipath itself:

[root@localhost ~]# multipath -ll -v3
cciss!c0d0: exception-listed
cciss!c0d0: not found in pathvec
cciss!c0d0: mask = 0x5
.
.
.

Could you please run 

# multipath -v3

and attach the output to the bugzilla

Created attachment 425676 [details]
Output of multipath -v3

It appears as though the cciss driver and the cciss_id callout may provide the needed information. I'll re-assign this to Ben to look at it from the multipath side. 

Ben probably does not have access to this hardware, so please relay any information he requests as needed. 

Tom

Nevermind. I see that it's at:

/sys/block/<devname>/device/cciss<X>/c<X>d<Y>/model

device-mapper-multipath can now create multipath devices on top of cciss devices.

I've just commited the fix. I haven't built a package with it yet.  If you'd like one to try out, I can build one today.

I would definitely like to try out the fix.

Sorry. I put them up but forgot to post a link in the bugzilla.

you can get them at:

http://people.redhat.com/~bmarzins/device-mapper-multipath/rpms/x86_64/

or

http://people.redhat.com/~bmarzins/device-mapper-multipath/rpms/i386/

The packages you want are:

device-mapper-multipath-0.4.7-38.el5.i386.rpm
device-mapper-multipath-debuginfo-0.4.7-38.el5.i386.rpm
kpartx-0.4.7-38.el5.i386.rpm

Verified on RHEL5.6-Server-20101110.0, device-mapper-multipath-0.4.7-39.el5.

I'm hitting dm-multipath + cciss issues while running RHEL 5.6 on an HP Blade. I was trying to configure dm-multipath on the NetApp LUNs (and not on the cciss devices) mapped to my RHEL 5.6 host, but always end up with the error mentioned below:

Config:
RHEL 5.6 Snap1 (2.6.18-231.el5)
cciss driver v3.6.22-RH1

Running any multipath command always throws up the following error:
'can't get driver link: No such file or directory'. 

Running an strace on the multipath command shows the issue at:

...
stat("/sys/block/cciss!c0d0/device/cciss0/c0d0/vendor", 0x7fffc71c4b00) = -1 ENOENT (No such file or directory)
ioctl(3, DM_LIST_DEVICES, 0x110e6460)   = 0
ioctl(3, DM_TABLE_STATUS, 0x110ea470)   = 0
ioctl(3, DM_TABLE_STATUS, 0x110ea470)   = 0
close(3)                                = 0
write(1, "can't get driver link: No such f"..., 49can't get driver link: No such file or directory
) = 49

Seems dm-multipath throws this error when it fails to get the cciss vendor sysfs attribute. I tried blacklisting the local cciss devices in the multipath.conf, but to no avail i.e. the error is seen irrespective of blacklisting the cciss devices.

Is this a known issue? Should I file a new bug for this?

could you attach your /etc/multipath.conf?  Blacklisting cciss devices should solve the issue.  The multipath command works correctly otherwise, right?

Created attachment 461328 [details]
multipath.conf for this scenario

To be honest, I have no idea what

[p[0-9]*]

matches in a POSIX extended regular expression.
could you try replacing

devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"

with something simpler, like

devnode "^cciss.*"

or

devnode "^cciss!c[0-9]d.*"

and see if that works.

Hi Barry,
Since we have a similar bug in hand, could give us some detail information about your verification with this bug, like testing steps?
Thanks very much~

I installed the release on a system with CCISS disks and verified that the expected attributes were in sysfs and where they were expected. Then run multipath and verify no errors are generated.

   I supposed this bug(In reply to comment #65)
> Hi Barry,
> Since we have a similar bug in hand, could give us some detail information
> about your verification with this bug, like testing steps?
> Thanks very much~

I installed the release on a system with CCISS disks and verified that the expected attributes were in sysfs and where they were expected. Then run multipath and verify no errors are generated.

   I supposed this bug should have a precise test case from devel indicating exactly what sysfs attributes were added.

(In reply to comment #64)
> could you try replacing
> 
> devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
> 
> with something simpler, like
> 
> devnode "^cciss.*"
> 
> or
> 
> devnode "^cciss!c[0-9]d.*"
> 
> and see if that works.

Yes, that works. Blacklisting cciss entries using "^cciss.*" & "^cciss!c[0-9]d.*" both work fine. 

Actually the original entry worked fine in previous RHEL5 kernels. So it does look like the multipath.conf parsing is more strict now in RHEL 5.6.

I have some test packages that will hopefully fix this. You can get them at

http://people.redhat.com/bmarzins/device-mapper-multipath/rpms/x86_64/

or

http://people.redhat.com/bmarzins/device-mapper-multipath/rpms/i386/

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
CCISS devices are now supported.

Reminder! There should be a fix present for this BZ in snapshot 3 -- unless otherwise noted in a previous comment.

Please test and update this BZ with test results as soon as possible.

Is this verified or not?

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-0074.html