Bug 491323 - [RHEL 5.3] MRG SAN install problems: RHEL 5.3 base
[RHEL 5.3] MRG SAN install problems: RHEL 5.3 base
Status: CLOSED DUPLICATE of bug 515326
Product: Red Hat Enterprise MRG
Classification: Red Hat
Component: realtime-utilities (Show other bugs)
1.1
other All
low Severity high
: ---
: ---
Assigned To: Red Hat Real Time Maintenance
David Sommerseth
:
Depends On: 482753
Blocks:
  Show dependency treegraph
 
Reported: 2009-03-20 09:02 EDT by IBM Bug Proxy
Modified: 2016-05-22 19:28 EDT (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2009-09-14 16:18:28 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
dmesg output of bad boot RHEL5.3+ MRG 101 (54.18 KB, application/octet-stream)
2009-03-20 09:02 EDT, IBM Bug Proxy
no flags Details

  None (edit)
Description IBM Bug Proxy 2009-03-20 09:02:36 EDT
=Comment: #0=================================================
KEITH MANNTHEY <kmannth@us.ibm.com> - 
I am unable to get a working RHEL5.3 + MRG Beta install working. 

Setup is as follows.  

HS21XM with QLA SAN card. 

DS4k as backend of the SAN.  Requires Multipath and DM_RDAC driver to properly work.  MRG has the
DM-RDAC driver. So Summary of where the box is at. 

Installed RHEL 5.3 onto the SAN unsing "mpath" on the command line. Both paths are up. 

After install make the following changes:

in /etc/multipath.conf :

defaults {
        user_friendly_names yes
}

devices {
        device {
                vendor "IBM"
                product "1814"
                hardware_handler "1 rdac"
                path_checker rdac
                failback immediate
                path_grouping_policy group_by_prio
                prio_callout "/sbin/mpath_prio_rdac /dev/%n"
                no_path_retry queue
        }
}

In /etc/modprobe.conf

alias eth0 bnx2
alias eth1 bnx2
alias eth2 netxen_nic
alias eth3 netxen_nic
alias scsi_hostadapter scsi_dh_rdac
alias scsi_hostadapter1 ata_piix
alias scsi_hostadapter0 qla2xxx


When you install the kernel the machine is not right.  Seeing the output from 
 mkinitrd -f -v /boot/initrd-2.6.24.7-101.el5rt.img 2.6.24.7-101.el5rt

Creating initramfs
Looking for deps of module ehci-hcd
Looking for deps of module ohci-hcd
Looking for deps of module uhci-hcd
Looking for deps of module ext3: mbcache jbd 
Looking for deps of module mbcache
Looking for deps of module jbd
Looking for driver for device mapper/mpath0p2
Found DM device mapper/mpath0p2
Looking for deps of module pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00: scsi_mod scsi_tgt
scsi_transport_fc qla2xxx 
Looking for deps of module scsi_mod
Looking for deps of module sd_mod: scsi_mod 
Looking for deps of module scsi_tgt: scsi_mod 
Looking for deps of module scsi_transport_fc: scsi_mod scsi_tgt 
Looking for deps of module qla2xxx: scsi_mod scsi_tgt scsi_transport_fc 
Looking for deps of module pci:v00008086d0000350Csv00000000sd00000000bc06sc04i00: shpchp 
Looking for deps of module shpchp
Looking for deps of module pci:v00008086d000025F7sv00000000sd00000000bc06sc04i00: shpchp 
Looking for deps of module pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00: scsi_mod scsi_tgt
scsi_transport_fc qla2xxx 
Looking for deps of module pci:v00008086d0000350Csv00000000sd00000000bc06sc04i00: shpchp 
Looking for deps of module pci:v00008086d000025F7sv00000000sd00000000bc06sc04i00: shpchp 
Looking for deps of module scsi_dh_rdac: scsi_mod scsi_dh 
Looking for deps of module scsi_dh: scsi_mod 
Looking for deps of module ata_piix: scsi_mod libata 
Looking for deps of module libata: scsi_mod 
Looking for deps of module ide-disk
Looking for deps of module dm-mod
Looking for deps of module dm-mirror: dm-mod 
Looking for deps of module dm-zero: dm-mod 
Looking for deps of module dm-snapshot: dm-mod 
Looking for deps of module dm-multipath: scsi_mod scsi_dh dm-mod 
Looking for deps of module dm-round-robin: scsi_mod scsi_dh dm-mod dm-multipath 
Looking for deps of module dm-mem-cache
No module dm-mem-cache found for kernel 2.6.24.7-101.el5rt, aborting.

If I remove or comment out 
alias scsi_hostadapter scsi_dh_rdac

I can build a kernel but it is broken.  I will attach dmesg. 

I don't seem to have these sorts of issues with RHEL 5.2 
~
=Comment: #1=================================================
KEITH MANNTHEY <kmannth@us.ibm.com> - 

dmesg output of bad boot RHEL5.3+ MRG 101


=Comment: #3=================================================
KEITH MANNTHEY <kmannth@us.ibm.com> - 
Last kernel tried was MRG -101
=Comment: #4=================================================
KEITH MANNTHEY <kmannth@us.ibm.com> - 
I have not figured out how to work around this. I suspect hacking the mkinitrd scripts would be a
good place to start.
Comment 1 IBM Bug Proxy 2009-03-20 09:02:42 EDT
Created attachment 336040 [details]
dmesg output of bad boot RHEL5.3+ MRG 101
Comment 2 Luis Claudio R. Goncalves 2009-03-20 09:52:58 EDT
Please check:

   Mkinitrd erroneously inserts dependencies to dm-mem-cache (not present on MRG)
   https://bugzilla.redhat.com/show_bug.cgi?id=482753

In short, the version of mkinitrd present in RHEL5.3 forces the inclusion of the following modules: dm-mem-cache dm-region_hash  dm-message dm-raid45

As these modules are not present (or have different names under MRG RT), mkinitrd shouts warnings during kernel installs and aborts when user runs mkinitrd manually.

To workaround this issue you can:

a) Add the parameter "--allow-missing" to your mkinitrd command line or

b) Reproduce the command line used by rpm during a kernel install:

   # /sbin/new-kernel-pkg --package kernel-rt --banner "Red Hat Enterprise Linux 
   (realtime)" --mkinitrd --depmod --install <your-kernel-version>

<your-kernel-version> will be something like: 2.6.24.7-107.el5rt
Comment 3 IBM Bug Proxy 2009-03-21 08:49:01 EDT
------- Comment From jvrao@us.ibm.com 2009-03-20 21:21 EDT-------
(In reply to comment #14)
> a) Add the parameter "--allow-missing" to your mkinitrd command line or
>
> b) Reproduce the command line used by rpm during a kernel install:
>
> # /sbin/new-kernel-pkg --package kernel-rt --banner "Red Hat Enterprise Linux
> (realtime)" --mkinitrd --depmod --install <your-kernel-version>

There are two problems. One is the problem with missing modules that initrd is expecting. That can be easily worked around by the procedure mentioned above.

The second problem is, 5.3 mkinitrd became little smarter(??). It is probing the root device, determining what modules are needed and loading them into initrd.
This step happens before even looking into thge modprobe.conf

Unfortunately mkinitrd does not think that for dm root device scsi_dh_rdac is needed. Hence in the stack of loaded modules scsi_dh_rdac is falling below the qla2xxx. Because of this, there will be TONs of error messages for each lun on each path. This causes long boot times, and sometimes random luns get only one path.

[root@elm3c27 ~]# mkinitrd -f -v  /boot/initrd-2.6.24.7-101.el5rt.img 2.6.24.7-101.el5rt --allow-missing
Creating initramfs
Looking for deps of module ehci-hcd
Looking for deps of module ohci-hcd
Looking for deps of module uhci-hcd
Looking for deps of module ext3: mbcache jbd
Looking for deps of module mbcache
Looking for deps of module jbd
Looking for driver for device mapper/mpath0p2
Found DM device mapper/mpath0p2
Looking for deps of module scsi:t-0x00: scsi_mod sd_mod
Looking for deps of module scsi_mod
Looking for deps of module sd_mod: scsi_mod
Looking for deps of module pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00: scsi_mod scsi_tgt scsi_transport_fc qla2xxx  <<< no scsi_dh_rdac
Looking for deps of module scsi_tgt: scsi_mod
Looking for deps of module scsi_transport_fc: scsi_mod scsi_tgt
Looking for deps of module qla2xxx: scsi_mod scsi_tgt scsi_transport_fc

..
..
Adding module scsi_mod
Adding module sd_mod
Adding module scsi_tgt
Adding module scsi_transport_fc
Adding module qla2xxx
Adding module shpchp
Adding module scsi_dh
Adding module scsi_dh_rdac  << This needs to be loaded before qla2xxx

The current work around for this is using preload option:

mkinitrd -f -v  /boot/initrd-2.6.24.7-101.el5rt.img 2.6.24.7-101.el5rt --allow-missing --preload scsi_dh_rdac

This lines up scsi_dh_rdac perfectly.

It may not be elegant solution to customers.
Need to check with Redhat on their plans on fixing these issues.
Comment 4 IBM Bug Proxy 2009-03-30 16:50:36 EDT
------- Comment From jvrao@us.ibm.com 2009-03-30 16:47 EDT-------
Redhat bz link:

https://bugzilla.redhat.com/show_bug.cgi?id=491323
Comment 5 IBM Bug Proxy 2009-05-11 19:20:45 EDT
------- Comment From jvrao@us.ibm.com 2009-05-11 19:11 EDT-------
(In reply to comment #16)
> Redhat bz link:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=491323
>

Same issue is observed on RHEL 5.2 also.

+ echo -n 'Looking for deps of module pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00'
Looking for deps of module pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00++ modprobe --set-version 2.6.24.7-108ibmrt2.1.03 --show-depends pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00
++ awk '/^insmod / { print gensub(".*/","","g",$2) }'
++ read foo
++ '[' scsi_mod '!=' pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00 ']'
++ echo -n 'scsi_mod '
++ read foo
++ '[' scsi_tgt '!=' pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00 ']'
++ echo -n 'scsi_tgt '
++ read foo
++ '[' scsi_transport_fc '!=' pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00 ']'
++ echo -n 'scsi_transport_fc '
++ read foo
++ '[' qla2xxx '!=' pci:v00001077d00002422sv00001077sd0000014Dbc0Csc04i00 ']'
++ echo -n 'qla2xxx '
++ read foo
+ deps='scsi_mod scsi_tgt scsi_transport_fc qla2xxx '

No scsi_dh_rdac.
Comment 6 IBM Bug Proxy 2009-07-28 21:41:39 EDT
------- Comment From jvrao@us.ibm.com 2009-07-28 21:30 EDT-------
Clark gave me the mkinitrd RPM but need more dependent RPMS.
Informed to Clark..and waiting.
Comment 7 Clark Williams 2009-09-14 16:18:28 EDT

*** This bug has been marked as a duplicate of bug 515326 ***

Note You need to log in before you can comment on or make changes to this bug.