Bug 1347548

(In reply to zguo from comment #0)
...
> While installing RHEL-7.3-20160615.n.0 via PXE over IPoIB
> Message from console output:
> [FAILED] Failed to start Initialize the iWAR...niBand/RDMA stack in the
> kernel.

Please try generating an install image using this updated rdma test package:

http://hp-dl360pgen8-07.khw.lab.eng.bos.redhat.com/~jwilson/misc/rdma/rdma-7.3_4.7_rc2-3.el7.noarch.rpm

Basic gist of the change in the updated test package linked in the prior comment:

diff --git a/rdma.modules-setup.sh b/rdma.modules-setup.sh
index ca9322d..15f1627 100644
--- a/rdma.modules-setup.sh
+++ b/rdma.modules-setup.sh
@@ -26,4 +26,5 @@ install() {

 installkernel() {
        hostonly='' instmods =drivers/infiniband =drivers/net/ethernet/mellanox =drivers/net/ethernet/chelsio =drivers/net/ethernet/cisco =drive
+       hostonly='' instmods target_core_mod crc-t10dif crct10dif_common
 }

(In reply to Jarod Wilson from comment #4)
> Basic gist of the change in the updated test package linked in the prior
> comment:
> 

At least, it is not a completed solution for this issue. The kernel/target drivers will be removed via /usr/share/lorax/runtime-cleanup.tmpl.

# vim /usr/share/lorax/runtime-cleanup.tmpl
 83 ## clean up kernel modules
 84 removekmod sound drivers/media drivers/hwmon \
 85            net/atm net/bluetooth net/sched net/sctp \
 86            net/rds net/l2tp net/decnet net/netfilter net/ipv4 net/ipv6 \
 87            drivers/watchdog drivers/target drivers/rtc drivers/input/joystick \
 88            drivers/bluetooth drivers/hid drivers/edac \
 89            drivers/usb/serial drivers/usb/host drivers/usb/misc \
 90            fs/ocfs2 fs/ceph fs/nfsd fs/ubifs fs/nilfs2 \
 91            arch/x86/kvm

https://github.com/rhinstaller/lorax/issues/133

Managed to start rdma service.

[root@ib2-qa-03 ~]# diff -Nurp /usr/share/lorax/runtime-cleanup.tmpl.bak /usr/share/lorax/runtime-cleanup.tmpl
--- /usr/share/lorax/runtime-cleanup.tmpl.bak	2016-06-19 10:07:08.540835900 -0400
+++ /usr/share/lorax/runtime-cleanup.tmpl	2016-06-19 10:07:36.257351230 -0400
@@ -84,7 +84,7 @@ remove /usr/share/icons/*/icon-theme.cac
 removekmod sound drivers/media drivers/hwmon \
            net/atm net/bluetooth net/sched net/sctp \
            net/rds net/l2tp net/decnet net/netfilter net/ipv4 net/ipv6 \
-           drivers/watchdog drivers/target drivers/rtc drivers/input/joystick \
+           drivers/watchdog drivers/rtc drivers/input/joystick \
            drivers/bluetooth drivers/hid drivers/edac \
            drivers/usb/serial drivers/usb/host drivers/usb/misc \
            fs/ocfs2 fs/ceph fs/nfsd fs/ubifs fs/nilfs2 \
[root@ib2-qa-03 ~]# 


[anaconda root@localhost ~]# cat /lib/dracut/modules.d/05rdma/module-setup.sh 
#!/bin/bash

check() {
	[ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0
	[ -n "$hostonly" ] && return 255
	return 0
}

depends() {
	return 0
}

install() {
	inst /etc/rdma/rdma.conf
	inst /etc/rdma/mlx4.conf
	inst /etc/rdma/sriov-vfs
	inst /usr/libexec/rdma-init-kernel
	inst /usr/libexec/rdma-fixup-mtrr.awk
	inst /usr/libexec/mlx4-setup.sh
	inst /usr/libexec/rdma-set-sriov-vf
	inst /usr/lib/modprobe.d/libmlx4.conf
	inst_multiple lspci setpci awk sleep
	inst_multiple -o /etc/modprobe.d/mlx4.conf
	inst_rules 98-rdma.rules 70-persistent-ipoib.rules
}

installkernel() {
	hostonly='' instmods =drivers/infiniband =drivers/net/ethernet/mellanox =drivers/net/ethernet/chelsio =drivers/net/ethernet/cisco =drivers/net/ethernet/emulex =drivers/target
	hostonly='' instmods crc-t10dif crct10dif_common
}
[anaconda root@localhost ~]# 


[anaconda root@localhost ~]# systemctl status rdma.service -l
● rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
   Loaded: loaded (/usr/lib/systemd/system/rdma.service; disabled; vendor preset: disabled)
   Active: active (exited) since Sun 2016-06-19 14:32:37 UTC; 1min 35s ago
     Docs: file:/etc/rdma/rdma.conf
  Process: 2046 ExecStart=/usr/libexec/rdma-init-kernel (code=exited, status=0/SUCCESS)
 Main PID: 2046 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/rdma.service

Jun 19 14:32:36 localhost systemd[1]: Starting Initialize the iWARP/InfiniBand/RDMA stack in the kernel...
Jun 19 14:32:37 localhost systemd[1]: Started Initialize the iWARP/InfiniBand/RDMA stack in the kernel.

(In reply to Honggang LI from comment #7)
> Managed to start rdma service.
> 
> [root@ib2-qa-03 ~]# diff -Nurp /usr/share/lorax/runtime-cleanup.tmpl.bak
> /usr/share/lorax/runtime-cleanup.tmpl
> --- /usr/share/lorax/runtime-cleanup.tmpl.bak	2016-06-19 10:07:08.540835900
> -0400
> +++ /usr/share/lorax/runtime-cleanup.tmpl	2016-06-19 10:07:36.257351230 -0400
> @@ -84,7 +84,7 @@ remove /usr/share/icons/*/icon-theme.cac
>  removekmod sound drivers/media drivers/hwmon \
>             net/atm net/bluetooth net/sched net/sctp \
>             net/rds net/l2tp net/decnet net/netfilter net/ipv4 net/ipv6 \
> -           drivers/watchdog drivers/target drivers/rtc
> drivers/input/joystick \
> +           drivers/watchdog drivers/rtc drivers/input/joystick \
>             drivers/bluetooth drivers/hid drivers/edac \
>             drivers/usb/serial drivers/usb/host drivers/usb/misc \
>             fs/ocfs2 fs/ceph fs/nfsd fs/ubifs fs/nilfs2 \

Okay, so we need another bug for another component here.


> [anaconda root@localhost ~]# cat
> /lib/dracut/modules.d/05rdma/module-setup.sh 
> #!/bin/bash
> 
> check() {
> 	[ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0
...
> installkernel() {
> 	hostonly='' instmods =drivers/infiniband =drivers/net/ethernet/mellanox
> =drivers/net/ethernet/chelsio =drivers/net/ethernet/cisco
> =drivers/net/ethernet/emulex =drivers/target
> 	hostonly='' instmods crc-t10dif crct10dif_common

This wasn't from the test package I requested to be tested. The -d in check() is wrong (it should be -c), and I included only what I believe was the only target module needed, target_core_mod. But good that we're getting close. Is anything else under drivers/target/ actually required here?

(In reply to Jarod Wilson from comment #8)
> (In reply to Honggang LI from comment #7)

> Okay, so we need another bug for another component here.

zguo, please file a lorax bug and set it as a blocker of this bug. 

> 
> This wasn't from the test package I requested to be tested. 

No, it is not your test package. I tried it. Failed to load ib_srpt module because of missing target modules. And lorax runtime-cleanup.tmpl file removes entire drivers/target directory. So, I dropped your and overwrite the old rdma package with my hacked one.

> The -d in
> check() is wrong (it should be -c), 

Yes, you are right. That is why I force lorax to install rdma with 'lorax -i rdma blabla ....'.

> and I included only what I believe was
> the only target module needed, target_core_mod. But good that we're getting
> close. Is anything else under drivers/target/ actually required here?

I'd suggest to package up drivers/target. When pxeboot/initrd.img and LiveOS/squashfs.img have different modules, it is really confusing for debugging.

(In reply to Honggang LI from comment #9)
> (In reply to Jarod Wilson from comment #8)
> > (In reply to Honggang LI from comment #7)
> 
> > Okay, so we need another bug for another component here.
> 
> zguo, please file a lorax bug and set it as a blocker of this bug. 

https://bugzilla.redhat.com/show_bug.cgi?id=1348381

> > 
> > This wasn't from the test package I requested to be tested. 
> 
> No, it is not your test package. I tried it. Failed to load ib_srpt module
> because of missing target modules. And lorax runtime-cleanup.tmpl file
> removes entire drivers/target directory. So, I dropped your and overwrite
> the old rdma package with my hacked one.
> 
> > The -d in
> > check() is wrong (it should be -c), 
> 
> Yes, you are right. That is why I force lorax to install rdma with 'lorax -i
> rdma blabla ....'.
> 
> > and I included only what I believe was
> > the only target module needed, target_core_mod. But good that we're getting
> > close. Is anything else under drivers/target/ actually required here?
> 
> I'd suggest to package up drivers/target. When pxeboot/initrd.img and
> LiveOS/squashfs.img have different modules, it is really confusing for
> debugging.

(In reply to Honggang LI from comment #9)
> (In reply to Jarod Wilson from comment #8)
...
> > This wasn't from the test package I requested to be tested. 
> 
> No, it is not your test package. I tried it. Failed to load ib_srpt module
> because of missing target modules. And lorax runtime-cleanup.tmpl file
> removes entire drivers/target directory. So, I dropped your and overwrite
> the old rdma package with my hacked one.

That was the feedback I was looking for when asking to test the package I created. :)

I'm not quite sure why that change didn't work, based on:

$ modinfo ib_srpt |grep target
description:    InfiniBand SCSI RDMA Protocol target v2.0.0 (2011-02-14)
depends:        ib_core,target_core_mod,ib_cm,ib_mad

All of those should have been present and satisfied.

> > and I included only what I believe was
> > the only target module needed, target_core_mod. But good that we're getting
> > close. Is anything else under drivers/target/ actually required here?
> 
> I'd suggest to package up drivers/target. When pxeboot/initrd.img and
> LiveOS/squashfs.img have different modules, it is really confusing for
> debugging.

Despite the head-scratching about ib_srpt failing, since iser looks to need more of drivers/target/* anyway, and it's working for you, I'll add all of drivers/target/*.

(In reply to Jarod Wilson from comment #11)

> Despite the head-scratching about ib_srpt failing, since iser looks to need
> more of drivers/target/* anyway, and it's working for you, I'll add all of
> drivers/target/*.

Thank you!

Issue is gone with latest rdma package. 

Check the rdma service after installation:

[root@localhost log]# service rdma status
Redirecting to /bin/systemctl status  rdma.service
● rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
   Loaded: loaded (/usr/lib/systemd/system/rdma.service; disabled; vendor preset: disabled)
   Active: active (exited) since Wed 2016-07-27 05:24:15 EDT; 3min 32s ago
     Docs: file:/etc/rdma/rdma.conf
  Process: 699 ExecStart=/usr/libexec/rdma-init-kernel (code=exited, status=0/SUCCESS)
 Main PID: 699 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/rdma.service

Jul 27 05:24:11 localhost.localdomain systemd[1]: Starting Initialize the iWA...
Jul 27 05:24:15 localhost.localdomain systemd[1]: Started Initialize the iWAR...
Hint: Some lines were ellipsized, use -l to show in full.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2309.html

Hi Andre,

Any plans to have this fix for RHEL 7.2 as well?
What is the status for RHEL 7.1?

Thanks,
Noa

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days