Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1346113

Summary: RHEL-7.x/images/pxeboot/initrd.img should package up RDMA drivers to support PXE over IPoiB
Product: Red Hat Enterprise Linux 7 Reporter: zguo <zguo>
Component: rdmaAssignee: Jarod Wilson <jarod>
Status: CLOSED ERRATA QA Contact: zguo <zguo>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 7.3CC: abeausol, ddutile, dhoward, dledford, harald, honli, infiniband-qe, jarod, jkachuck, mbanas, noas, shanim, zguo
Target Milestone: rcKeywords: Reopened, TestBlocker
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Rebase: Bug Fixes and Enhancements
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 03:26:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1093978, 1171868, 1213541, 1255429, 1274397, 1289204, 1313485    
Attachments:
Description Flags
rdma04 rdsosreport
none
rdma04 rdsosreport
none
rdma04 rdsosreport with rd.debug enabled
none
PXE over IPoIB log none

Description zguo 2016-06-14 02:59:04 UTC
Description of problem:


Version-Release number of selected component (if applicable):

kernel-3.10.0-433.el7.rdma4dot5el7433v1.x86_64.rpm

How reproducible:

Always.

Steps to Reproduce:
1. PXE server(rdma03.rhts.eng.pek2.redhat.com) configuration.

[root@ib2-qa-03 pxelinux.cfg]#  cat /etc/dhcp/dhcpd.conf 
#
# DHCP Server Configuration file.
#   see /usr/share/doc/dhcp*/dhcpd.conf.example
#   see dhcpd.conf(5) man page
#
allow booting;
allow bootp;
ddns-update-style interim;
ignore client-updates;

DHCPDARGS=mlx4_ib1;
ddns-update-style none;

subnet 172.31.2.0 netmask 255.255.255.0 {
        next-server 172.31.2.3;
	filename "pxelinux/pxelinux.0";
        range dynamic-bootp 172.31.2.200 172.31.2.201;
        always-broadcast on;
==
[root@ib2-qa-03 pxelinux.cfg]# cat /var/lib/tftpboot/pxelinux/pxelinux.cfg/default
default rhel73
prompt 1
timeout 600

label  rhel7
  menu label ^Install or upgrade an existing system
  menu default
  kernel RHEL-7.3-20160610.n.0/vmlinuz
  ipappend 2
  append initrd=RHEL-7.3-20160610.n.0/initrd.img inst.repo=http://172.31.2.3/RHEL-7.3-20160610.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text

label  rhel73
  menu label ^Install or upgrade an existing system
  kernel rdma4dot5el7433v1/vmlinuz-3.10.0-433.el7.rdma4dot5el7433v1.x86_64
  ipappend 2
  append initrd=rdma4dot5el7433v1/test.img inst.repo=http://172.31.2.3/RHEL-7.3-20160610.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text
  #append initrd=rdma4dot5el7433v1/initramfs-3.10.0-433.el7.rdma4dot5el7433v1.x86_64.img inst.repo=http://172.31.2.3/RHEL-7.3-20160610.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text root=/dev/sda


#root=live:http://172.31.2.3/RHEL-7.3-20160610.n.0/LiveOS/squashfs.img
==

2. Boot rdma04.rhts.eng.pek2.redhat.com from IPoIB
3.

Actual results:
Welcome to Red Hat Enterprise Linux Server 7.3 Beta (Maipo) dracut-033-360.el7_2 (Initramfs)!

[   11.807034] systemd[1]: Set hostname to <localhost.localdomain>.
[  OK  ] Reached target Timers.
[   11.832957] systemd[1]: Reached target Timers.
[   11.834575] systemd[1]: Starting Timers.
[  OK  ] Reached target Local File Systems.
[   11.837932] systemd[1]: Reached target Local File Systems.
[   11.839894] systemd[1]: Starting Local F[  OK  ] Created slice -.slice.
[   12.343925] systemd[1]: Created slice -.slice.
[   12.345507] systemd[1]: Starting -.slice.
[  OK  ] Created slice System Slice.
[   12.348920] systemd[1]: Created slice System Slice.
[   12.350589] systemd[1]: Starting System Slice.
[  OK  ] Listening on udev Kernel Socket.
[   12.353923] systemd[1]: Listening on udev Kernel Socket.
[   12.355795] systemd[1]: Starting udev Kernel Socket.
[  OK  ] Listening on Journal Socket.
[   12.359928] systemd[1]: Listening on Journal Socket.
[   12.361615] systemd[1]: Starting Journal Socket.
[   12.363380] systemd[1]: Started Load Kernel Modules.
[   12.365603] systemd[1]: Starting Setup Virtual Console...
         Starting Setup Virtual Console...
[   12.370286] systemd[1]: Starting Create list of required static device nodes for the current kernel...
         Starting Create list of required st... nodes for the current kernel...
[   12.382240] systemd[1]: Starting Apply Kernel Variables...
         Starting Apply Kernel Variables...
[   12.386934] systemd[1]: Started dracut ask for additional cmdline parameters.
[   12.389886] systemd[1]: Starting dracut cmdline hook...
         Starting dracut cmdline hook...
[  OK  [   12.486002] dracut: FATAL: No or empty root= argument
[   12.815591] dracut: Refusing to continue
] Reached target Swap.
[   12.835938] systemd[1]: Reached target Swap.
[   12.837760] systemd[1]: Starting Swap.
[  OK  ] Listening on udev Control Socket.
[   12.840912] systemd[1]: Listening on udev Control Socket.
[   12.842920] systemd[1]: Starting udev Control Socket.
[  OK  ] Reached target Sockets.
[   12.846910] systemd[1]: Reached target Sockets.
[   12.848504] systemd[1]: Starting Sockets.
[   12.850265] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[  OK  ] Reached target Slices.
[   12.855877] systemd[1]: Reached target Slices.
[   12.857504] systemd[1]: Starting Slices.
[  OK  ] Started Setup Virtual Console.
[   12.860878] systemd[1]: Started Setup Virtual Console.
[  OK  ] Started Create list of required sta...ce nodes for the current kernel.
[   12.864879] systemd[1]: Started Create list of required static device nodes for the current kernel.
[  OK  ] Started Apply Kernel Variables.
[   12.869877] systemd[1]: Started Apply Kernel [   11.353921] dracut-cmdline[216]: /lib/dracut-lib.sh: line 130: read: read error: 0: Is a directory
[  OK  ] Started Journal Service.
[   13.375901] systemd[1]: Started Journal Service.
[   11.846385] dracut-cmdline[216]: /lib/dracut-lib.sh: line 130: read: read error: 0: Is a directory
[   11.852227] dracut-cmdline[216]: /lib/dracut-lib.sh: line 130: read: read error: 0: Is a directory
[   11.854332] dracut-cmdline[216]: /lib/dracut-lib.sh: line 130: read: read error: 0: Is a directory
[   11.856172] dracut-cmdline[216]: /lib/dracut-lib.sh: line 130[   13.384335] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
: read: read error: rrrrrrrrrrrr[   13.421500] systemd-journald[320]: Received SIGTERM from PID 1 (systemd-shutdow).
[   13.424571] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
[   13.431490] systemd-shutdown[1]: Unmounting file systems.
[   13.433466] systemd-shutdown[1]: All filesystems unmounted.
[   13.435422] systemd-shutdown[1]: Deactivating swaps.
[   13.437269] systemd-shutdown[1]: All swaps deactivated.
[   13.439161] systemd-shutdown[1]: Detaching loop devices.
[   13.440990] systemd-shutdown[1]: All loop devices detached.
[   13.442912] systemd-shutdown[1]: Detaching DM devices.
[ ystemd-shutdown[1]: All DM devices detached.
[   13.847018] systemd-shutdown[1]: Halting system.
[   13.850668] System halted.

Expected results:

No this error.

Additional info:

Comment 2 David Shea 2016-06-14 13:54:46 UTC
Please boot with rd.debug=1 and attach the rdsosreport.txt from the failing system. It looks like you have a directory in /etc/cmdline.d.

Comment 3 Honggang LI 2016-06-14 15:10:43 UTC
(In reply to David Shea from comment #2)
> Please boot with rd.debug=1 and attach the rdsosreport.txt from the failing
> system. It looks like you have a directory in /etc/cmdline.d.

Please ignore it. The console log in comment #0 had been generated with a scratch img file "test.img" created by me.

https://bugzilla.redhat.com/show_bug.cgi?id=785815#c5

===========================================================
[root@ib2-qa-03 rdma4dot5el7433v1]# cat anaconda.conf 
rd.live.dir=/images

[root@ib2-qa-03 rdma4dot5el7433v1]#  lsinitrd test.img | grep anaconda.conf
Arguments: --include 'anaconda.conf' '/etc/cmdline.d/anaconda.conf'
drwxr-xr-x   2 root     root            0 Jun 13 09:58 etc/cmdline.d/anaconda.conf
[root@ib2-qa-03 rdma4dot5el7433v1]# 

===========================================================

I will replace test.img with initramfs-3.10.0-433.el7.rdma4dot5el7433v1.x86_64.img. console log will be uploaded ASAP.

Comment 4 Honggang LI 2016-06-14 15:11:34 UTC
[root@ib2-qa-03 pxelinux.cfg]# cat default
default rhel73
prompt 1
timeout 600

label  rhel7
  menu label ^Install or upgrade an existing system
  menu default
  kernel RHEL-7.3-20160610.n.0/vmlinuz
  ipappend 2
  append initrd=RHEL-7.3-20160610.n.0/initrd.img inst.repo=http://172.31.2.3/RHEL-7.3-20160610.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text

label  rhel73
  menu label ^Install or upgrade an existing system
  kernel rdma4dot5el7433v1/vmlinuz-3.10.0-433.el7.rdma4dot5el7433v1.x86_64
  ipappend 2
  append initrd=rdma4dot5el7433v1/initramfs-3.10.0-433.el7.rdma4dot5el7433v1.x86_64.img inst.repo=http://172.31.2.3/RHEL-7.3-20160610.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.debug=1 rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text

Comment 5 Honggang LI 2016-06-14 15:16:57 UTC
Created attachment 1167920 [details]
rdma04 rdsosreport

Comment 6 Honggang LI 2016-06-14 15:18:08 UTC
(In reply to David Shea from comment #2)
> Please boot with rd.debug=1 and attach the rdsosreport.txt

Hello, David
 rdsosreport.txt had been uploaded, please check.

thanks

Comment 7 David Shea 2016-06-14 15:48:32 UTC
(In reply to Honggang LI from comment #6)
> (In reply to David Shea from comment #2)
> > Please boot with rd.debug=1 and attach the rdsosreport.txt
> 
> Hello, David
>  rdsosreport.txt had been uploaded, please check.
> 
> thanks

That just has "The server is not powered on.  The Virtual Serial Port is not available."

Comment 8 Honggang LI 2016-06-14 22:54:03 UTC
(In reply to David Shea from comment #7)

> That just has "The server is not powered on.  The Virtual Serial Port is not
> available."

This serial console message is nothing to do with the installation failure.


"The server is not powered on." means the power of rdma04 had been turn off. As system is hang on (without rd.debug=1), hard power cycle is the only way to bring rdma04 back.

https://beaker.engineering.redhat.com/view/rdma04.rhts.eng.pek2.redhat.com#power

The PDU of rdma04 can be accessed remotely through above URL.

Comment 9 David Shea 2016-06-15 13:37:10 UTC
(In reply to Honggang LI from comment #4)
> initrd=rdma4dot5el7433v1/initramfs-3.10.0-433.el7.rdma4dot5el7433v1.x86_64.
> img

This initrd does not contain the anaconda dracut modules. I do not know how it was generated, but it cannot be used to start anaconda.

Comment 10 Honggang LI 2016-06-15 14:08:56 UTC
[root@ib2-qa-03 pxelinux.cfg]# cat default
default rhel7
prompt 1
timeout 600

label  rhel7
  menu label ^Install or upgrade an existing system
  menu default
  kernel RHEL-7.3-20160610.n.0/vmlinuz
  ipappend 2
  append initrd=RHEL-7.3-20160610.n.0/initrd.img inst.repo=http://172.31.2.3/RHEL-7.3-20160610.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text

------
RHEL-7.3-20160610.n.0/images/pxeboot/initrd.img
RHEL-7.3-20160610.n.0/images/pxeboot/vmlinuz

Comment 11 Honggang LI 2016-06-15 14:13:28 UTC
Created attachment 1168397 [details]
rdma04 rdsosreport

console log and rdsosreport for comment #10.

Comment 12 Honggang LI 2016-06-15 14:19:35 UTC
It seems the default initrd.img or vmlinuz file dose not include all RDMA drivers. At least, ib_ipoib.ko, mlx4_ib.ko, mlx4_core.ko mlx4_en.ko are missing.

dracut:/# ls /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/
core  ulp
dracut:/# ls /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/core
ib_addr.ko  ib_cm.ko  ib_core.ko  ib_mad.ko  ib_sa.ko
dracut:/# ls /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/ulp/
srp
p/acut:/# ls /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/ulp/srp
ib_srp.ko
dracut:/#

Comment 13 Honggang LI 2016-06-15 15:19:50 UTC
Created attachment 1168421 [details]
rdma04 rdsosreport with rd.debug enabled

Comment 14 David Shea 2016-06-15 15:42:39 UTC
(In reply to Honggang LI from comment #12)
> It seems the default initrd.img or vmlinuz file dose not include all RDMA
> drivers. At least, ib_ipoib.ko, mlx4_ib.ko, mlx4_core.ko mlx4_en.ko are
> missing.
> 
> dracut:/# ls /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/
> core  ulp
> dracut:/# ls
> /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/core
> ib_addr.ko  ib_cm.ko  ib_core.ko  ib_mad.ko  ib_sa.ko
> dracut:/# ls
> /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/ulp/
> srp
> p/acut:/# ls
> /lib/modules/3.10.0-429.el7.x86_64/kernel/drivers/infiniband/ulp/srp
> ib_srp.ko
> dracut:/#


Adding the inifiniband and mellanox drivers to the initrd is handled by /usr/lib/dracut/modules.d/05rdma/module-setup.sh, which is owned by rdma.

Comment 15 Don Dutile (Red Hat) 2016-06-15 15:53:05 UTC
Jarod,
isn't this a dupe of existing bz of failed ipoib install due to need to update rdma pkg, which I thought I saw you state was awaiting another pkg update to complete.
if so, let's close this as dupe & point to in-progress bz. thanks.

Comment 16 Jarod Wilson 2016-06-15 17:06:01 UTC
(In reply to David Shea from comment #14)
...
> Adding the inifiniband and mellanox drivers to the initrd is handled by
> /usr/lib/dracut/modules.d/05rdma/module-setup.sh, which is owned by rdma.

As noted in comment #21 of bug 1303589, that piece is 100% functional. The issue appears to be that this file didn't exist (wasn't installed) or didn't get sourced at anaconda boot initrd image creation time. Dracut definitely pulls in all the required modules if I have the rdma package installed. It's just not happening for the anaconda initrd, for reasons which are beyond my knowledge right now, but so far as I know, aren't because of anything the rdma package is doing wrong.

(In reply to Don Dutile from comment #15)
> Jarod,
> isn't this a dupe of existing bz of failed ipoib install due to need to
> update rdma pkg, which I thought I saw you state was awaiting another pkg
> update to complete.
> if so, let's close this as dupe & point to in-progress bz. thanks.

There seem to be two different issues. One is the rdma package having a proper line to get the modules into a dracut initrd. The other is getting them into the Anaconda initrd.

Comment 17 David Shea 2016-06-15 17:12:33 UTC
(In reply to Jarod Wilson from comment #16)
> (In reply to David Shea from comment #14)
> ...
> > Adding the inifiniband and mellanox drivers to the initrd is handled by
> > /usr/lib/dracut/modules.d/05rdma/module-setup.sh, which is owned by rdma.
> 
> As noted in comment #21 of bug 1303589, that piece is 100% functional. The
> issue appears to be that this file didn't exist (wasn't installed) or didn't
> get sourced at anaconda boot initrd image creation time.

What version of rdma is this fix in?

Comment 18 Jarod Wilson 2016-06-15 17:44:41 UTC
(In reply to David Shea from comment #17)
> What version of rdma is this fix in?

rdma-7.3_4.7_rc2-1.el7, but see bug 1303589 for some additional details. It seems forcing dracut to add the dracut rdma module is required to get the infiniband drivers all included in anaconda mode. i.e.:

dracut --xz --add 'anaconda pollcdrom rdma' foo.img

Comment 19 David Shea 2016-06-15 18:25:10 UTC
(In reply to Jarod Wilson from comment #18)
> (In reply to David Shea from comment #17)
> > What version of rdma is this fix in?
> 
> rdma-7.3_4.7_rc2-1.el7, but see bug 1303589 for some additional details. It
> seems forcing dracut to add the dracut rdma module is required to get the
> infiniband drivers all included in anaconda mode. i.e.:
> 
> dracut --xz --add 'anaconda pollcdrom rdma' foo.img

The module is being excluded because you're specifying that it be excluded. 05rdma/module-setup.sh in rdma-7.3_4.7_rc2-1.el7.noarch contains the following:

check() {
        [ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0 || return 255
        return 0
}

The first line specifies that if hostonly is set, and /sys/class/infiniband_verbs/uverbs0 exists, the module should be included, otherwise it should be excluded. The second line is never reached.

There is no "anaconda mode", and anaconda is not used to generate the installation media. If the rdma module should be included in --no-hostonly initrds, please return 0 when $hostonly is not set. If it should continue to be excluded from --no-hostonly initrds but should be treated as a special case for installation media, please open a bug against lorax.

Comment 20 Doug Ledford 2016-06-15 20:15:15 UTC
Got it, I see the issue now.  The test should be:

[ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0
[ -n "$hostonly" ] && return 255
return 0

The net effect being hostonly and rdma hardware == install, otherwise if hostonly == don't install, not hostonly == install.  Thanks for the catch.

Comment 21 Jarod Wilson 2016-06-15 20:21:21 UTC
(In reply to Doug Ledford from comment #20)
> Got it, I see the issue now.  The test should be:
> 
> [ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0
> [ -n "$hostonly" ] && return 255
> return 0
> 
> The net effect being hostonly and rdma hardware == install, otherwise if
> hostonly == don't install, not hostonly == install.  Thanks for the catch.

Whee, so much fun inheriting black magic voodoo packages. I'll make that change. Thanks David, sorry for being clueless about this. ;)

Comment 22 Honggang LI 2016-06-16 01:30:51 UTC
(In reply to Doug Ledford from comment #20)
> Got it, I see the issue now.  The test should be:
> 
> [ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0
> [ -n "$hostonly" ] && return 255
> return 0
> 

It works for me.

Comment 23 Honggang LI 2016-06-16 01:32:50 UTC
Jarod

 You also have to package up all modules depended on by any RDMA drivers. For example, target_core_mod is needed by ib_srpt.

[  983.904220] intel_rapl: Found RAPL domain package
[  983.904221] intel_rapl: Found RAPL domain core
[  984.954994] ib_srpt: Unknown symbol transport_init_session (err 0)
[  984.955012] ib_srpt: Unknown symbol core_tpg_add_initiator_node_acl (err 0)
[  984.955022] ib_srpt: Unknown symbol target_execute_cmd (err 0)
[  984.955037] ib_srpt: Unknown symbol target_submit_tmr (err 0)
[  984.955046] ib_srpt: Unknown symbol target_wait_for_sess_cmds (err 0)
[  984.955056] ib_srpt: Unknown symbol target_fabric_configfs_register (err 0)
[  984.955093] ib_srpt: Unknown symbol transport_register_session (err 0)
[  984.955103] ib_srpt: Unknown symbol transport_send_check_condition_and_sense (err 0)
[  984.955120] ib_srpt: Unknown symbol transport_check_aborted_status (err 0)
[  984.955166] ib_srpt: Unknown symbol target_fabric_configfs_init (err 0)
[  984.955174] ib_srpt: Unknown symbol target_fabric_configfs_free (err 0)
[  984.955184] ib_srpt: Unknown symbol transport_deregister_session_configfs (err 0)
[  984.955240] ib_srpt: Unknown symbol core_tpg_del_initiator_node_acl (err 0)
[  984.955276] ib_srpt: Unknown symbol target_submit_cmd (err 0)
[  984.955288] ib_srpt: Unknown symbol core_tpg_deregister (err 0)
[  984.955296] ib_srpt: Unknown symbol target_fabric_configfs_deregister (err 0)
[  984.955303] ib_srpt: Unknown symbol target_sess_cmd_list_set_waiting (err 0)
[  984.955330] ib_srpt: Unknown symbol transport_generic_free_cmd (err 0)
[  984.955343] ib_srpt: Unknown symbol core_tpg_register (err 0)
[  984.955352] ib_srpt: Unknown symbol target_put_sess_cmd (err 0)
[  984.955360] ib_srpt: Unknown symbol transport_deregister_session (err 0)
[  985.010187] ib_isert: Unknown symbol iscsit_handle_task_mgt_cmd (err 0)
[  985.010205] ib_isert: Unknown symbol target_execute_cmd (err 0)
[  985.010229] ib_isert: Unknown symbol target_wait_for_sess_cmds (err 0)
[  985.010241] ib_isert: Unknown symbol iscsit_register_transport (err 0)
[  985.010249] ib_isert: Unknown symbol iscsit_process_nop_out (err 0)
[  985.010281] ib_isert: Unknown symbol iscsit_build_rsp_pdu (err 0)
[  985.010290] ib_isert: Unknown symbol transport_send_check_condition_and_sense (err 0)
[  985.010297] ib_isert: Unknown symbol iscsit_build_text_rsp (err 0)
[  985.010305] ib_isert: Unknown symbol iscsit_find_cmd_from_itt (err 0)
[  985.010318] ib_isert: Unknown symbol iscsit_check_dataout_payload (err 0)
[  985.010344] ib_isert: Unknown symbol iscsit_tmr_post_handler (err 0)
[  985.010358] ib_isert: Unknown symbol iscsit_build_reject (err 0)
[  985.010366] ib_isert: Unknown symbol iscsit_logout_post_handler (err 0)
[  985.010382] ib_isert: Unknown symbol iscsit_sequence_cmd (err 0)
[  985.010392] ib_isert: Unknown symbol iscsit_setup_nop_out (err 0)
[  985.010399] ib_isert: Unknown symbol iscsit_process_text_cmd (err 0)
[  985.010417] ib_isert: Unknown symbol iscsit_release_cmd (err 0)
[  985.010429] ib_isert: Unknown symbol iscsit_allocate_cmd (err 0)
[  985.010438] ib_isert: Unknown symbol iscsit_cause_connection_reinstatement (err 0)
[  985.010447] ib_isert: Unknown symbol iscsit_setup_scsi_cmd (err 0)
[  985.010455] ib_isert: Unknown symbol iscsit_setup_text_cmd (err 0)
[  985.010472] ib_isert: Unknown symbol iscsit_build_nopin_rsp (err 0)
[  985.010495] ib_isert: Unknown symbol iscsit_check_dataout_hdr (err 0)
[  985.010516] ib_isert: Unknown symbol target_sess_cmd_list_set_waiting (err 0)
[  985.010524] ib_isert: Unknown symbol iscsit_set_unsoliticed_dataout (err 0)
[  985.010536] ib_isert: Unknown symbol iscsit_handle_logout_cmd (err 0)
[  985.010559] ib_isert: Unknown symbol transport_generic_free_cmd (err 0)
[  985.010577] ib_isert: Unknown symbol iscsit_stop_dataout_timer (err 0)
[  985.010585] ib_isert: Unknown symbol target_put_sess_cmd (err 0)
[  985.010592] ib_isert: Unknown symbol iscsit_build_logout_rsp (err 0)
[  985.010600] ib_isert: Unknown symbol iscsit_unregister_transport (err 0)
[  985.010608] ib_isert: Unknown symbol iscsit_build_task_mgt_rsp (err 0)
[  985.010616] ib_isert: Unknown symbol iscsit_process_scsi_cmd (err 0)
[  985.057016] RPC: Registered rdma transport module.

Comment 24 Honggang LI 2016-06-16 06:45:17 UTC
I mananaged to install RHEL-7.3-20160615.n.0 via PXE over IPoIB.
The default RHEL-7.x/images/pxeboot/{initrd.img,vmlinuz} do not
support PXE over IPoIB (and SRPT/ISER). We have to generate
initrd.img for ourself.

1) Create a new rdma package with patch in comment #20.
[root@ib2-qa-03 SPECS]# cat bz1346113.patch 
--- /usr/lib/dracut/modules.d/05rdma/module-setup.sh.old	2016-06-15 18:21:06.484598058 -0400
+++ /usr/lib/dracut/modules.d/05rdma/module-setup.sh	2016-06-15 18:22:15.056876089 -0400
@@ -1,7 +1,8 @@
 #!/bin/bash
 
 check() {
-	[ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0 || return 255
+	[ -n "$hostonly" -a -d /sys/class/infiniband_verbs/uverbs0 ] && return 0
+	[ -n "$hostonly" ] && return 255
 	return 0
 }
 
2) Create a local repo with the updated rdma package
# mkdir /var/www/html/local
# cp /root/rpmbuild/RPMS/noarch/rdma-7.3_4.7_rc2-2.el7.noarch.rpm /var/www/html/local/
# cd /var/www/html/local && createrepo -d .

3) Download RHEL-7.3-20160615.n.0 and replace old rdma package with the new one.
# mount -t nfs storage.eng.pek2.redhat.com:/pub/rhel /iso
# cp -rp /iso/nightly/RHEL-7.3-20160615.n.0/compose/Server/x86_64/os /var/www/html/RHEL-7.3-20160615.n.0
# rm -f /var/www/html/RHEL-7.3-20160615.n.0/Packages/rdma-7*
# cp /root/rpmbuild/RPMS/noarch/rdma-7.3_4.7_rc2-2.el7.noarch.rpm /var/www/html/RHEL-7.3-20160615.n.0/Packages
# rm -fr /var/www/html/RHEL-7.3-20160615.n.0/repodata
# cd /var/www/html/RHEL-7.3-20160615.n.0/ && createrepo -d .

3) Run lorax to create new initrd.img
# yum install -y lorax
# setenforce 0
# lorax  -v 7 -r 3 -p rhel73 -s http://localhost/local -s http://download.eng.pek2.redhat.com/pub/rhel/nightly/RHEL-7.3-20160615.n.0/compose/Server/x86_64/os ./results/ -s http://download.eng.pek2.redhat.com/pub/rhel/nightly/RHEL-7.3-20160615.n.0/compose/Server-optional/x86_64/os

4) Substitute pxeboot/*
# rm -f /var/www/html/RHEL-7.3-20160615.n.0/images/pxeboot/*
# cp ./result/images/pxeboot/{initrd.img,upgrade.img,mlinuz} /var/www/html/RHEL-7.3-20160615.n.0/images/pxeboot
# mkdir /var/lib/tftpboot/pxelinux/RHEL-7.3-20160615.n.0/
# cp /result/images/pxeboot/{initrd.img,upgrade.img,mlinuz} /var/lib/tftpboot/pxelinux/RHEL-7.3-20160615.n.0/

5) Create a ks.cfg file
--------------------------------
[root@ib2-qa-03 RHEL-7.3-20160615.n.0]# cat /var/www/html/RHEL-7.3-20160615.n.0/anaconda-ks.cfg 
#version=DEVEL
# Install OS instead of upgrade
install
text
# Use NFS installation media
# Firewall configuration
firewall --disabled
firstboot --disable
ignoredisk --only-use=sda
# Keyboard layouts
# old format: keyboard us
# new format:
keyboard --vckeymap=us --xlayouts=''
# System language
lang en_US.UTF-8

# Network information
network  --hostname=localhost.localdomain
# Reboot after installation
reboot
# Root password
rootpw --iscrypted $1$kaan1hKf$qNnU4hFLgAtcTalZQaf5Y/
# SELinux configuration
selinux --enforcing
# System services
services --enabled="chronyd"
# Do not configure the X Window System
skipx
# System timezone
timezone America/New_York --ntpservers=ns1.pek.redhat.com,clock01.util.phx2.redhat.com,lanczos.maths.tcd.ie
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda
autopart --type=lvm
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel

%packages
@core
chrony
kexec-tools

%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end
[root@ib2-qa-03 RHEL-7.3-20160615.n.0]# 
--------------------------------

6) create new pxe configure file
---------------------------------
[root@ib2-qa-03 RHEL-7.3-20160615.n.0]# cat /var/lib/tftpboot/pxelinux/pxelinux.cfg/default
default n15
prompt 1
timeout 600

label n15
  menu label ^nightly 15
  kernel RHEL-7.3-20160615.n.0/vmlinuz
  ipappend 2
  append initrd=RHEL-7.3-20160615.n.0/initrd.img inst.repo=http://172.31.2.3/RHEL-7.3-20160615.n.0 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.debug=0 rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib console=ttyS1,115200n81 inst.text rd.retry=10 rd.timeout=100 inst.ks=http://172.31.2.3/RHEL-7.3-20160615.n.0/anaconda-ks.cfg

---------------------------------

7) It is almost ready for PXE over IPoIB. You may need fix minor issues, such as file permissions.

Comment 25 Honggang LI 2016-06-16 07:07:14 UTC
Even PXE over IPoIB works now, there are some issues need to be fixed.

1) It can't start md service.
2) It can't start rdma service. Some rdma drivers are not loaded for reasons. As modules we need for IPoIB are loaded, so I ignored the unloaded modules.
3) The dracut will failed and drop into an emergency shell. As dracult debug is really time consuming and it is safe to ignore it. So, just run 'exit' to leave the emergency shell. anaconda will start the automatic installation. And it will finish without issue.
4) I believe PXE installation over SRPT/ISERT can be fixed as this one.

Comment 26 Honggang LI 2016-06-16 07:09:48 UTC
*** Bug 1093978 has been marked as a duplicate of this bug. ***

Comment 27 Honggang LI 2016-06-16 07:21:13 UTC
Created attachment 1168576 [details]
PXE over IPoIB log

Comment 29 Jarod Wilson 2016-06-16 18:37:41 UTC
(In reply to Honggang LI from comment #25)
> Even PXE over IPoIB works now, there are some issues need to be fixed.
> 
> 1) It can't start md service.
> 2) It can't start rdma service. Some rdma drivers are not loaded for
> reasons. As modules we need for IPoIB are loaded, so I ignored the unloaded
> modules.
> 3) The dracut will failed and drop into an emergency shell. As dracult debug
> is really time consuming and it is safe to ignore it. So, just run 'exit' to
> leave the emergency shell. anaconda will start the automatic installation.
> And it will finish without issue.
> 4) I believe PXE installation over SRPT/ISERT can be fixed as this one.

I'd assumed that dracut would do some sort of module dependency checking to pull in any requisite additional modules. Are we needing another bug then to track remaining issues with other missing rdma modules?

Comment 31 Honggang LI 2016-06-17 02:24:12 UTC
We need the attention of release eng team. Fix the rdma package is just fix part of the issue. As long as pxeboot/intrid.img file dose not have RDMA drivers, PXE over IPoIB would not work. We also need documentation for this issue.

Comment 32 zguo 2016-06-17 09:18:26 UTC
(In reply to Jarod Wilson from comment #29)
> (In reply to Honggang LI from comment #25)
> > Even PXE over IPoIB works now, there are some issues need to be fixed.
> > 
> > 1) It can't start md service.
> > 2) It can't start rdma service. Some rdma drivers are not loaded for
> > reasons. As modules we need for IPoIB are loaded, so I ignored the unloaded
> > modules.
> > 3) The dracut will failed and drop into an emergency shell. As dracult debug
> > is really time consuming and it is safe to ignore it. So, just run 'exit' to
> > leave the emergency shell. anaconda will start the automatic installation.
> > And it will finish without issue.
> > 4) I believe PXE installation over SRPT/ISERT can be fixed as this one.
> 
> I'd assumed that dracut would do some sort of module dependency checking to
> pull in any requisite additional modules. Are we needing another bug then to
> track remaining issues with other missing rdma modules?

1) bz1347622
2) bz1347548
3) bz1347545
4) bz1093972

Comment 35 Harald Hoyer 2016-07-22 13:19:14 UTC
(In reply to Honggang LI from comment #31)
> We need the attention of release eng team. Fix the rdma package is just fix
> part of the issue. As long as pxeboot/intrid.img file dose not have RDMA
> drivers, PXE over IPoIB would not work. We also need documentation for this
> issue.

"lorax" is the component to bug here.

Comment 36 Honggang LI 2016-07-24 23:49:40 UTC
(In reply to Harald Hoyer from comment #35)
> (In reply to Honggang LI from comment #31)
> > We need the attention of release eng team. Fix the rdma package is just fix
> > part of the issue. As long as pxeboot/intrid.img file dose not have RDMA
> > drivers, PXE over IPoIB would not work. We also need documentation for this
> > issue.
> 
> "lorax" is the component to bug here.

Hi, zguo

https://bugzilla.redhat.com/show_bug.cgi?id=1348381

Please test and feedback the result for this bug, as the lorax bug had been fixed too.

Comment 37 zguo 2016-07-26 00:44:53 UTC
1)
[root@ib2-qa-03 pxeboot]# cat /var/lib/tftpboot/pxelinux/pxelinux.cfg/default
default rhel73
prompt 1
timeout 600

label  rhel73
  menu label ^Install or upgrade an existing system
  kernel RHEL-7.3-20160719.1/vmlinuz
  ipappend 2
  append initrd=RHEL-7.3-20160719.1/initrd.img inst.repo=http://172.31.2.3/RHEL-7.3-20160719.1 ks.device=bootif ip=ib0:dhcp biosdevname=0  sshd rd.shell rd.debug=1 rd.neednet=1 rdloaddriver=mlx4_ib,ib_ipoib,ib_iser console=ttyS1,115200n81 inst.text

2) Get image from http://download.eng.pek2.redhat.com/pub/rhel/rel-eng/RHEL-7.3-20160719.1/compose/Server/x86_64/os/images/

[root@ib2-qa-03 pxeboot]# lsinitrd initrd.img | grep drivers/target
drwxr-xr-x   5 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target     
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/iscsi
-rw-r--r--   1 root     root       526993 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/iscsi/iscsi_target_mod.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/loopback
-rw-r--r--   1 root     root        45673 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/loopback/tcm_loop.ko
-rw-r--r--   1 root     root        56417 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/target_core_file.ko
-rw-r--r--   1 root     root        52073 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/target_core_iblock.ko
-rw-r--r--   1 root     root       634025 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/target_core_mod.ko
-rw-r--r--   1 root     root        37721 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/target_core_pscsi.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/tcm_fc
-rw-r--r--   1 root     root        53793 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/target/tcm_fc/tcm_fc.ko
[root@ib2-qa-03 pxeboot]# lsinitrd initrd.img | grep infiniband
drwxr-xr-x   6 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core
-rw-r--r--   1 root     root        22241 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_addr.ko
-rw-r--r--   1 root     root        82937 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_cm.ko
-rw-r--r--   1 root     root       200513 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_core.ko
-rw-r--r--   1 root     root        76705 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_mad.ko
-rw-r--r--   1 root     root        62689 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_sa.ko
-rw-r--r--   1 root     root        33369 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_ucm.ko
-rw-r--r--   1 root     root        35625 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_umad.ko
-rw-r--r--   1 root     root        93617 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/ib_uverbs.ko
-rw-r--r--   1 root     root        68913 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/iw_cm.ko
-rw-r--r--   1 root     root        91801 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/rdma_cm.ko
-rw-r--r--   1 root     root        40137 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/core/rdma_ucm.ko
drwxr-xr-x  13 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/cxgb3
-rw-r--r--   1 root     root       228065 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/cxgb3/iw_cxgb3.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/cxgb4
-rw-r--r--   1 root     root       285497 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/hfi1
-rw-r--r--   1 root     root       960273 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/hfi1/hfi1.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/ipath
-rw-r--r--   1 root     root       438377 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/ipath/ib_ipath.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/mlx4
-rw-r--r--   1 root     root       256593 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/mlx5
-rw-r--r--   1 root     root       249617 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/mthca
-rw-r--r--   1 root     root       219913 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/nes
-rw-r--r--   1 root     root       270721 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/nes/iw_nes.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/ocrdma
-rw-r--r--   1 root     root       125617 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/qib
-rw-r--r--   1 root     root       546569 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/qib/ib_qib.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/usnic
-rw-r--r--   1 root     root       134921 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko
drwxr-xr-x   3 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/sw
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/sw/rdmavt
-rw-r--r--   1 root     root        98529 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/sw/rdmavt/rdmavt.ko
drwxr-xr-x   7 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/ipoib
-rw-r--r--   1 root     root       165761 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/iser
-rw-r--r--   1 root     root        81769 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/iser/ib_iser.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/isert
-rw-r--r--   1 root     root        91945 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/isert/ib_isert.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/srp
-rw-r--r--   1 root     root        93289 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/srp/ib_srp.ko
drwxr-xr-x   2 root     root            0 Jul 19 04:28 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/srpt
-rw-r--r--   1 root     root        87049 Jul 15 19:08 usr/lib/modules/3.10.0-470.el7.x86_64/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko
[root@ib2-qa-03 pxeboot]# lsinitrd initrd.img | grep infiniband | wc -l
50


3) Boot client from ipoib,

...
Starting installer, one moment...
anaconda 21.48.22.80-1 for Red Hat Enterprise Linux 7.3 (pre-release) started.
 * installation log files are stored in /tmp during the installation
 * shell is available on TTY2
 * when reporting a bug add logs from /tmp as separate text/plain attachments
================================================================================
================================================================================
VNC

Text mode provides a limited set of installation options. It does not offer
custom partitioning for full control over the disk layout. Would you like to use
VNC mode instead?

 1) Start VNC

 2) Use text mode

  Please make your choice from above ['q' to quit | 'c' to continue |
  'r' to refresh]:

Comment 38 zguo 2016-07-26 00:55:46 UTC
Per c#37, we can see

1) drivers/target is loaded
2) RDMA drivers are packaged in RHEL-7.x/images/pxeboot/initrd.img
3) The dracut does not fail and drop into an emergency shell

I need continue c#37 3) to get below configuration done, then finish the installation process.

Installation

 1) [x] Language settings                 2) [x] Time settings
        (Afrikaans (South Africa))               (US/Pacific timezone)
 3) [!] Installation source               4) [!] Software selection
        (Error setting up software               (Installation source not set
        source)                                  up)
 5) [x] Installation Destination          6) [x] Kdump
        (Automatic partitioning                  (Kdump is disabled)
        selected)                         8) [x] Root password
 7) [x] Network configuration                    (Password is set.)
        (Unknown)
 9) [ ] User creation
        (No user will be created)

Comment 39 zguo 2016-07-26 07:32:07 UTC
[root@ib2-qa-03 RHEL-7.3-20160719.1]# cat anaconda-ks.cfg 
#version=DEVEL
# Install OS instead of upgrade
install
text
# Use NFS installation media
# Firewall configuration
firewall --disabled
firstboot --disable
ignoredisk --only-use=sda
# Keyboard layouts
# old format: keyboard us
# new format:
keyboard --vckeymap=us --xlayouts=''
# System language
lang en_US.UTF-8

# Network information
network  --hostname=localhost.localdomain
# Reboot after installation
reboot
# Root password
rootpw --iscrypted $1$kaan1hKf$qNnU4hFLgAtcTalZQaf5Y/
# SELinux configuration
selinux --enforcing
# System services
services --enabled="chronyd"
# Do not configure the X Window System
skipx
# System timezone
timezone America/New_York --ntpservers=ns1.pek.redhat.com,clock01.util.phx2.redhat.com,lanczos.maths.tcd.ie
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda
autopart --type=lvm
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel

%packages
@core
chrony
kexec-tools

%end

%addon com_redhat_kdump --enable --reserve-mb='auto'

%end

Comment 40 zguo 2016-07-26 07:40:05 UTC
Installation


** (anaconda:8617): WARNING **: Could not open X display

An unknown error has occured, look at the /tmp/anaconda-tb* file(s) for more details

===============================================================================
An unknown error has occurred
===============================================================================
anaconda 21.48.22.80-1 exception report
Traceback (most recent call first):
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/spokes/software.py", line 183, in _apply
    self.environment = self.payload.environments[self._selection]
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/spokes/software.py", line 172, in apply
    self._apply()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/spokes/software.py", line 79, in status
    self.apply()
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/spokes/__init__.py", line 89, in render
    title = _(self.title), text = self.status)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/simpleline/widgets.py", line 123, in render
    item.render(col_max_width)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/simpleline/widgets.py", line 123, in render
    item.render(col_max_width)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/simpleline/base.py", line 590, in show_all
    w.render(self.app.width)
  File "/usr/lib64/python2.7/site-packages/pyanaconda/ui/tui/simpleline/base.py", line 289, in _do_redraw
    screen.show_all()
IndexError: list index out of range

Comment 43 zguo 2016-07-27 01:00:12 UTC
*** Bug 1347545 has been marked as a duplicate of this bug. ***

Comment 44 zguo 2016-07-27 09:24:07 UTC
RHEL-7.3-20160726.n.0 is installed successfully by pxe over ipoib. Many thanks to Honggang's pointer.

Comment 45 Shany 2016-08-10 12:46:39 UTC
Hi,
Can we have the same fix for RHEL7.2 as well?

Thanks, Shany,

Comment 56 errata-xmlrpc 2016-11-04 03:26:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2309.html

Comment 57 Noa Spanier 2016-11-16 09:25:38 UTC
Hi Andre,

Any plans to have this fix for RHEL 7.2 as well?
What is the status for RHEL 7.1?

Thanks,
Noa

Comment 58 Red Hat Bugzilla 2023-09-14 03:26:46 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days