Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionManish Rangankar
2019-08-23 09:53:27 UTC
Created attachment 1607263[details]
iSCSI BFS tmp logs
Description of problem:
Unable to perform boot from SAN installation for iSCSI initiator over L2.
From the installation /tmp/syslogs, We can see ipaddress gets acquired correctly in the initial stag of installation, but at the end of the installation connection with iSCSI target gets reset because of interface loses its IPaddress.
I see some network manager logs before connection error i.e. iSCSI connection ping timeout (5 sec expiry). The network manager is messing up with ibft associated nic interface causes iscsi connection to timeout
Version-Release number of selected component (if applicable):
How reproducible:
Steps to Reproduce:
1. HPE Quick AH adapter under HPE Gen10
2. Configure iSCSI L2 BFS in HII and make sure target login is successful at pre-boot level
3. Mount RHEL-8.1.0-Partner-Pre-Beta ISO
4. Add kernel boot parameter 'rd.iscsi.ibft' during OS installation
5. Make sure target login successful with inbox drivers and start installation on the target Boot LUN
6. OS installation fails to proceed further and the observation is iBFT interface doesn't have any IP and fails to ping the target
Actual results:
L2 iSCSI Boot from SAN failed during installation
Expected results:
L2 iSCSI Boot from SAN to succeed.
Additional info:
/tmp directory debug logs are attached
Comment 2sudarsana.kalluru
2019-08-26 12:11:35 UTC
Posting query on behalf of Marvell QA engineer.
Following command is not persistent after the reboot. We need to try it during BFS OS installation, not on the installed OS.
configure level=TRACE in /etc/NetworkManager/NetworkManager.conf and reproduce (reboot).
Could you please provide instructions on how to collect the required logs during OS installation?
In Comment2 QA folks provided the needed information
Comment 4Beniamino Galvani
2020-04-02 07:58:29 UTC
Relevant excerpt from logs:
06:05:44,503 INFO dracut-initqueue:dhcp: PREINIT ibft0 up
06:05:44,522 INFO dhclient:DHCPDISCOVER on ibft0 to 255.255.255.255 port 67 interval 6 (xid=0xaf84e91a)
06:05:44,701 INFO dhclient:DHCPREQUEST on ibft0 to 255.255.255.255 port 67 (xid=0xaf84e91a)
06:05:44,701 INFO dhclient:DHCPOFFER from 192.168.100.5
06:05:44,732 INFO dhclient:DHCPACK from 192.168.100.5 (xid=0xaf84e91a)
06:05:46,760 INFO dhclient:bound to 192.168.100.25 -- renewal in 225 seconds.
06:06:18,965 INFO NetworkManager:<info> [1562911578.9651] NetworkManager (version 1.14.0-14.el8) is starting... (for the first time)
06:06:18,972 DEBUG NetworkManager:<debug> [1562911578.9728] platform: signal: address 4 added: 192.168.100.25/24 lft 568sec pref 568sec lifetime 1-1[568,568] dev 6 src kernel
06:06:24,085 DEBUG nm-dispatcher:req:3 'up' [ibft0]: environment: DEVICE_IFACE=ibft0
06:06:24,085 DEBUG nm-dispatcher:req:3 'up' [ibft0]: environment: DEVICE_IP_IFACE=ibft0
06:06:24,085 DEBUG nm-dispatcher:req:3 'up' [ibft0]: environment: IP4_ADDRESS_0=192.168.100.25/24 192.168.100.5
06:15:46,032 DEBUG NetworkManager:<debug> [1562912146.0326] platform: signal: address 4 removed: 192.168.100.25/24 lft 0sec pref 0sec lifetime 569-1[568,568] dev 6 src kernel
06:15:46,033 DEBUG NetworkManager:<debug> [1562912146.0327] device[0x5562139cf6a0] (ibft0): queued IP4 config change
06:15:52,176 ERR kernel: connection1:0: ping timeout of 5 secs expired, recv timeout 5, last rx 4295278592, last ping 4295283712, now 4295288832
06:15:52,433 WARNING iscsid:Kernel reported iSCSI connection 1:0 error (1022 - ISCSI_ERR_NOP_TIMEDOUT: A NOP has timed out) state (3)
where it can be seen that the initrd gets address 192.168.100.5 via
DHCP with a lifetime of 10 minutes and NetworkManager lets the address
expire, leading to iSCSI errors in accessing the filesystem.
This will be fixed in RHEL 8.2 by commit [1].
[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/commit/f2fe6c03ee3fe5e262cbaa5b898f05bb8e349d0e
Got verification confirmation from our testing team also and they have verified iSCSI IBFT (non-offload) DHCP IPv4 on RHEL-8.2.0-Partner-Snapshot3 with Inbox drivers (qed/qede:8.37.0.20). It works fine.
Thanks,
Manish
It seems this is effectively fixed on rhel-8.2.
Note that from rhel-8.3 onwards, RHEL will default to run NetworkManager in dracut. That means, the implementation for setting up networking in early boot changes completely. Of course, the functionality is supposed to still work.
Moving to MODIFIED.
We have some iSCSI dracut tests covered, but we are not able to test ibft.table with DHCP scenario. We test ibft.table with static adresses and getting iSCSI root from DHCP (dynamic adresses).
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (NetworkManager bug fix and enhancement update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHBA-2020:4499
Created attachment 1607263 [details] iSCSI BFS tmp logs Description of problem: Unable to perform boot from SAN installation for iSCSI initiator over L2. From the installation /tmp/syslogs, We can see ipaddress gets acquired correctly in the initial stag of installation, but at the end of the installation connection with iSCSI target gets reset because of interface loses its IPaddress. I see some network manager logs before connection error i.e. iSCSI connection ping timeout (5 sec expiry). The network manager is messing up with ibft associated nic interface causes iscsi connection to timeout Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. HPE Quick AH adapter under HPE Gen10 2. Configure iSCSI L2 BFS in HII and make sure target login is successful at pre-boot level 3. Mount RHEL-8.1.0-Partner-Pre-Beta ISO 4. Add kernel boot parameter 'rd.iscsi.ibft' during OS installation 5. Make sure target login successful with inbox drivers and start installation on the target Boot LUN 6. OS installation fails to proceed further and the observation is iBFT interface doesn't have any IP and fails to ping the target Actual results: L2 iSCSI Boot from SAN failed during installation Expected results: L2 iSCSI Boot from SAN to succeed. Additional info: /tmp directory debug logs are attached