RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1768635 - iscsid: can not create NETLINK_ISCSI socket
Summary: iscsid: can not create NETLINK_ISCSI socket
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: iscsi-initiator-utils
Version: 7.7
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: rc
: ---
Assignee: Chris Leech
QA Contact: Filip Suba
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-04 20:57 UTC by tcleveng
Modified: 2024-06-13 22:17 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-08-27 13:19:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description tcleveng 2019-11-04 20:57:07 UTC
Description of problem:
Customer cannot start iscsid service due FAILURE issues.


Version-Release number of selected component (if applicable):
- We have tried with newest test iscsi-initiator-utils package initiator-utils-6.2.0.874-17.el7.x86_64

How reproducible:
- Customer states iscsid errors out with almost every single attempt to start service.


Steps to Reproduce:
1. Boot system
2. Try to start iscsid
3. Monitor logfiles

Actual results:
Get following logs:
ar/log/messages-20191103:Oct 31 11:11:57 odb12c-1 systemd: Listening on Open-iSCSI iscsid Socket.
var/log/messages-20191103:Oct 31 11:11:57 odb12c-1 systemd: Listening on Open-iSCSI iscsiuio Socket.
var/log/messages-20191103:Oct 31 11:11:58 odb12c-1 NetworkManager[1523]: <info>  [1572534718.0169] ifcfg-rh: new connection /etc/sysconfig/network-scripts/ifcfg-iSCSI (1e187a71-a031-4af6-9b06-d14c746400e4,"iSCSI")
var/log/messages-20191103:Oct 31 11:11:58 odb12c-1 NetworkManager[1523]: <info>  [1572534718.2785] policy: auto-activating connection 'iSCSI' (1e187a71-a031-4af6-9b06-d14c746400e4)
var/log/messages-20191103:Oct 31 11:11:58 odb12c-1 NetworkManager[1523]: <info>  [1572534718.2794] device (ens256): Activation: starting connection 'iSCSI' (1e187a71-a031-4af6-9b06-d14c746400e4)
var/log/messages-20191103:Oct 31 11:11:59 odb12c-1 network: Bringing up interface iSCSI:  [  OK  ]
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Starting Open-iSCSI...
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 iscsid: iscsid: can not create NETLINK_ISCSI socket
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: iscsid.service: main process exited, code=exited, status=1/FAILURE
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Failed to start Open-iSCSI.
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Unit iscsid.service entered failed state.
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: iscsid.service failed.
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Starting Logout off all iSCSI sessions on shutdown...
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Starting Login and scanning of iSCSI devices...
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Started Logout off all iSCSI sessions on shutdown.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: iscsid.service holdoff time over, scheduling restart.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Stopped Open-iSCSI.

Expected results:
- iscsid to start with NO issues

Additional info:
- SERVICE INFO:
* iscsi.service - Login and scanning of iSCSI devices
   Loaded: loaded (/usr/lib/systemd/system/iscsi.service; enabled; vendor preset: disabled)
     Docs: man:iscsiadm(8)
           man:iscsid(8)
  Process: 2678 ExecStart=/sbin/iscsiadm -m node --loginall=automatic (code=exited, status=18)
   CGroup: /system.slice/iscsi.service
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: got read error (-1/104), daemon died?
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: Could not login to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:oracle12c-ebsdb01-v01c453ab45a53bf3.0000002e.44616858, portal: 172.16.252.150,3260].
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: initiator reported error (18 - could not communicate to iscsid)
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: got read error (-1/104), daemon died?
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: Could not login to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:sql14-temp-v01c453ab45a53bf3.00000036.44616858, portal: 172.16.252.150,3260].
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: initiator reported error (18 - could not communicate to iscsid)
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: Could not log into all portals
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: Logging in to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:oracle12c-ebsdb01-v01c453ab45a53bf3.0000002e.44616858, portal: 172.16.252.150,3260] (multiple)
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: Logging in to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:sql14-temp-v01c453ab45a53bf3.00000036.44616858, portal: 172.16.252.150,3260] (multiple)


* iscsid.service - Open-iSCSI
   Loaded: loaded (/usr/lib/systemd/system/iscsid.service; enabled; vendor preset: disabled)
     Docs: man:iscsid(8)
           man:iscsiuio(8)
           man:iscsiadm(8)
  Process: 18445 ExecStart=/sbin/iscsid -f (code=exited, status=1/FAILURE)
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: start request repeated too quickly for iscsid.service
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: Unit iscsid.service entered failed state.
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: iscsid.service failed.
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: start request repeated too quickly for iscsid.service
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: iscsid.service failed.
Oct 31 15:11:14 odb12c-1.telamon.telamon-corp.com systemd[1]: start request repeated too quickly for iscsid.service
Oct 31 15:11:14 odb12c-1.telamon.telamon-corp.com systemd[1]: iscsid.service failed.

Comment 2 tcleveng 2019-11-04 21:05:30 UTC
o ISSUE:
- Cannot start iscsid service

o NOTES:
- Tried new iscsi-utils package initiator-utils-6.2.0.874-17.el7.x86_64 with no luck 
- We are not getting iscsid.service never wrote its PID file. Failing anymore but seeing below errors






o NEW SOSREPORT:
sosreport-odb12c-1-02493028-2019-11-04-zmhbymr]$ xsos -o .
OS
  Hostname: odb12c-1.telamon.telamon-corp.com
  Distro:   [redhat-release] Red Hat Enterprise Linux Server release 7.6 (Maipo)

  Kernel:
    Booted kernel:  3.10.0-1062.el7.x86_64
    GRUB default:   3.10.0-1062.el7.x86_64  
=======================================================================================



sosreport-odb12c-1-02493028-2019-11-04-zmhbymr]$ less installed-rpms |grep iscsi
iscsi-initiator-utils-6.2.0.874-11.el7.x86_64               Fri Oct 11 11:21:13 2019
iscsi-initiator-utils-6.2.0.874-17.el7.x86_64               Thu Oct 31 11:10:09 2019 <-- installed
iscsi-initiator-utils-debuginfo-6.2.0.874-17.el7.x86_64     Thu Oct 31 11:10:10 2019
iscsi-initiator-utils-devel-6.2.0.874-17.el7.x86_64         Thu Oct 31 11:10:10 2019
iscsi-initiator-utils-iscsiuio-6.2.0.874-11.el7.x86_64      Fri Oct 11 11:21:12 2019
iscsi-initiator-utils-iscsiuio-6.2.0.874-17.el7.x86_64      Thu Oct 31 11:10:09 2019
libiscsi-1.9.0-7.el7.x86_64                                 Fri Jan  4 13:44:36 2019
libvirt-daemon-driver-storage-iscsi-4.5.0-23.el7.x86_64     Mon Aug 19 14:47:59 2019


[tcleveng@localhost sosreport-odb12c-1-02493028-2019-11-04-zmhbymr]$ less sos_commands/systemd/systemctl_status_--all |grep iscsi
           | |   |-18545 iscsid
           | |   `-18546 iscsid
             |-iscsi.service
             |-iscsi-shutdown.service
           |-18545 iscsid
           `-18546 iscsid
Oct 31 15:11:21 odb12c-1.telamon.telamon-corp.com iscsid[18544]: iSCSI logger with pid=18545 started!
Oct 31 15:11:22 odb12c-1.telamon.telamon-corp.com iscsid[18545]: iSCSI daemon with pid=18546 started!
* iscsi-shutdown.service - Logout off all iSCSI sessions on shutdown
   Loaded: loaded (/usr/lib/systemd/system/iscsi-shutdown.service; static; vendor preset: disabled)
     Docs: man:iscsid(8)
           man:iscsiadm(8)
   CGroup: /system.slice/iscsi-shutdown.service


* iscsi.service - Login and scanning of iSCSI devices
   Loaded: loaded (/usr/lib/systemd/system/iscsi.service; enabled; vendor preset: disabled)
     Docs: man:iscsiadm(8)
           man:iscsid(8)
  Process: 2678 ExecStart=/sbin/iscsiadm -m node --loginall=automatic (code=exited, status=18)
   CGroup: /system.slice/iscsi.service
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: got read error (-1/104), daemon died?
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: Could not login to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:oracle12c-ebsdb01-v01c453ab45a53bf3.0000002e.44616858, portal: 172.16.252.150,3260].
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: initiator reported error (18 - could not communicate to iscsid)
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: got read error (-1/104), daemon died?
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: Could not login to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:sql14-temp-v01c453ab45a53bf3.00000036.44616858, portal: 172.16.252.150,3260].
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: initiator reported error (18 - could not communicate to iscsid)
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: iscsiadm: Could not log into all portals
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: Logging in to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:oracle12c-ebsdb01-v01c453ab45a53bf3.0000002e.44616858, portal: 172.16.252.150,3260] (multiple)
Oct 31 14:48:22 odb12c-1.telamon.telamon-corp.com iscsiadm[2678]: Logging in to [iface: iface_ens256, target: iqn.2007-11.com.nimblestorage:sql14-temp-v01c453ab45a53bf3.00000036.44616858, portal: 172.16.252.150,3260] (multiple)


* iscsid.service - Open-iSCSI
   Loaded: loaded (/usr/lib/systemd/system/iscsid.service; enabled; vendor preset: disabled)
     Docs: man:iscsid(8)
           man:iscsiuio(8)
           man:iscsiadm(8)
  Process: 18445 ExecStart=/sbin/iscsid -f (code=exited, status=1/FAILURE)
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: start request repeated too quickly for iscsid.service
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: Unit iscsid.service entered failed state.
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: iscsid.service failed.
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: start request repeated too quickly for iscsid.service
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: iscsid.service failed.
Oct 31 15:11:14 odb12c-1.telamon.telamon-corp.com systemd[1]: start request repeated too quickly for iscsid.service
Oct 31 15:11:14 odb12c-1.telamon.telamon-corp.com systemd[1]: iscsid.service failed.


* iscsiuio.service - iSCSI UserSpace I/O driver
   Loaded: loaded (/usr/lib/systemd/system/iscsiuio.service; disabled; vendor preset: disabled)
     Docs: man:iscsiuio(8)
           |-iscsi.service
           |-iscsi-shutdown.service
             |-18545 iscsid
             `-18546 iscsid
           |   |-18545 iscsid
           |   `-18546 iscsid
* iscsid.socket - Open-iSCSI iscsid Socket
   Loaded: loaded (/usr/lib/systemd/system/iscsid.socket; enabled; vendor preset: disabled)
     Docs: man:iscsid(8)
           man:iscsiadm(8)
Oct 31 15:11:13 odb12c-1.telamon.telamon-corp.com systemd[1]: Listening on Open-iSCSI iscsid Socket.
Oct 31 15:11:14 odb12c-1.telamon.telamon-corp.com systemd[1]: Unit iscsid.socket entered failed state.
* iscsiuio.socket - Open-iSCSI iscsiuio Socket
   Loaded: loaded (/usr/lib/systemd/system/iscsiuio.socket; enabled; vendor preset: disabled)
     Docs: man:iscsiuio(8)
Oct 31 14:48:17 odb12c-1.telamon.telamon-corp.com systemd[1]: Listening on Open-iSCSI iscsiuio Socket.


========================================================================================================
o REBOOTS: (After installing iscsi test package)

var/log/messages-20191103:Oct 31 11:11:53 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 13:50:17 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 14:02:04 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 14:18:15 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 14:24:03 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 14:28:33 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 14:33:39 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019
var/log/messages-20191103:Oct 31 14:48:13 odb12c-1 kernel: Linux version 3.10.0-1062.el7.x86_64 (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) ) #1 SMP Thu Jul 18 20:25:13 UTC 2019



o LOGS:
ar/log/messages-20191103:Oct 31 11:11:57 odb12c-1 systemd: Listening on Open-iSCSI iscsid Socket.
var/log/messages-20191103:Oct 31 11:11:57 odb12c-1 systemd: Listening on Open-iSCSI iscsiuio Socket.
var/log/messages-20191103:Oct 31 11:11:58 odb12c-1 NetworkManager[1523]: <info>  [1572534718.0169] ifcfg-rh: new connection /etc/sysconfig/network-scripts/ifcfg-iSCSI (1e187a71-a031-4af6-9b06-d14c746400e4,"iSCSI")
var/log/messages-20191103:Oct 31 11:11:58 odb12c-1 NetworkManager[1523]: <info>  [1572534718.2785] policy: auto-activating connection 'iSCSI' (1e187a71-a031-4af6-9b06-d14c746400e4)
var/log/messages-20191103:Oct 31 11:11:58 odb12c-1 NetworkManager[1523]: <info>  [1572534718.2794] device (ens256): Activation: starting connection 'iSCSI' (1e187a71-a031-4af6-9b06-d14c746400e4)
var/log/messages-20191103:Oct 31 11:11:59 odb12c-1 network: Bringing up interface iSCSI:  [  OK  ]
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Starting Open-iSCSI...
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 iscsid: iscsid: can not create NETLINK_ISCSI socket
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: iscsid.service: main process exited, code=exited, status=1/FAILURE
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Failed to start Open-iSCSI.
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Unit iscsid.service entered failed state.
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: iscsid.service failed.
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Starting Logout off all iSCSI sessions on shutdown...
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Starting Login and scanning of iSCSI devices...
var/log/messages-20191103:Oct 31 11:12:00 odb12c-1 systemd: Started Logout off all iSCSI sessions on shutdown.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: iscsid.service holdoff time over, scheduling restart.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Stopped Open-iSCSI.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Starting Open-iSCSI...
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 iscsid: iscsid: can not create NETLINK_ISCSI socket
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: iscsid.service: main process exited, code=exited, status=1/FAILURE
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Failed to start Open-iSCSI.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Unit iscsid.service entered failed state.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: iscsid.service failed.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: iscsid.service holdoff time over, scheduling restart.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Stopped Open-iSCSI.
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Starting Open-iSCSI...
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 iscsid: iscsid: can not create NETLINK_ISCSI socket
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: iscsid.service: main process exited, code=exited, status=1/FAILURE
var/log/messages-20191103:Oct 31 11:12:01 odb12c-1 systemd: Failed to start Open-iSCSI.
...



o NOTES:
- This is interesting, claiming this could be a sexlinux issue:
https://bugzilla.redhat.com/show_bug.cgi?id=1273252

- Checked it and the customer has selinux disabled:
SELinux status:                 disabled

Comment 4 Chris Williams 2020-11-11 21:37:54 UTC
Red Hat Enterprise Linux 7 shipped it's final minor release on September 29th, 2020. 7.9 was the last minor releases scheduled for RHEL 7.
From intial triage it does not appear the remaining Bugzillas meet the inclusion criteria for Maintenance Phase 2 and will now be closed. 

From the RHEL life cycle page:
https://access.redhat.com/support/policy/updates/errata#Maintenance_Support_2_Phase
"During Maintenance Support 2 Phase for Red Hat Enterprise Linux version 7,Red Hat defined Critical and Important impact Security Advisories (RHSAs) and selected (at Red Hat discretion) Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available."

If this BZ was closed in error and meets the above criteria please re-open it flag for 7.9.z, provide suitable business and technical justifications, and follow the process for Accelerated Fixes:
https://source.redhat.com/groups/public/pnt-cxno/pnt_customer_experience_and_operations_wiki/support_delivery_accelerated_fix_release_handbook  

Feature Requests can re-opened and moved to RHEL 8 if the desired functionality is not already present in the product. 

Please reach out to the applicable Product Experience Engineer[0] if you have any questions or concerns.  

[0] https://bugzilla.redhat.com/page.cgi?id=agile_component_mapping.html&product=Red+Hat+Enterprise+Linux+7

Comment 10 Jon Magrini 2021-07-19 20:10:53 UTC
It looks like the socket starts listening before the network is started:

21:41:12 hostname systemd: Listening on Open-iSCSI iscsid Socket.
21:41:12 hostname systemd: Listening on Open-iSCSI iscsiuio Socket.

Then NetworkManager starts:

21:41:12 hostname systemd: Starting Network Manager...

Then we see the open-iscsi service try to start right after the network target is reached, but before the network-online target is reached, and then the service fails immediately after:

21:41:13 hostname systemd: Started LSB: Bring up/down networking.
21:41:13 hostname systemd: Reached target Network.
21:41:13 hostname systemd: Starting OpenSSH server daemon...
21:41:13 hostname systemd: Starting Open-iSCSI...
21:41:13 hostname systemd: Reached target Network is Online.
21:41:13 hostname iscsid: iscsid: sysfs_init: sysfs_path='/sys'
21:41:13 hostname iscsid: iscsid: in ctldev_open
21:41:13 hostname iscsid: iscsid: can not create NETLINK_ISCSI socket
21:41:13 hostname systemd: iscsid.service: main process exited, code=exited, status=1/FAILURE
21:41:13 hostname systemd: Failed to start Open-iSCSI.
21:41:13 hostname systemd: Unit iscsid.service entered failed state.
21:41:13 hostname systemd: iscsid.service failed.

Comment 15 loberman 2021-08-24 16:40:07 UTC
This is strange

I created a test program

#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <netdb.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <unistd.h>
#include <errno.h>
#include <arpa/inet.h> 
#include <linux/netlink.h>

int main(int argc, char *argv[])
{
    int sockfd = 0;
    if((sockfd = socket(PF_NETLINK, SOCK_RAW, NETLINK_ISCSI)) < 0)
    {
        printf("\n Error : Could not create socket \n");
        return 1;
    } 
    return 0;
}


I unloaded the module

[root@loberhel iscsi_socket_issue]# rmmod scsi_transport_iscsi
[root@loberhel iscsi_socket_issue]# lsmod | grep iscsi


Running the program autoloads the module so it should be happening as part of systemd as well

[root@loberhel iscsi_socket_issue]# ./iscsi_socket
[root@loberhel iscsi_socket_issue]# 

Aug 24 12:39:39 loberhel kernel: Loading iSCSI transport class v2.0-870.

Comment 16 loberman 2021-08-24 16:44:36 UTC
Note that I can reproduce this if I make the module unavailable.

[root@loberhel iscsi_socket_issue]# ./iscsi_socket

 Error : Could not create socket 


Regards
Laurence

Comment 17 loberman 2021-08-24 18:13:32 UTC
[root@loberhel iscsi_socket_issue]# ./iscsi_socket

 Error : Could not create socket 
Failed to create
: Protocol not supported

After modifying test code with module unavailable to load.
So confirms the issue here.

Will look to code a fix after I know the outcome of the sleep delay.

Comment 18 loberman 2021-08-25 19:49:28 UTC
We are cautiously optimistic this may be resolved.
We are waiting on the customer to test.

David Jeffery decided it may be a module load alias issue so he had the customer check for the existence of
modules.alias.bin.
It was indeed missing and seems to have been the reason why the alias for 
alias net-pf-16-proto-8 scsi_transport_iscsi was missing sporadically.

I cant explain why it worked sometimes for this customer but does explain the missing ISCSI_NETLINK issue.

We will finalize this as notabug in iscsi once we have confirmation.

As usual David comes through like always. Thank you David.

Regards
Laurence and David

Comment 20 loberman 2021-08-27 13:19:33 UTC
This is closing notabaug
Resolved by running depmod -a to rebuild the binary alias file.
Veritas seems to be trying to rebuild this.

Comment 22 Red Hat Bugzilla 2024-01-04 04:25:02 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.