+++ This bug was initially created as a clone of Bug #1901021 +++ Description of problem: According OpenShift 4.6 Official document, the RHCOS worker node's default ISCSI initiator name is in /etc/iscsi/initiatorname.iscsi https://docs.openshift.com/container-platform/4.6/storage/persistent_storage/persistent-storage-iscsi.html But we cannot find this file in RHCOS 4.6.1 !!! [core@worker-1 ~]$ ls -a /etc/iscsi/ . .. iscsid.conf [core@worker-1 ~]$ hostnamectl Static hostname: n/a Transient hostname: worker-1.ocp4-1.example.internal Icon name: computer-vm Chassis: vm Machine ID: ada997b5da89423e8255633f232d8db4 Boot ID: f4d48fddb7f0499c97dede5fd1ec47c1 Virtualization: vmware Operating System: Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) CPE OS Name: cpe:/o:redhat:enterprise_linux:8::coreos Kernel: Linux 4.18.0-193.24.1.el8_2.dt1.x86_64 Architecture: x86-64 --- Additional comment from kevin on 2020-11-24 10:15:31 UTC --- In RHCOS 4.5, the initiatorname.iscsi default existed. But in RHCOS 4.6.1, this file is missing! --- Additional comment from Luca BRUNO on 2020-11-24 11:04:29 UTC --- Misc notes: * The file under /etc should is not supposed be part of the base OS in both RHCOS-4.5 and RHCOS-4.6. The InitatorName is machine-specific, thus it shouldn't be backed into the OS image * The RHEL package has some buggy %post logic in the RPM that we have to bypass on CoreOS side, see https://bugzilla.redhat.com/show_bug.cgi?id=1734144 * We carry some service units specific to CoreOS in order to generate the initiatorname on first-boot, see https://github.com/openshift/os/blob/a06951cb0b9ec63d79087ca257a2135601e027dd/overlay.d/05rhcos/usr/lib/systemd/system/coreos-generate-iscsi-initiatorname.service * There have been some work around this area in the 4.5->4.6 timeframe which may have resulted in a regression, see https://gitlab.cee.redhat.com/coreos/redhat-coreos/-/merge_requests/1098 I would be good to attach here the full journal from this 4.6 node to pinpoint what's going on. --- Additional comment from Luca BRUNO on 2020-11-24 11:14:53 UTC --- Ah, I think MR#1098 above renamed the service unit (s/regenerate/generate/), but the `43-manifest-rhcos.preset` content still tries to reference/enable the unit using the old name. Jonathan, can you maybe have a look here? Micah, did you verify https://bugzilla.redhat.com/show_bug.cgi?id=1868174 using a patched bootimage? --- Additional comment from Scott Dodson on 2020-11-24 13:55:32 UTC --- --- Additional comment from Jonathan Lebon on 2020-11-24 14:25:01 UTC --- (In reply to Luca BRUNO from comment #3) > Ah, I think MR#1098 above renamed the service unit (s/regenerate/generate/), > but the `43-manifest-rhcos.preset` content still tries to reference/enable > the unit using the old name. Hmm indeed, you're right. I'm confused now how this worked during testing and verification. Anyway, will fix this and add a test for it as well. --- Additional comment from Jonathan Lebon on 2020-11-24 15:08:49 UTC --- https://github.com/openshift/os/pull/453 https://gitlab.cee.redhat.com/coreos/redhat-coreos/-/merge_requests/1180 --- Additional comment from Benjamin Gilbert on 2020-11-24 16:06:32 UTC --- Adding to bootimage bump per Jonathan. --- Additional comment from kevin on 2020-11-24 16:30:39 UTC --- I have aggregate some log from journal related to iscsi [root@worker-1 ~]# journalctl |grep "can't open InitiatorAlias configuration file /etc/iscsi/initiatorname.iscsi" -B 5 -A 5 Nov 24 08:58:12 localhost multipathd[688]: /etc/multipath.conf. See man mpathconf(8) for more details Nov 24 08:58:12 localhost iscsid[690]: iscsid: can't open InitiatorName configuration file /etc/iscsi/initiatorname.iscsi Nov 24 08:58:12 localhost iscsid[690]: iscsid: Warning: InitiatorName file /etc/iscsi/initiatorname.iscsi does not exist or does not contain a properly formatted InitiatorName. If using software iscsi (iscsi_tcp or ib_iser) or partial offload (bnx2i or cxgbi iscsi), you may not be able to log into or discover targets. Please create a file /etc/iscsi/initiatorname.iscsi that contains a sting with the format: InitiatorName=iqn.yyyy-mm.<reversed domain name>[:identifier]. Nov 24 08:58:12 localhost iscsid[690]: Example: InitiatorName=iqn.2001-04.com.redhat:fc6. Nov 24 08:58:12 localhost iscsid[690]: If using hardware iscsi like qla4xxx this message can be ignored. Nov 24 08:58:12 localhost iscsid[690]: iscsid: can't open InitiatorAlias configuration file /etc/iscsi/initiatorname.iscsi Nov 24 08:58:12 localhost systemd[1]: Started Open-iSCSI. Nov 24 08:58:12 localhost systemd[1]: Started Create Volatile Files and Directories. Nov 24 08:58:12 localhost systemd[1]: Reached target System Initialization. Nov 24 08:58:12 localhost systemd[1]: Reached target Basic System. Nov 24 08:58:12 localhost systemd[1]: Starting Ignition (fetch-offline)... -- Nov 24 09:01:25 localhost multipathd[638]: /etc/multipath.conf. See man mpathconf(8) for more details Nov 24 09:01:25 localhost iscsid[639]: iscsid: can't open InitiatorName configuration file /etc/iscsi/initiatorname.iscsi Nov 24 09:01:25 localhost iscsid[639]: iscsid: Warning: InitiatorName file /etc/iscsi/initiatorname.iscsi does not exist or does not contain a properly formatted InitiatorName. If using software iscsi (iscsi_tcp or ib_iser) or partial offload (bnx2i or cxgbi iscsi), you may not be able to log into or discover targets. Please create a file /etc/iscsi/initiatorname.iscsi that contains a sting with the format: InitiatorName=iqn.yyyy-mm.<reversed domain name>[:identifier]. Nov 24 09:01:25 localhost iscsid[639]: Example: InitiatorName=iqn.2001-04.com.redhat:fc6. Nov 24 09:01:25 localhost iscsid[639]: If using hardware iscsi like qla4xxx this message can be ignored. Nov 24 09:01:25 localhost iscsid[639]: iscsid: can't open InitiatorAlias configuration file /etc/iscsi/initiatorname.iscsi Nov 24 09:01:25 localhost systemd[1]: Started Open-iSCSI. Nov 24 09:01:25 localhost systemd[1]: Started Create Volatile Files and Directories. Nov 24 09:01:25 localhost systemd[1]: Reached target System Initialization. Nov 24 09:01:25 localhost systemd[1]: Reached target Basic System. Nov 24 09:01:25 localhost systemd[1]: Starting dracut initqueue hook... --- Additional comment from kevin on 2020-11-25 01:59:27 UTC --- [root@worker-1 ~]# journalctl -u coreos-generate-iscsi-initiatorname.service -- Logs begin at Tue 2020-11-24 08:58:09 UTC, end at Wed 2020-11-25 01:58:17 UTC. -- -- No entries -- --- Additional comment from Jonathan Lebon on 2020-11-26 16:32:46 UTC --- Fixed by https://github.com/openshift/installer/pull/4422. --- Additional comment from Micah Abbott on 2020-11-30 21:32:20 UTC --- (In reply to Jonathan Lebon from comment #10) > Fixed by https://github.com/openshift/installer/pull/4422. This PR has merged, so moving to MODIFIED --- Additional comment from kevin on 2020-12-01 16:13:17 UTC --- which OCP 4.6 minor version can be fix ? --- Additional comment from Micah Abbott on 2020-12-01 21:44:46 UTC --- (In reply to kevin from comment #12) > which OCP 4.6 minor version can be fix ? I would expect it to be part of 4.6.7 --- Additional comment from kevin on 2020-12-02 12:34:56 UTC --- Thank you for your reply
This is fixed with https://github.com/openshift/os/pull/453 and https://github.com/openshift/installer/pull/4414
This is pending the merge of the installer PR; setting UpcomingSprint to appease the bots.
Installer PR is merged, moving to MODIFIED
Verified with 4.6.0-0.nightly-2020-12-09-162454 / RHCOS 46.82.202012082041-0 ``` [core@cosa-devsh ~]$ rpm-ostree status State: idle Deployments: * ostree://0c0f8983e459f70654a92f1deb03e74814d881c814416f43a8433bc7f91bb1c8 Version: 46.82.202012082041-0 (2020-12-08T20:45:12Z) [core@cosa-devsh ~]$ ls /etc/iscsi/ initiatorname.iscsi iscsid.conf [core@cosa-devsh ~]$ cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.1994-05.com.redhat:26bc2b78e478 [core@cosa-devsh ~]$ systemctl status coreos-generate-iscsi-initiatorname ● coreos-generate-iscsi-initiatorname.service - CoreOS Generate iSCSI Initiator Name Loaded: loaded (/usr/lib/systemd/system/coreos-generate-iscsi-initiatorname.service; enabled; vendor preset: enabled) Active: active (exited) since Wed 2020-12-09 22:06:49 UTC; 2min 23s ago Docs: https://bugzilla.redhat.com/show_bug.cgi?id=1493296 https://bugzilla.redhat.com/show_bug.cgi?id=1687722 Process: 1422 ExecStart=/usr/bin/sh -c echo "InitiatorName=`/usr/sbin/iscsi-iname`" > /etc/iscsi/initiatorname.iscsi (code=exited, status=0/SUCCESS) Main PID: 1422 (code=exited, status=0/SUCCESS) Tasks: 0 (limit: 5724) Memory: 0B CGroup: /system.slice/coreos-generate-iscsi-initiatorname.service Dec 09 22:06:49 localhost systemd[1]: Starting CoreOS Generate iSCSI Initiator Name... Dec 09 22:06:49 localhost systemd[1]: Started CoreOS Generate iSCSI Initiator Name. ```
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5633