Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1813047

Summary:	[4.3.z] RHCOS fails to boot with coreos-luks-open.service: Main process exited, code=killed, status=15/TERM error
Product:	OpenShift Container Platform	Reporter:	Micah Abbott <miabbott>
Component:	RHCOS	Assignee:	Ben Howard <behoward>
Status:	CLOSED DEFERRED	QA Contact:	Michael Nguyen <mnguyen>
Severity:	high	Docs Contact:
Priority:	medium
Version:	4.3.0	CC:	bbreard, behoward, cfergeau, dustymabe, imcleod, jlebon, jligon, miabbott, mnguyen, nstielau, prkumar, smilner
Target Milestone:	---
Target Release:	4.3.z
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1803130	Environment:
Last Closed:	2020-08-12 15:58:31 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1803130, 1847673
Bug Blocks:

Comment 1 Micah Abbott 2020-03-12 19:33:21 UTC

Merged to 4.3 here - https://gitlab.cee.redhat.com/coreos/redhat-coreos/merge_requests/863

Will move to MODIFIED once a 4.3 build is made

Comment 2 Praveen Kumar 2020-03-17 03:54:09 UTC

Do let us know which 4.3.z it is available so we can test it out.

Comment 3 Micah Abbott 2020-03-17 14:50:49 UTC

(In reply to Praveen Kumar from comment #2)
> Do let us know which 4.3.z it is available so we can test it out.

It should be in RHCOS 43.81.202003161653.0; I believe that will be included in 4.3.8 or later.

It's currently available in the 4.3 nightly release payload, but I think it missed the cut for 4.3.7

Comment 11 Michael Nguyen 2020-03-24 12:49:27 UTC

(In reply to Praveen Kumar from comment #2)
> Do let us know which 4.3.z it is available so we can test it out.

@praveen 4.3.8 has it.  Can you test it out when you get the chance?

Comment 13 Praveen Kumar 2020-03-26 12:21:13 UTC

@Michael So It tested 4.3.8 on centos CI and it again failed with more or like with same issue and it does goes to emergency shell mode :(


```
Escape character is ^]
[***   ] (2 of 2) A start job is running for…S LUKS Opener (1min 3s / no limit)[   68.915308] systemd-udevd[509]: seq 1508 '/devices/pci0000:00/0000:00:05.0/virtio2/block/vda' is taking a long time
[[   94.915128] systemd[1]: dev-disk-by\x2dlabel-root.device: Job dev-disk-by\x2dlabel-root.device/start timed out.
 TIME ] Timed out waiting for device dev-disk-by\x2dlabel-root.device.
[[   94.939739] systemd[1]: Timed out waiting for device dev-disk-by\x2dlabel-root.device.
DEPEND] Dependency failed for CoreOS: Mo[   94.949088] systemd[1]: Dependency failed for CoreOS: Mount (subsequent) /sysroot.
unt (subsequent) /sysroot.
[   94.957640] systemd[1]: ignition-ostree-mount-subsequent-sysroot.service: Job ignition-ostree-mount-subsequent-sysroot.service/start failed with result 'dependency'.
[   94.971401] systemd[1]: dev-disk-by\x2dlabel-root.device: Job dev-disk-by\x2dlabel-root.device/start failed with result 'timeout'.
         Starting OSTree Prepare OS/...[   94.983400] systemd[1]: Starting OSTree Prepare OS/...

[   95.009168] ostree-prepare-root[696]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ostree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111c/0': No such file or directory
[[   95.027372] o[   95.028123] systemd[1]: ostree-prepare-root.service: Main process exited, code=exited, status=1/FAILURE
stree-prepare-root[696]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ost[   95.039266] systemd[1]: ostree-prepare-root.service: Failed with result 'exit-code'.
ree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111[   95.049335] systemd[1]: Failed to start OSTree Prepare OS/.
c/0': No such file or directory
FAILED] Failed to start OSTree Prepare OS/.
See 'systemctl status ostree-prepare-root.service' for details.
[   95.062827] systemd[1]: ostree-prepare-root.service: Triggering OnFailure= dependencies.
[[   95.069918] systemd[1]: Stopped dracut pre-mount hook.
  OK  ] Stopped dracut pre-mount hook.
[   95.077722] systemd[1]: Stopped target Subsequent (Not Ignition) boot complete.
[  OK  ] Stopped target Subsequent (Not Ignition) boot complete.
[   95.088374] systemd[1]: Stopped dracut initqueue hook.
[  OK  ] Stopped dracut initqueue hook.
[   95.096683] systemd[1]: Stopped dracut pre-trigger hook.
[  OK  ] Stopped dracut pre-trigger hook.
[   95.104879] systemd[1]: Stopped dracut pre-udev hook.
[  OK  ] Stopped dracut pre-udev hook.
[   95.112921] systemd[1]: Stopped dracut cmdline hook.
[  OK  ] Stopped dracut cmdline hook.
[[   95.121845] systemd[1]: Stopped dracut ask for additional cmdline parameters.
  OK  ] Stopped dracut ask for additional cmdline parameters.
         [   95.137543] systemd[1]: Starting Setup Virtual Console...
Starting Setup Virtual Console...
[   95.146152] systemd[1]: Reached target Initrd Root File System.
[  OK  ] Reached target Initrd Root File System.
[   95.158284] systemd[1]: Starting Reload Configuration from the Real Root...
         Starting Reload Configuration from the Real Root...
[   95.169468] systemd[1]: coreos-luks-open.service: Main process exited, code=killed, status=15/TERM
[   95.264163] systemd[1]: Reloading.
[   95.487090] systemd[1]: Started Setup Virtual Console.
[  OK  ] Started Setup Virtual Console.
[   95.652828] systemd[1]: Started Reload Configuration from the Real Root.
[  OK  ] Started Reload Configuration from the Real Root.
[   95.660429] systemd[1]: Reached target Initrd File Systems.
[  OK  ] Reached target Initrd File Systems.
[   95.667628] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...
         Starting Cleaning Up and Shutting Down Daemons...
[[   95.743164] systemd[1]: Stopped target Timers.
  OK  ] Stopped target Timers.
[  OK     95.748355] systemd[1]: Stopped target Initrd Root Device.
m] Stopped target Initrd Root Device.
         [   95.757913] systemd[1]: Starting OSTree Prepare OS/...
Starting OSTree Prepare OS/...
[[   95.767094] systemd[1]: Stopped target Remote File Systems.
  OK  ] Stopped target Remote File Systems.
[   95.776736] systemd[1]: Stopped target Remote File Systems (Pre).
[  OK  ] Stopped target Remote File Systems (Pre).
[   95.782702] ostree-prepare-root[755]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ostree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111c/0': No such file or directory
         Stopping Open-iSCSI...
[   95.794028] ostree-prepare-root[755]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ostree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111c/0': No such file or directory
[[   95.803154] systemd[1]: Stopping Open-iSCSI...
FAILED[   95.811032] systemd[1]: ostree-prepare-root.service: Main process exited, code=exited, status=1/FAILURE
] Failed to start [   95.816655] systemd[1]: ostree-prepare-root.service: Failed with result 'exit-code'.
OSTree Prepare OS/.
[   95.822319] systemd[1]: Failed to start OSTree Prepare OS/.
See 'systemctl status ostree-prepare-root.service' for details.
[   95.829724] systemd[1]: ostree-prepare-root.service: Triggering OnFailure= dependencies.
[   95.839754] systemd[1]: Started Cleaning Up and Shutting Down Daemons.
[  OK  ] Started Cleaning Up and Shutting Down Daemons.
[   95.849179] systemd[1]: Starting Setup Virtual Console...
         Starting Setup Virtual Console...
[   95.866780] iscsid[624]: iscsid: iscsid shutting down.
[   95.872142] systemd[1]: Stopped Open-iSCSI.
[  OK  ] Stopped Open-iSCSI.
[   95.875721] systemd[1]: Stopping iSCSI UserSpace I/O driver...
         Stopping iSCSI UserSpace I/O driver...
[   95.881940] systemd[1]: Stopped iSCSI UserSpace I/O driver.
[  OK  ] Stopped iSCSI UserSpace I/O driver.
[  OK     96.018982] systemd[1]: Started Setup Virtual Console.
0m] Started Setup Virtual Console.
[  OK[  135.159432] systemd[1]: coreos-luks-open.service: Failed with result 'signal'.
  ] Stopped CoreOS LUKS Opener.
         Startin[  135.172185] systemd[1]: Stopped CoreOS LUKS Opener.
g Setup Virtual Console...
[  OK    135.180380] systemd[1]: coreos-luks-open.service: Triggering OnFailure= dependencies.
0m] Stopped target Initrd Root File System.
[  OK    135.192443] systemd[1]: Starting Setup Virtual Console...
0m] Stopped target Initrd File Systems.
[  OK    135.200599] systemd[1]: Stopped target Initrd Root File System.
0m] Stopped target Basic System.
[  OK    135.208928] systemd[1]: Stopped target Initrd File Systems.
0m] Stopped target Paths.
[  OK    135.216974] systemd[1]: Stopped target Basic System.
0m] Stopped target Sockets.
[  OK    135.224528] systemd[1]: Stopped target Paths.
0m] Closed Open-iSCSI iscsid Socket.
[  OK    135.230740] systemd[1]: Stopped target Sockets.
0m] Closed Open-iSCSI iscsiuio Socket.
[  OK    135.237076] systemd[1]: Closed Open-iSCSI iscsid Socket.
0m] Stopped target Slices.
[  135.242706] systemd[1]: Closed Open-iSCSI iscsiuio Socket.
[  OK  ] Stopped target System Initialization.
[  OK    135.248870] systemd[1]: Stopped target Slices.
0m] Stopped target Local Encrypted Volumes.
[  135.254456] systemd[1]: Stopped target System Initialization.
[  OK    135.259805] systemd[1]: Stopped target Local Encrypted Volumes.
0m] Stopped Dispatch Password Requests to Console Directory Watch.
[  135.266788] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.
[  OK    135.274358] systemd[1]: Stopped Create Volatile Files and Directories.
0m] Stopped Create Volatile Files and Directories.
[  135.281269] systemd[1]: Stopping Journal Service...
[  135.281643] systemd-journald[230]: Received SIGTERM from PID 1 (systemd).
         Stopping Journal Service...
[  135.291193] systemd[1]: Stopped target Swap.
[  OK  ] Stopped target Swap.
[  135.294649] systemd[1]: Stopped target Local File Systems.
[  OK  ] Stopped target Local File Systems.
[  135.298936] systemd[1]: Stopped target Local File Systems (Pre).
[  OK  ] Stopped target Local File Systems (Pre).
[  135.305112] systemd[1]: Stopping Device-Mapper Multipath Device Controller...
         Stopping Device-Mapper Multipath Device Controller...
[  135.311768] systemd[1]: Stopping udev Kernel Device Manager...
         Stopping udev Kernel Device Manager...
[  135.411476] systemd[1]: Stopped Apply Kernel Variables.
[  OK  ] Stopped Apply Kernel Variables.
[  135.417285] systemd[1]: Stopped Journal Service.
[  OK  ] Stopped Journal Service.
[  135.422312] systemd[1]: Stopped udev Kernel Device Manager.
[  OK  ] Stopped udev Kernel Device Manager.
[  135.430443] systemd[1]: Started Setup Virtual Console.
[  OK  ] Started Setup Virtual Console.
[  135.439125] systemd[1]: Stopped Device-Mapper Multipath Device Controller.
[  OK  ] Stopped Device-Mapper Multipath Device Controller.
[  135.447577] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[  135.452690] systemd[1]: Stopped udev Wait for Complete Device Initialization.
[  OK  ] Stopped udev Wait for Complete Device Initialization.
[  135.459905] systemd[1]: Stopped udev Coldplug all Devices.
[  OK  ] Stopped udev Coldplug all Devices.
[  135.466289] systemd[1]: Started Emergency Shell.
[  OK  ] Started Emergency Shell.
[  135.470544] systemd[1]: Reached target Emergency Mode.
[  OK  ] Reached target Emergency Mode.
[  135.476095] systemd[1]: Stopped Create Static Device Nodes in /dev.
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  135.482223] systemd[1]: Stopped Create list of required static device nodes for the current kernel.
[  OK  ] Stopped Create list of required sta…vice nodes for the current kernel.
[  135.490153] systemd[1]: Closed udev Kernel Socket.
[  OK  ] Closed udev Kernel Socket.
[  135.494477] systemd[1]: Closed udev Control Socket.
[  OK  ] Closed udev Control Socket.
[  136.179459] systemd[1]: Started Journal Service.
[  136.179674] multipathd[767]: Mar 26 12:18:03 | /etc/multipath.conf does not exist, blacklisting all devices.
[  136.187283] multipathd[767]: Mar 26 12:18:03 | You can run "/sbin/mpathconf --enable" to create
[  OK  ] Started Journal Service.
[  136.193248] multipathd[767]: Mar 26 12:18:03 | /etc/multipath.conf. See man mpathconf(8) for more details
[  136.198221] multipathd[767]: ok
[  136.200449] systemd[1]: Startup finished in 4.443s (kernel) + 0 (initrd) + 2min 11.749s (userspace) = 2min 16.193s.
Press Enter for emergency shell or wait 4 minutes 45 seconds for reboot.      


```

Comment 14 Steve Milner 2020-03-26 13:38:29 UTC

Based on the report back I'm moving this to assigned as the issue seems to be not be addressed as expected.

Comment 16 Micah Abbott 2020-06-16 19:07:06 UTC

Higher priority BZs and feature work prevented this BZ from being looked at during the latest sprint.

Resetting priority to 'medium' to match BZ targeted for 4.4/4.5 and noting the UpcomingSprint keyword.

Comment 17 Ben Howard 2020-06-24 20:37:51 UTC

Higher priority BZs and feature work prevented this BZ from being looked at during the latest sprint.

Comment 19 Red Hat Bugzilla 2023-09-14 05:54:20 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days