Bug 1813047 - [4.3.z] RHCOS fails to boot with coreos-luks-open.service: Main process exited, code=killed, status=15/TERM error
Summary: [4.3.z] RHCOS fails to boot with coreos-luks-open.service: Main process exite...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: RHCOS
Version: 4.3.0
Hardware: x86_64
OS: Linux
medium
high
Target Milestone: ---
: 4.3.z
Assignee: Ben Howard
QA Contact: Michael Nguyen
URL:
Whiteboard:
Depends On: 1803130 1847673
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-12 19:31 UTC by Micah Abbott
Modified: 2023-09-14 05:54 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1803130
Environment:
Last Closed: 2020-08-12 15:58:31 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Comment 1 Micah Abbott 2020-03-12 19:33:21 UTC
Merged to 4.3 here - https://gitlab.cee.redhat.com/coreos/redhat-coreos/merge_requests/863

Will move to MODIFIED once a 4.3 build is made

Comment 2 Praveen Kumar 2020-03-17 03:54:09 UTC
Do let us know which 4.3.z it is available so we can test it out.

Comment 3 Micah Abbott 2020-03-17 14:50:49 UTC
(In reply to Praveen Kumar from comment #2)
> Do let us know which 4.3.z it is available so we can test it out.

It should be in RHCOS 43.81.202003161653.0; I believe that will be included in 4.3.8 or later.

It's currently available in the 4.3 nightly release payload, but I think it missed the cut for 4.3.7

Comment 11 Michael Nguyen 2020-03-24 12:49:27 UTC
(In reply to Praveen Kumar from comment #2)
> Do let us know which 4.3.z it is available so we can test it out.

@praveen 4.3.8 has it.  Can you test it out when you get the chance?

Comment 13 Praveen Kumar 2020-03-26 12:21:13 UTC
@Michael So It tested 4.3.8 on centos CI and it again failed with more or like with same issue and it does goes to emergency shell mode :(


```
Escape character is ^]
[***   ] (2 of 2) A start job is running for…S LUKS Opener (1min 3s / no limit)[   68.915308] systemd-udevd[509]: seq 1508 '/devices/pci0000:00/0000:00:05.0/virtio2/block/vda' is taking a long time
[[   94.915128] systemd[1]: dev-disk-by\x2dlabel-root.device: Job dev-disk-by\x2dlabel-root.device/start timed out.
 TIME ] Timed out waiting for device dev-disk-by\x2dlabel-root.device.
[[   94.939739] systemd[1]: Timed out waiting for device dev-disk-by\x2dlabel-root.device.
DEPEND] Dependency failed for CoreOS: Mo[   94.949088] systemd[1]: Dependency failed for CoreOS: Mount (subsequent) /sysroot.
unt (subsequent) /sysroot.
[   94.957640] systemd[1]: ignition-ostree-mount-subsequent-sysroot.service: Job ignition-ostree-mount-subsequent-sysroot.service/start failed with result 'dependency'.
[   94.971401] systemd[1]: dev-disk-by\x2dlabel-root.device: Job dev-disk-by\x2dlabel-root.device/start failed with result 'timeout'.
         Starting OSTree Prepare OS/...[   94.983400] systemd[1]: Starting OSTree Prepare OS/...

[   95.009168] ostree-prepare-root[696]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ostree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111c/0': No such file or directory
[[   95.027372] o[   95.028123] systemd[1]: ostree-prepare-root.service: Main process exited, code=exited, status=1/FAILURE
stree-prepare-root[696]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ost[   95.039266] systemd[1]: ostree-prepare-root.service: Failed with result 'exit-code'.
ree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111[   95.049335] systemd[1]: Failed to start OSTree Prepare OS/.
c/0': No such file or directory
FAILED] Failed to start OSTree Prepare OS/.
See 'systemctl status ostree-prepare-root.service' for details.
[   95.062827] systemd[1]: ostree-prepare-root.service: Triggering OnFailure= dependencies.
[[   95.069918] systemd[1]: Stopped dracut pre-mount hook.
  OK  ] Stopped dracut pre-mount hook.
[   95.077722] systemd[1]: Stopped target Subsequent (Not Ignition) boot complete.
[  OK  ] Stopped target Subsequent (Not Ignition) boot complete.
[   95.088374] systemd[1]: Stopped dracut initqueue hook.
[  OK  ] Stopped dracut initqueue hook.
[   95.096683] systemd[1]: Stopped dracut pre-trigger hook.
[  OK  ] Stopped dracut pre-trigger hook.
[   95.104879] systemd[1]: Stopped dracut pre-udev hook.
[  OK  ] Stopped dracut pre-udev hook.
[   95.112921] systemd[1]: Stopped dracut cmdline hook.
[  OK  ] Stopped dracut cmdline hook.
[[   95.121845] systemd[1]: Stopped dracut ask for additional cmdline parameters.
  OK  ] Stopped dracut ask for additional cmdline parameters.
         [   95.137543] systemd[1]: Starting Setup Virtual Console...
Starting Setup Virtual Console...
[   95.146152] systemd[1]: Reached target Initrd Root File System.
[  OK  ] Reached target Initrd Root File System.
[   95.158284] systemd[1]: Starting Reload Configuration from the Real Root...
         Starting Reload Configuration from the Real Root...
[   95.169468] systemd[1]: coreos-luks-open.service: Main process exited, code=killed, status=15/TERM
[   95.264163] systemd[1]: Reloading.
[   95.487090] systemd[1]: Started Setup Virtual Console.
[  OK  ] Started Setup Virtual Console.
[   95.652828] systemd[1]: Started Reload Configuration from the Real Root.
[  OK  ] Started Reload Configuration from the Real Root.
[   95.660429] systemd[1]: Reached target Initrd File Systems.
[  OK  ] Reached target Initrd File Systems.
[   95.667628] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...
         Starting Cleaning Up and Shutting Down Daemons...
[[   95.743164] systemd[1]: Stopped target Timers.
  OK  ] Stopped target Timers.
[  OK     95.748355] systemd[1]: Stopped target Initrd Root Device.
m] Stopped target Initrd Root Device.
         [   95.757913] systemd[1]: Starting OSTree Prepare OS/...
Starting OSTree Prepare OS/...
[[   95.767094] systemd[1]: Stopped target Remote File Systems.
  OK  ] Stopped target Remote File Systems.
[   95.776736] systemd[1]: Stopped target Remote File Systems (Pre).
[  OK  ] Stopped target Remote File Systems (Pre).
[   95.782702] ostree-prepare-root[755]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ostree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111c/0': No such file or directory
         Stopping Open-iSCSI...
[   95.794028] ostree-prepare-root[755]: ostree-prepare-root: Couldn't find specified OSTree root '/sysroot//ostree/boot.1/rhcos/039cc1e4bbdb3119ff487119cc505de2c6691b02c9f0d34ee38bb7674907111c/0': No such file or directory
[[   95.803154] systemd[1]: Stopping Open-iSCSI...
FAILED[   95.811032] systemd[1]: ostree-prepare-root.service: Main process exited, code=exited, status=1/FAILURE
] Failed to start [   95.816655] systemd[1]: ostree-prepare-root.service: Failed with result 'exit-code'.
OSTree Prepare OS/.
[   95.822319] systemd[1]: Failed to start OSTree Prepare OS/.
See 'systemctl status ostree-prepare-root.service' for details.
[   95.829724] systemd[1]: ostree-prepare-root.service: Triggering OnFailure= dependencies.
[   95.839754] systemd[1]: Started Cleaning Up and Shutting Down Daemons.
[  OK  ] Started Cleaning Up and Shutting Down Daemons.
[   95.849179] systemd[1]: Starting Setup Virtual Console...
         Starting Setup Virtual Console...
[   95.866780] iscsid[624]: iscsid: iscsid shutting down.
[   95.872142] systemd[1]: Stopped Open-iSCSI.
[  OK  ] Stopped Open-iSCSI.
[   95.875721] systemd[1]: Stopping iSCSI UserSpace I/O driver...
         Stopping iSCSI UserSpace I/O driver...
[   95.881940] systemd[1]: Stopped iSCSI UserSpace I/O driver.
[  OK  ] Stopped iSCSI UserSpace I/O driver.
[  OK     96.018982] systemd[1]: Started Setup Virtual Console.
0m] Started Setup Virtual Console.
[  OK[  135.159432] systemd[1]: coreos-luks-open.service: Failed with result 'signal'.
  ] Stopped CoreOS LUKS Opener.
         Startin[  135.172185] systemd[1]: Stopped CoreOS LUKS Opener.
g Setup Virtual Console...
[  OK    135.180380] systemd[1]: coreos-luks-open.service: Triggering OnFailure= dependencies.
0m] Stopped target Initrd Root File System.
[  OK    135.192443] systemd[1]: Starting Setup Virtual Console...
0m] Stopped target Initrd File Systems.
[  OK    135.200599] systemd[1]: Stopped target Initrd Root File System.
0m] Stopped target Basic System.
[  OK    135.208928] systemd[1]: Stopped target Initrd File Systems.
0m] Stopped target Paths.
[  OK    135.216974] systemd[1]: Stopped target Basic System.
0m] Stopped target Sockets.
[  OK    135.224528] systemd[1]: Stopped target Paths.
0m] Closed Open-iSCSI iscsid Socket.
[  OK    135.230740] systemd[1]: Stopped target Sockets.
0m] Closed Open-iSCSI iscsiuio Socket.
[  OK    135.237076] systemd[1]: Closed Open-iSCSI iscsid Socket.
0m] Stopped target Slices.
[  135.242706] systemd[1]: Closed Open-iSCSI iscsiuio Socket.
[  OK  ] Stopped target System Initialization.
[  OK    135.248870] systemd[1]: Stopped target Slices.
0m] Stopped target Local Encrypted Volumes.
[  135.254456] systemd[1]: Stopped target System Initialization.
[  OK    135.259805] systemd[1]: Stopped target Local Encrypted Volumes.
0m] Stopped Dispatch Password Requests to Console Directory Watch.
[  135.266788] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.
[  OK    135.274358] systemd[1]: Stopped Create Volatile Files and Directories.
0m] Stopped Create Volatile Files and Directories.
[  135.281269] systemd[1]: Stopping Journal Service...
[  135.281643] systemd-journald[230]: Received SIGTERM from PID 1 (systemd).
         Stopping Journal Service...
[  135.291193] systemd[1]: Stopped target Swap.
[  OK  ] Stopped target Swap.
[  135.294649] systemd[1]: Stopped target Local File Systems.
[  OK  ] Stopped target Local File Systems.
[  135.298936] systemd[1]: Stopped target Local File Systems (Pre).
[  OK  ] Stopped target Local File Systems (Pre).
[  135.305112] systemd[1]: Stopping Device-Mapper Multipath Device Controller...
         Stopping Device-Mapper Multipath Device Controller...
[  135.311768] systemd[1]: Stopping udev Kernel Device Manager...
         Stopping udev Kernel Device Manager...
[  135.411476] systemd[1]: Stopped Apply Kernel Variables.
[  OK  ] Stopped Apply Kernel Variables.
[  135.417285] systemd[1]: Stopped Journal Service.
[  OK  ] Stopped Journal Service.
[  135.422312] systemd[1]: Stopped udev Kernel Device Manager.
[  OK  ] Stopped udev Kernel Device Manager.
[  135.430443] systemd[1]: Started Setup Virtual Console.
[  OK  ] Started Setup Virtual Console.
[  135.439125] systemd[1]: Stopped Device-Mapper Multipath Device Controller.
[  OK  ] Stopped Device-Mapper Multipath Device Controller.
[  135.447577] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[  135.452690] systemd[1]: Stopped udev Wait for Complete Device Initialization.
[  OK  ] Stopped udev Wait for Complete Device Initialization.
[  135.459905] systemd[1]: Stopped udev Coldplug all Devices.
[  OK  ] Stopped udev Coldplug all Devices.
[  135.466289] systemd[1]: Started Emergency Shell.
[  OK  ] Started Emergency Shell.
[  135.470544] systemd[1]: Reached target Emergency Mode.
[  OK  ] Reached target Emergency Mode.
[  135.476095] systemd[1]: Stopped Create Static Device Nodes in /dev.
[  OK  ] Stopped Create Static Device Nodes in /dev.
[  135.482223] systemd[1]: Stopped Create list of required static device nodes for the current kernel.
[  OK  ] Stopped Create list of required sta…vice nodes for the current kernel.
[  135.490153] systemd[1]: Closed udev Kernel Socket.
[  OK  ] Closed udev Kernel Socket.
[  135.494477] systemd[1]: Closed udev Control Socket.
[  OK  ] Closed udev Control Socket.
[  136.179459] systemd[1]: Started Journal Service.
[  136.179674] multipathd[767]: Mar 26 12:18:03 | /etc/multipath.conf does not exist, blacklisting all devices.
[  136.187283] multipathd[767]: Mar 26 12:18:03 | You can run "/sbin/mpathconf --enable" to create
[  OK  ] Started Journal Service.
[  136.193248] multipathd[767]: Mar 26 12:18:03 | /etc/multipath.conf. See man mpathconf(8) for more details
[  136.198221] multipathd[767]: ok
[  136.200449] systemd[1]: Startup finished in 4.443s (kernel) + 0 (initrd) + 2min 11.749s (userspace) = 2min 16.193s.
Press Enter for emergency shell or wait 4 minutes 45 seconds for reboot.      


```

Comment 14 Steve Milner 2020-03-26 13:38:29 UTC
Based on the report back I'm moving this to assigned as the issue seems to be not be addressed as expected.

Comment 16 Micah Abbott 2020-06-16 19:07:06 UTC
Higher priority BZs and feature work prevented this BZ from being looked at during the latest sprint.

Resetting priority to 'medium' to match BZ targeted for 4.4/4.5 and noting the UpcomingSprint keyword.

Comment 17 Ben Howard 2020-06-24 20:37:51 UTC
Higher priority BZs and feature work prevented this BZ from being looked at during the latest sprint.

Comment 19 Red Hat Bugzilla 2023-09-14 05:54:20 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days


Note You need to log in before you can comment on or make changes to this bug.