Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2105441 - RHEL 9 provisioned host goes into emergency mode after initial reboot
Summary: RHEL 9 provisioned host goes into emergency mode after initial reboot
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Provisioning
Version: 6.11.0
Hardware: x86_64
OS: Linux
high
high
Target Milestone: 6.13.0
Assignee: Leos Stejskal
QA Contact: sganar
URL:
Whiteboard:
: 2105440 2130998 (view as bug list)
Depends On:
Blocks: 2134412
TreeView+ depends on / blocked
 
Reported: 2022-07-08 19:03 UTC by Mike Allmen
Modified: 2024-11-12 07:54 UTC (History)
35 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
RHEL 9 provisioned host via Satellite goes into emergency mode after the initial reboot, the issue is seen only when there is a kernel update in the POST section of kickstart.
Clone Of:
: 2131759 2134412 (view as bug list)
Environment:
Last Closed: 2023-05-03 13:21:28 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 35331 0 High Closed RHEL 9 provisioned host goes into emergency mode after initial reboot 2022-09-15 13:01:04 UTC
Github theforeman foreman pull 9425 0 None Merged refs #35331 - Anaconda rhsm RHEL 9 params fix 2022-09-19 09:28:40 UTC
Red Hat Bugzilla 1940288 1 None None None 2023-09-09 07:43:25 UTC
Red Hat Issue Tracker SAT-12011 0 None None None 2022-09-15 12:08:00 UTC
Red Hat Knowledge Base (Solution) 6970059 0 None None None 2022-08-01 15:41:29 UTC
Red Hat Product Errata RHSA-2023:2097 0 None None None 2023-05-03 13:22:01 UTC

Internal Links: 2136901

Description Mike Allmen 2022-07-08 19:03:14 UTC 
Description of problem:
Provisioning a RHEL 9 host where kernel updates are applied after initial kickstart results in booting to emergency mode due to failure of initrd-switch-root.service

Version-Release number of selected component (if applicable):
6.11 Satellite, 9.0 RHEL

How reproducible:
Every time so far

Steps to Reproduce:
1. Create new host to provision with RHEL 9
2. Include activation key so host registers with Satellite during provisioning
3. Host applies available updates as part of 'Kickstart default' provisioning template with default to upgrade packages

Actual results:
Host provisions but upon first boot, when attempting to boot new kernel, system goes to emergency boot:

ignited-switch-root.service fails, error message on console:
Specified switch root path '/sysroot' does not seem to be an OS tree. os-release file is missing..

Expected results:
system boot fully into new kernel

Additional info:
default 9.0 kernel is 5.14.0-70.13.1.el9_0
failed updated kernel is 5.14.0-70.17.1.el9_0
Comment 1 Mike Allmen 2022-07-08 19:06:46 UTC 
If I boot to the original kernel, I can `dnf remove kernel*-5.14.0-70.17.1.el9_0`, `dnf update -y` which reinstalls the new kernel, reboot the host, and it boots correctly.
Comment 8 Brad Buckingham 2022-07-11 13:16:47 UTC 
*** Bug 2105440 has been marked as a duplicate of this bug. ***
Comment 10 Satyajit Das 2022-07-29 15:01:27 UTC 
A quick workaround would be to add the below parameter:-

Satellite GUI -> Hosts -> Operating Systems -> RedHat 9.0 -> Edit -> Parameters -> Add Parameter

Name              Type          Value
package_upgrade   boolean       false


Regards,
Comment 14 Leos Stejskal 2022-08-04 11:19:29 UTC 
Created redmine issue https://projects.theforeman.org/issues/35331 from this bug
Comment 18 Bryan Kearney 2022-09-15 12:04:23 UTC 
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/35331 has been resolved.
Comment 19 sganar 2022-09-27 08:42:19 UTC 
Verified.

Tested on Satellite 6.12.0 Snap 12.0

Steps followed: 
1. Create a new host to provision with RHEL 9
2. Include activation key so host registers with Satellite during provisioning
3. Host applies available updates as part of 'Kickstart default' provisioning template with default to upgrade packages


Observation: 
Provisioning is successful and the system boots fully into the new kernel without any errors.
Comment 21 Brad Buckingham 2022-10-06 13:15:49 UTC 
*** Bug 2130998 has been marked as a duplicate of this bug. ***
Comment 41 vijsingh 2022-11-15 16:50:00 UTC 
FailedQA as DEV and QE still able to reproduce the issue.
Comment 52 Leos Stejskal 2022-12-13 07:45:04 UTC 
So far no update, we'll get back to you once we know more.
Comment 53 Daniel 2023-01-10 09:31:03 UTC 
Hi,

I have the same problem.
Satellite: 6.12

I want to deploy RHEL 9.1 via satellite deployment.
I get the same error.
Same problem with RHEL 9.0
In the BaseOS Kickstart Repo, the kernel version is now 5.14.0-70.22.1 by default.
I think the kernel has the same problem.
How can I fix it?
Kernel 5.14.0-70.13.1 is no longer provided in the kickstart repo for RHEL 9.0.
The parameters don't help me at the moment.
Thank you
Comment 55 Anand Jambhulkar 2023-01-11 14:13:23 UTC 
Hi,

My customer is facing exactly the same issue as specified in Comment-53.

They would need a resolution soon to get past the issue to provision the RHEL-9 clients successfully.


Thanks & Regards,
Anand Jambhulkar
Comment 58 Leos Stejskal 2023-02-20 12:52:55 UTC 
Looks like JIRA ticket [0] has been resolved,
can we confirm that resyncing kickstart repos solve the provisioning issue?

[0] https://issues.redhat.com/browse/RHELDST-15385
Comment 66 errata-xmlrpc 2023-05-03 13:21:28 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097
Note You need to log in before you can comment on or make changes to this bug.