Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2105441

Summary:	RHEL 9 provisioned host goes into emergency mode after initial reboot
Product:	Red Hat Satellite	Reporter:	Mike Allmen <mallmen>
Component:	Provisioning	Assignee:	Leos Stejskal <lstejska>
Status:	CLOSED ERRATA	QA Contact:	sganar
Severity:	high	Docs Contact:
Priority:	high
Version:	6.11.0	CC:	ahumbe, ajambhul, aperotti, aruzicka, bdm, craig.parker, daniel.pilawa, dparke05, fperalta, huali, inecas, jalviso, j.becker, jbhatia, jcastran, jesper.schmidt, jyoung, lstejska, lvaleeva, mfuruta, mhulan, msunil, pcreech, pdwyer, pmendezh, riehecky, rlavi, sadas, saydas, sganar, sgardner, sigbjorn.lie, sshtein, vijsingh, yferszt
Target Milestone:	6.13.0	Keywords:	PrioBumpGSS, Triaged
Target Release:	Unused
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Known Issue
Doc Text:	RHEL 9 provisioned host via Satellite goes into emergency mode after the initial reboot, the issue is seen only when there is a kernel update in the POST section of kickstart.	Story Points:	---
Clone Of:
Clones:	2131759 2134412 (view as bug list)		Environment:
Last Closed:	2023-05-03 13:21:28 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2134412

Description Mike Allmen 2022-07-08 19:03:14 UTC

Description of problem:
Provisioning a RHEL 9 host where kernel updates are applied after initial kickstart results in booting to emergency mode due to failure of initrd-switch-root.service

Version-Release number of selected component (if applicable):
6.11 Satellite, 9.0 RHEL

How reproducible:
Every time so far

Steps to Reproduce:
1. Create new host to provision with RHEL 9
2. Include activation key so host registers with Satellite during provisioning
3. Host applies available updates as part of 'Kickstart default' provisioning template with default to upgrade packages

Actual results:
Host provisions but upon first boot, when attempting to boot new kernel, system goes to emergency boot:

ignited-switch-root.service fails, error message on console:
Specified switch root path '/sysroot' does not seem to be an OS tree. os-release file is missing..

Expected results:
system boot fully into new kernel

Additional info:
default 9.0 kernel is 5.14.0-70.13.1.el9_0
failed updated kernel is 5.14.0-70.17.1.el9_0

Comment 1 Mike Allmen 2022-07-08 19:06:46 UTC

If I boot to the original kernel, I can `dnf remove kernel*-5.14.0-70.17.1.el9_0`, `dnf update -y` which reinstalls the new kernel, reboot the host, and it boots correctly.

Comment 8 Brad Buckingham 2022-07-11 13:16:47 UTC

*** Bug 2105440 has been marked as a duplicate of this bug. ***

Comment 10 Satyajit Das 2022-07-29 15:01:27 UTC

A quick workaround would be to add the below parameter:-

Satellite GUI -> Hosts -> Operating Systems -> RedHat 9.0 -> Edit -> Parameters -> Add Parameter

Name              Type          Value
package_upgrade   boolean       false


Regards,

Comment 14 Leos Stejskal 2022-08-04 11:19:29 UTC

Created redmine issue https://projects.theforeman.org/issues/35331 from this bug

Comment 18 Bryan Kearney 2022-09-15 12:04:23 UTC

Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/35331 has been resolved.

Comment 19 sganar 2022-09-27 08:42:19 UTC

Verified.

Tested on Satellite 6.12.0 Snap 12.0

Steps followed: 
1. Create a new host to provision with RHEL 9
2. Include activation key so host registers with Satellite during provisioning
3. Host applies available updates as part of 'Kickstart default' provisioning template with default to upgrade packages


Observation: 
Provisioning is successful and the system boots fully into the new kernel without any errors.

Comment 21 Brad Buckingham 2022-10-06 13:15:49 UTC

*** Bug 2130998 has been marked as a duplicate of this bug. ***

Comment 41 vijsingh 2022-11-15 16:50:00 UTC

FailedQA as DEV and QE still able to reproduce the issue.

Comment 52 Leos Stejskal 2022-12-13 07:45:04 UTC

So far no update, we'll get back to you once we know more.

Comment 53 Daniel 2023-01-10 09:31:03 UTC

Hi,

I have the same problem.
Satellite: 6.12

I want to deploy RHEL 9.1 via satellite deployment.
I get the same error.
Same problem with RHEL 9.0
In the BaseOS Kickstart Repo, the kernel version is now 5.14.0-70.22.1 by default.
I think the kernel has the same problem.
How can I fix it?
Kernel 5.14.0-70.13.1 is no longer provided in the kickstart repo for RHEL 9.0.
The parameters don't help me at the moment.
Thank you

Comment 55 Anand Jambhulkar 2023-01-11 14:13:23 UTC

Hi,

My customer is facing exactly the same issue as specified in Comment-53.

They would need a resolution soon to get past the issue to provision the RHEL-9 clients successfully.


Thanks & Regards,
Anand Jambhulkar

Comment 58 Leos Stejskal 2023-02-20 12:52:55 UTC

Looks like JIRA ticket [0] has been resolved,
can we confirm that resyncing kickstart repos solve the provisioning issue?

[0] https://issues.redhat.com/browse/RHELDST-15385

Comment 66 errata-xmlrpc 2023-05-03 13:21:28 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.13 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2097