Created attachment 1980832 [details] kickstart file Description of problem: Running an kickstart installation produces an invalid /boot/loader/entries/XXX.conf file which causes boot problems. The options line contains the option from the kickstart and not for the final OS. e.g.: options inst.ks=http://192.168.0.4/autoinst/testcentos9.cfg inst.text net.ifnames=0 biosdevname=0 selinux=0 nameserver=192.168.0.4 ip=10.70.56.13::10.70.56.1:255.255.255.224:testcentos9:eth0:none Version-Release number of selected component (if applicable): Compose_ID CentOS-Stream-9-20230727.1 How reproducible: Run a kickstart installation with the attached ks.cfg file. We are booting from the ISO image inside a VMware VM. Steps to Reproduce: 1. Modify the ks.cfg to you test environment and place the file on webserver 2. Boot up from the ISO 3. Use the following options inst.ks=http://192.168.0.4/autoinst/testcentos9.cfg inst.text net.ifnames=0 biosdevname=0 selinux=0 nameserver=192.168.0.4 ip=10.70.56.13::10.70.56.1:255.255.255.224:testcentos9:eth0:none Actual results: anaconda installs with an non functional /boot/loader/entries/XXX.conf System will not boot up (e.g. initrd-switch-root.service fails) Expected results: anaconda installs the system with functional /boot/loader/entries/XXX.conf, system will boot Additional info:
the CentOS Stream 9 test environment also confirm that it's not installable : https://testing.stream.centos.org/ : all failing for cs9 deployments. Manually redeployed one of the kvm guests and can confirm I see the same problem reported in this ticket
Created attachment 1980836 [details] package diff
I did some tests with local freezed BaseOS copy from 20230706. Installing with the latest install.img against the local freezed copy is working fine. I attached a list of new packages between my old copy and the current BaseOS. Hopefully this helps to find the root cause.
Quickly looking at older runs on testing.stream.centos.org, the compose CentOS-Stream-9-20230720.0 installed fine (even though the tests failed later on): https://testing.stream.centos.org/job/CentOS_9-stream_x86_64_kvm/22/consoleFull
Also the DVD (CentOS-Stream-9-20230727.1-aarch64-dvd1.iso) installs fine using Anaconda, I tested that on Friday.
Reproduced by running a manual installation from https://odcs.stream.centos.org/production/CentOS-Stream-9-20230731.0/compose/BaseOS/x86_64/os/ in a VM (KVM), using the defaults in the installer. The kernel boot arguments on the installed system are the ones used for the installation: "inst.repo=https://odcs.stream.centos.org/.../x86_64/os/". Reassigning to grub2.
Also meet this issue when deploy instance with CentOS-Stream-GenericCloud-9-20230727.1.x86_64.qcow2 on OpenStack PSI: Failing to boot the instance, it enters to dracut and reports "can't mount root filesystem"
I was performing a test install of CentOS Stream 9 using the latest[1] available dvd iso from the mirror[2]. I manually select "minimal install" from the selectable packages. The install seems to go smoothly. Upon reboot I am greeted with: [FAILED] Failed to start Switch Root Further examination of the switch-root service shows the following: Failed to switch root: Specified switch root path '/sysroot' does not seem to be an OS tree. os-release file is missing. Booting into rescue media shows that the os-release file(s) actually do exist: ls -l /etc/os-release lwrxwrxwrx. 1 root root 21 Apr 6 08:27 /etc/os-release -> ../usr/lib/os- release ls -l /usr/lib/os-release -rw-r--r--. 1 root root 391 Apr 6 08:27 /usr/lib/os-release [1] - https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/iso/CentOS-Stream-9-latest-x86_64-dvd1.iso [2] - https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/iso/
Tagged the older version of grub2 temporarily into c9s and running a new c9s production compose now
*** Bug 2229122 has been marked as a duplicate of this bug. ***
I have verified the failure. I have finished a rebuild of the CentOS Stream 9 compose that has the latest grub in it. I have verified that this latest compose (CentOS-Stream-9-20230807.1) installs and boots correctly, using the same procedure that failed before. In short, I believe this latest compose has this fixed. I have already pushed this compose to the repo mirrors. We have an 8 hour delay so that the repo's can propagate around the world before we push out the images to our various places. If you want to test the various images before the 8 hours they are here https://composes.stream.centos.org/production/latest-CentOS-Stream/compose/BaseOS/x86_64/images/
I have confirmed the fix with newly synced repos (my installation was via Kickstart as in the original report). System now boots correctly after kickstart installation. Thanks Troy!
I can also confirm that the issue is solved with the latest compose (20230807.1). Thanks!
This issue is now fixed, both in the CentOS Stream 9 repository, and in all the CentOS Stream 9 images. I'm not sure if I should close this, or if the Bootloader team needs to do something before it's closed.
Thanks to everyone who caught and, importantly, fixed this! (New version of the patch that caused this has been tested more thoroughly, and will not go into a compose before further testing is performed) I am closing this one now... sorry for the hassle.
*** Bug 2228847 has been marked as a duplicate of this bug. ***