Bulk setting Target Milestone = 6.15.0 where sat-6.15.0+ is set.

> After some investigation and discussion with @ekohlvan, this issue looks with cpu_mode=default which sets "custom (qemu64)" on Libvirt and when tried other cpu modes 
> like host-model(set as custom (EPYC-Rome)) and host passthrough(set as host passthrough) this works correctly.
> Also, For RHEL7 and RHEL8 provisioning cpu_mode=default custom(qemu64) works.

Does this means that for the fix in Foreman we need it release new fog-libvirt and
bump dependency in the Foreman core?

I have done a release and packaged it in upstream nightly.

I hope you don't mind me making the reply public, because I think this design part should be openly discussed.

(In reply to Gaurav Talreja from comment #7)
> RHEL9 provisioning works (for BIOS), when using default cpu_mode which now
> sets cpu type as host passthrough and emulated machine as pc-q35-rhel8.6.0
> on Libvirt, I'd like ask a few observations/questions regarding this,
>  
> 1. As default cpu_mode which now sets cpu type as host passthrough, and when
> using host-passthrough cpu_mode which also sets cpu type as host passthrough
> on Libvirt, so should we keep this cpu_mode=host=passthrough for RHEL9 in
> Libvirt CR, and is this redundant option expected?

Excellent question. I didn't realize that part.

It's this part that defines CPU_MODES as either default, host-model or host-passthrough:

https://github.com/theforeman/foreman/blob/70b2144263dc6c3a1efdd5f26d16952b2e4266c3/app/models/compute_resources/foreman/model/libvirt.rb#L8

Now that the default is passthrough, I wonder if there's still a case for exposing the CPU type at all.

The QEMU documentation on the topic is https://qemu-project.gitlab.io/qemu/system/qemu-cpu-models.html and that recommends using passthrough. So now we're (finally) in line with the recommendation. The only downside is that live migration is not supported, but that's quite specialized and not something we generally support anyway.

After reading that, I'd suggest we drop the option altogether. This simplifies the UI a bit without noticeable loss of functionality.

I've raised this upstream: https://community.theforeman.org/t/drop-cpu-type-option-from-libvirt-compute-resource-provisioning/36267

> 2. RHEL9 provisioning with UEFI still doesn't work with "No bootable device"
> error now and mentioned kernel panic errors aren't seen. And, Libvirt sets
> the firmware BIOS instead of UEFI when PXELoader "Grub2 UEFI" is selected.

I think this is a known limitation now. The PXE Loader has no implication on how the (libvirt) VM is created. It's totally fair to assume to do so, but no code has been written to do so. I've discussed this before, so perhaps a separate RFE is needed.

(In reply to Ewoud Kohl van Wijngaarden from comment #9)
> (In reply to Gaurav Talreja from comment #7)
> > 2. RHEL9 provisioning with UEFI still doesn't work with "No bootable device"
> > error now and mentioned kernel panic errors aren't seen. And, Libvirt sets
> > the firmware BIOS instead of UEFI when PXELoader "Grub2 UEFI" is selected.
> 
> I think this is a known limitation now. The PXE Loader has no implication on
> how the (libvirt) VM is created. It's totally fair to assume to do so, but
> no code has been written to do so. I've discussed this before, so perhaps a
> separate RFE is needed.

The inability to create UEFI VMs using fog-libvirt is tracked in https://github.com/fog/fog-libvirt/issues/128. Depending on the implementation, it may need additional work in Foreman to provide a selection.

(In reply to Ewoud Kohl van Wijngaarden from comment #10)
> The inability to create UEFI VMs using fog-libvirt is tracked in
> https://github.com/fog/fog-libvirt/issues/128. Depending on the
> implementation, it may need additional work in Foreman to provide a
> selection.

I decided to play with this and wrote https://github.com/fog/fog-libvirt/pull/134 to implement it in fog-libvirt. Turns out UEFI and Secure Boot are very easy in the libvirt XML specification.

https://github.com/theforeman/foreman/pull/9965 implements it in Foreman, though I didn't test the UI yet and there is no Secure Boot support yet. By default the firmware (BIOS/UEFI) is derived from the PXE loader, still defaulting to BIOS if no loader is specified.

Verified.

Tested on Satellite 6.15.0 Snap 5.0
Version: rubygem-fog-libvirt-0.12.0-1.el8sat.noarch

Steps:
1. Prepare Satellite with provisioning setup and sync RHEL9 OS and Kickstart repos.
2. Configure Libvirt CR
3. # hammer host create --name="pbvlzmmshx" --location="SEXSGiSTJBd" --organization="kDurvTWwY" --hostgroup="RJvMlS" --compute-resource-id="1" --compute-attributes="cpus=1, memory=6442450944, cpu_mode=default, start=1" --interface="compute_type=bridge,compute_bridge=br-1001" --volume="capacity=10" --provision-method="build" '

Observation:
Provisioning works successfully for RHEL9 on Libvirt CR, and default cpu_mode is set as host-passthrough
I've reported the issues discussed in comment 7, and for UEFI support we got an RFE created by SD/CU already 
https://bugzilla.redhat.com/show_bug.cgi?id=2256234
https://bugzilla.redhat.com/show_bug.cgi?id=2258481


Thanks @ekohlvan for confirming this issues. I greatly appreciate your prompt action with community discussions and PRs

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.15.0 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2024:2010