Bug 2084568

Summary:	Disable 5-level page tables when using -cpu max
Product:	Red Hat Enterprise Linux 9	Reporter:	Richard W.M. Jones <rjones>
Component:	libguestfs	Assignee:	Richard W.M. Jones <rjones>
Status:	CLOSED ERRATA	QA Contact:	YongkuiGuo <yoguo>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	9.1	CC:	lersek, qzhang, rjones, virt-maint, ymankad, ymao, yoguo
Target Milestone:	rc	Keywords:	Triaged, ZStream
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	libguestfs-1.48.4-2.el9	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2085527 (view as bug list)		Environment:
Last Closed:	2022-11-15 09:52:35 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2082806, 2085527
Deadline:	2022-08-15

Description Richard W.M. Jones 2022-05-12 12:43:08 UTC

Description of problem:

In https://bugzilla.redhat.com/show_bug.cgi?id=2082806 we are
tracking an insidious qemu bug which intermittently prevents the
libguestfs appliance from starting.  The symptoms are that SeaBIOS
starts and displays its messages, but the kernel isn't reached.  We
found that the kernel does in fact start, but when it tries to set up
page tables and jump to protected mode it gets a triple fault which
causes the emulated CPU in qemu to reset (qemu exits).

This seems to only affect TCG (not KVM).

This is caused by using -cpu max which enables the "la57" feature
(5-level page tables[0]), and that we can make the problem go away
using -cpu max,la57=off.

Note this is only a workaround for bug 2082806.  We may in future
fix that bug properly (in qemu).  However the proposed workaround
for libguestfs should not have any negative effects.

This bug affects all versions of libguestfs that run qemu with
-cpu max or the libvirt equivalent <cpu mode="maximum"/>, which
includes RHEL 8.7 (not 8.6), and RHEL 9.0 and 9.1.

Version-Release number of selected component (if applicable):

libguestfs-1.44.0-6.el8

How reproducible:

100%

Steps to Reproduce:

These two commands test the libvirt and direct paths (which are
fixed separately):

while LIBGUESTFS_BACKEND_SETTINGS=force_tcg ./run libguestfs-test-tool  >&/tmp/log ; do echo -n . ; done

while LIBGUESTFS_BACKEND=direct LIBGUESTFS_BACKEND_SETTINGS=force_tcg ./run libguestfs-test-tool  >&/tmp/log ; do echo -n . ; done

Upstream bug fix:
https://listman.redhat.com/archives/libguestfs/2022-May/028853.html

Comment 1 Andrew Jones 2022-05-12 13:42:46 UTC

Does libguestfs need to use -cpu max at all? Or would always using the most basic (which hopefully means the most stable) cpu model work? I ask, because I have bug 2066824. My plan was to leave the cortex-a57 cpu model, as well as the max model, because the cortex-a57 should be the most basic/stable. I assumed libguestfs and other products that need TCG would prefer this stable model over 'max' as we don't have resources allocated to support TCG cpu models.

I do realize 'max' is nifty because it can be seamlessly used for both tcg and kvm. I'm just not sure how much of a requirement that seamlessness is. Also, if we start adding a list of features to disable from max, then I'm not 100% sure we'll maintain the seamless KVM support.

Comment 2 Richard W.M. Jones 2022-05-12 14:07:10 UTC

(In reply to Andrew Jones from comment #1)
> Does libguestfs need to use -cpu max at all? Or would always using the most
> basic (which hopefully means the most stable) cpu model work?

The surprising answer is yes, for two reasons, of which one is not obvious:

 - Better performance for RAID-y type stuff.  Probably makes no real difference
   with TCG, but matters with KVM and slightly simpler if we use the same path
   (in libguestfs) for both.

 - We need to run programs from the guest sometimes, and with RHEL 9 guest on
   RHEL 8 host that requires emulating at least x86_64-v2 (bug 2075424).
   -cpu max was one easy way to get this.

Prior to using max, we used -cpu host  or libvirt's host-model (KVM), or
-cpu qemu64 (TCG).

Note the above all applies to x86-64.

Source: https://github.com/libguestfs/libguestfs/blob/master/lib/appliance-cpu.c

> I ask, because
> I have bug 2066824. My plan was to leave the cortex-a57 cpu model, as well
> as the max model, because the cortex-a57 should be the most basic/stable. I
> assumed libguestfs and other products that need TCG would prefer this stable
> model over 'max' as we don't have resources allocated to support TCG cpu
> models.

On aarch64 we currently use -cpu host (KVM) or -cpu cortex-a57 (TCG), so
yes please leave that CPU!  I'm not sure why we're not using -cpu max though,
maybe aarch64 didn't / doesn't support it?

But for aarch64 we're not really focussed at the moment on performance, just
making it work.

> I do realize 'max' is nifty because it can be seamlessly used for both tcg
> and kvm. I'm just not sure how much of a requirement that seamlessness is.
> Also, if we start adding a list of features to disable from max, then I'm
> not 100% sure we'll maintain the seamless KVM support.

Comment 3 YongkuiGuo 2022-05-13 04:17:38 UTC

Tested the following packages on RHEL9.1 host:

qemu-kvm-7.0.0-3.el9.x86_64
libvirt-8.3.0-1.el9.x86_64
kernel-5.14.0-87.el9.x86_64
seabios-bin-1.16.0-2.el9.noarch
libguestfs-1.48.2-2.el9.x86_64


# while LIBGUESTFS_BACKEND_SETTINGS=force_tcg libguestfs-test-tool  >&/tmp/log; do echo -n . ; done
............................................................................................
# while LIBGUESTFS_BACKEND=direct LIBGUESTFS_BACKEND_SETTINGS=force_tcg libguestfs-test-tool  >&/tmp/log; do echo -n . ; done
......................................................................................................

The above two commands work well. This issue has been fixed.

Comment 4 Richard W.M. Jones 2022-05-13 14:22:44 UTC

*** Bug 2084567 has been marked as a duplicate of this bug. ***

Comment 8 YongkuiGuo 2022-06-22 10:53:05 UTC

Verified this bug since the test case for this bug has been automated and passed in the latest compose test.

Comment 14 errata-xmlrpc 2022-11-15 09:52:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: libguestfs security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7958