Bug 2060839

In RHEL-8 and before, RHEL was compiled for the x86_64 baseline. Theoretically someone could be running RHEL on a host that was only capable of using the qemu64 CPU model for guests, though such an old platform would not be in the RHEL hardware support matrix.

In RHEL-9 with our new baseline of x86_64-v2, it is impossible for someone to be running RHEL on a CPU older than Nehalem / Opteron G4 vintage.

IOW, we have a guarantee that anyone using KVM on RHEL-9 will be able to use the Nehalem CPU model with guests. Thus removing older CPU models predating x86_64-v2 has no impact on runnability of guests created on 9

In RHEL-10 the CPU baseline is likely to advance still further.

While qemu64 is the QEMU default, and also libvirt default, all of the applications we ship in RHEL and all of the layered products override this with an explicit CPU model choice.

As long as we ship the very old CPU models though, customers are going to be liable to accidentally make poor decisions by overriding the application's default to pick something unreasonably old.

With all this in mind it would be desirable to delete a number of the very old CPU models that don't affect runability of guests. B

efore we can delete them though, we need to deprecate them, so that we set expectations about ability to live migrate guests using the old CPU models to future RHEL.  ie if we want to delete 'kvm64' in RHEL-10, we need to deprecate in RHEL-9 so that users don't expect to live migrate from 9 -> 10 using 'kvm64'

Given RHEL-9 compilation baseline, we should aim to deprecate anything that is not satisfying the x86_64-v2 ABI per this table:

  https://www.qemu.org/docs/master/system/i386/cpu.html

This means deprecating  '486-v1', 'Conroe-v1', 'Opteron-G1-v1', 'Opteron-G2-v1', 'Opteron-G3-v1', 'Penryn-v1', 'athlon-v1', 'core2duo-v1', 'coreduo-v1', 'kvm32-v1', 'kvm64-v1', 'n270-v1', 'pentium-v1', 'pentium2-v1', 'pentium3-v1', 'phenom-v1', 'qemu32-v1', 'qemu64-v1'.

It might look a little wierd deprecating qemu64-v1, when that is the default in both QEMU and libvirt, but I think that's OK.

As mentioned earlier, we expect all apps to over overridden this default. So by deprecating qemu64, we will flush out any unexpected usage we may have missed.

When it comes to deleting deprecated CPUs in RHEL-10 though, we might still need to special case qemu64 and let it live longer than the others. I believe we'll need to change machine types to have a different default CPU, which will mean we need to wait for old machine types to age out before qemu64 can be deleted in perhaps RHEL-11.

Regarding other architectures:

IIUC, aarch64 by default operates in host-passthrough mode. Not sure if we'd want to deprecate anything (we seem to have some cortex-a?? models, no idea what their state is.)

For s390x, we have a bunch of per-machine-generation models.
- I don't quite remember what RHEL 7 was compiled for (z196? Pre-z12 for sure.)
- RHEL 8 is compiled for z13 (optimized for z14 IIRC).
- RHEL 9 is compiled for z14 (optimized for z15 IIRC).
- We don't know what RHEL 10 will use, although I suspect IBM will ask for another bump.

QEMU/KVM on s390x can easily present older cpus to the guest.

As long as we support RHEL7-as-a-guest, we will need support for the model it had been compiled for, I guess; but we might be able to mark z12 and earlier as deprecated?
@dhildenb : any further ideas?

(In reply to Cornelia Huck from comment #2)
> Regarding other architectures:
> 
> IIUC, aarch64 by default operates in host-passthrough mode. Not sure if we'd
> want to deprecate anything (we seem to have some cortex-a?? models, no idea
> what their state is.)
> 
> For s390x, we have a bunch of per-machine-generation models.
> - I don't quite remember what RHEL 7 was compiled for (z196? Pre-z12 for
> sure.)
> - RHEL 8 is compiled for z13 (optimized for z14 IIRC).
> - RHEL 9 is compiled for z14 (optimized for z15 IIRC).
> - We don't know what RHEL 10 will use, although I suspect IBM will ask for
> another bump.
> 
> QEMU/KVM on s390x can easily present older cpus to the guest.
> 
> As long as we support RHEL7-as-a-guest, we will need support for the model
> it had been compiled for, I guess; but we might be able to mark z12 and
> earlier as deprecated?
> @dhildenb : any further ideas?

As discussed on IRC, RHEL7 should work fine when booting under z14 or z15, as the CPU models are backwards-compatible.

For migrating VMs from RHEL8 -> RHEL9, we'll have to support z13 CPU models. On RHEL9, we should expect to always use a z14+ CPU model, except when migrating from RHEL8 via a z13 CPU model.

Daniel mentioned that we don't really support RHEL7->RHEL8->RHEL9 migration, so we should have sane scenarios where we'd end up using <z13 CPU models.

Do we need a newer equivalent of these cpu types - e.g. something like an x86-64-v2 cpu type which just has enough
to meet the spec and can run on anything vaguely modern?

(In reply to Dr. David Alan Gilbert from comment #6)
> Do we need a newer equivalent of these cpu types - e.g. something like an
> x86-64-v2 cpu type which just has enough
> to meet the spec and can run on anything vaguely modern?

That question is tracked in bug 1919389. Any x86_64-v2 would be 100% match for Nehalem, so adding that alone, was not compelling as it didn't unlock any functional benefit. -v3/-v4 were trickier to decide on a good impl for - especially v4. So ultimately the bug is stalled since -v2 wasn't compelling and -v3/-v4 had too many open questions. 

I don't think we need to have a dependancy between the two bugs, so leave bug 1919389 for the ABI discussion if anyone has more thoughts on that topic.

My suggestion for supported/deprecated status is as follows per architecture:

* x86_64

  Supported CPUs:

      base
      Broadwell
      Broadwell-IBRS
      Broadwell-noTSX
      Broadwell-noTSX-IBRS
      Cascadelake-Server
      Cascadelake-Server-noTSX
      Cooperlake
      Dhyana
      EPYC
      EPYC-IBPB
      EPYC-Milan
      EPYC-Rome
      Haswell
      Haswell-IBRS
      Haswell-noTSX
      Haswell-noTSX-IBRS
      host
      Icelake-Server
      Icelake-Server-noTSX
      IvyBridge
      IvyBridge-IBRS
      KnightsMill
      max
      Nehalem
      Nehalem-IBRS
      Opteron_G4
      Opteron_G5
      SandyBridge
      SandyBridge-IBRS
      Skylake-Client
      Skylake-Client-IBRS
      Skylake-Client-noTSX-IBRS
      Skylake-Server
      Skylake-Server-IBRS
      Skylake-Server-noTSX-IBRS
      Westmere
      Westmere-IBRS

  (I've omitted the many -vNNN versions for brevity)

  Deprecated CPUs:

      486
      athlon
      Conroe
      core2duo
      coreduo
      Denverton
      Icelake-Client-noTSX
      Icelake-Client
      kvm32
      kvm64
      n270
      Opteron_G1
      Opteron_G2
      Opteron_G3
      Penryn
      pentium2
      pentium3
      pentium
      phenom
      qemu32
      qemu64
      Snowridge

* s390x

  Supported CPUs:

      gen15a-base
      gen15a
      gen15b-base
      gen15b
      gen16a-base
      gen16a
      gen16b-base
      gen16b
      max
      z14.2-base
      z14.2
      z14-base
      z14
      z14ZR1-base
      z14ZR1


  Deprecated CPUs:

      qemu
      z10BC.2-base
      z10BC.2
      z10BC-base
      z10BC
      z10EC.2-base
      z10EC.2
      z10EC.3-base
      z10EC.3
      z10EC-base
      z10EC
      z114-base
      z114
      z13.2-base
      z13.2
      z13-base
      z13s-base
      z13s
      z13
      z196.2-base
      z196.2
      z196-base
      z196
      z800-base
      z800
      z890.2-base
      z890.2
      z890.3-base
      z890.3
      z890-base
      z890
      z900.2-base
      z900.2
      z900.3-base
      z900.3
      z900-base
      z900
      z990.2-base
      z990.2
      z990.3-base
      z990.3
      z990.4-base
      z990.4
      z990.5-base
      z990.5
      z990-base
      z990
      z9BC.2-base
      z9BC.2
      z9BC-base
      z9BC
      z9EC.2-base
      z9EC.2
      z9EC.3-base
      z9EC.3
      z9EC-base
      z9EC
      zBC12-base
      zBC12
      zEC12.2-base
      zEC12.2
      zEC12-base
      zEC12


* aarch64

  Supported CPUs:

      a64fx
      cortex-a53
      cortex-a57
      cortex-a72
      max

  Deprecated CPUs:

      cortex-a15


I'm most unclear about aarch64. cortex-a15 is 32-bit and I'm assuming we don't support 32-bit mode guests, not least because alot of hardware lacks ability to even run 32-bit IIUC. I'm furthuer unclear if RHEL aarch64 supports all those 64-bit CPUs I've left supported.

(In reply to Daniel Berrangé from comment #8)
> My suggestion for supported/deprecated status is as follows per architecture:
> 
> * x86_64
> 
>   Supported CPUs:
> 
>       base
>       Broadwell
>       Broadwell-IBRS
>       Broadwell-noTSX
>       Broadwell-noTSX-IBRS
>       Cascadelake-Server
>       Cascadelake-Server-noTSX
>       Cooperlake
>       Dhyana
>       EPYC
>       EPYC-IBPB
>       EPYC-Milan
>       EPYC-Rome
>       Haswell
>       Haswell-IBRS
>       Haswell-noTSX
>       Haswell-noTSX-IBRS
>       host
>       Icelake-Server
>       Icelake-Server-noTSX
>       IvyBridge
>       IvyBridge-IBRS
>       KnightsMill
>       max
>       Nehalem
>       Nehalem-IBRS
>       Opteron_G4
>       Opteron_G5
>       SandyBridge
>       SandyBridge-IBRS
>       Skylake-Client
>       Skylake-Client-IBRS
>       Skylake-Client-noTSX-IBRS
>       Skylake-Server
>       Skylake-Server-IBRS
>       Skylake-Server-noTSX-IBRS
>       Westmere
>       Westmere-IBRS
> 
>   (I've omitted the many -vNNN versions for brevity)
> 
>   Deprecated CPUs:
> 
>       486
>       athlon
>       Conroe
>       core2duo
>       coreduo
>       Denverton
>       Icelake-Client-noTSX
>       Icelake-Client
>       kvm32
>       kvm64
>       n270
>       Opteron_G1
>       Opteron_G2
>       Opteron_G3
>       Penryn
>       pentium2
>       pentium3
>       pentium
>       phenom
>       qemu32
>       qemu64
>       Snowridge

I'm not sure about Snowridge and Denverton; aren't they current Atom class embedded things?
(Aiming for base stations and the like)

> * aarch64
> 
>   Supported CPUs:
> 
>       a64fx
>       cortex-a53
>       cortex-a57
>       cortex-a72
>       max
> 
>   Deprecated CPUs:
> 
>       cortex-a15
> 
> 
> I'm most unclear about aarch64. cortex-a15 is 32-bit and I'm assuming we
> don't support 32-bit mode guests, not least because alot of hardware lacks
> ability to even run 32-bit IIUC. I'm furthuer unclear if RHEL aarch64
> supports all those 64-bit CPUs I've left supported.

a64fx I think is Fujitsu's supercomputer.
a53/a57 are the low end and A72 is the high end; you often get them paired
for big.little pairings.

(In reply to Daniel Berrangé from comment #8)
> My suggestion for supported/deprecated status is as follows per architecture:

(...)

> * aarch64
> 
>   Supported CPUs:
> 
>       a64fx
>       cortex-a53
>       cortex-a57
>       cortex-a72
>       max
> 
>   Deprecated CPUs:
> 
>       cortex-a15
> 
> 
> I'm most unclear about aarch64. cortex-a15 is 32-bit and I'm assuming we
> don't support 32-bit mode guests, not least because alot of hardware lacks
> ability to even run 32-bit IIUC. I'm furthuer unclear if RHEL aarch64
> supports all those 64-bit CPUs I've left supported.

IIRC, qemu+kvm on aarch64 uses the host model (by default? or always?) Hopefully @drjones can help here.

(In reply to Dr. David Alan Gilbert from comment #9)
> (In reply to Daniel Berrangé from comment #8)
> > My suggestion for supported/deprecated status is as follows per architecture:
> > 
> > * x86_64

snip

> > 
> >   Deprecated CPUs:
> > 
> >       486
> >       athlon
> >       Conroe
> >       core2duo
> >       coreduo
> >       Denverton
> >       Icelake-Client-noTSX
> >       Icelake-Client
> >       kvm32
> >       kvm64
> >       n270
> >       Opteron_G1
> >       Opteron_G2
> >       Opteron_G3
> >       Penryn
> >       pentium2
> >       pentium3
> >       pentium
> >       phenom
> >       qemu32
> >       qemu64
> >       Snowridge
> 
> I'm not sure about Snowridge and Denverton; aren't they current Atom class
> embedded things?
> (Aiming for base stations and the like)

Opps, those two are a mistake. They can support -v2, so should have been included.

> > * aarch64
> > 
> >   Supported CPUs:
> > 
> >       a64fx
> >       cortex-a53
> >       cortex-a57
> >       cortex-a72
> >       max
> > 
> >   Deprecated CPUs:
> > 
> >       cortex-a15
> > 
> > 
> > I'm most unclear about aarch64. cortex-a15 is 32-bit and I'm assuming we
> > don't support 32-bit mode guests, not least because alot of hardware lacks
> > ability to even run 32-bit IIUC. I'm furthuer unclear if RHEL aarch64
> > supports all those 64-bit CPUs I've left supported.
> 
> a64fx I think is Fujitsu's supercomputer.
> a53/a57 are the low end and A72 is the high end; you often get them paired
> for big.little pairings.

(In reply to Cornelia Huck from comment #10)

> IIRC, qemu+kvm on aarch64 uses the host model (by default? or always?)
> Hopefully @drjones can help here.

Hmm, if host model is mandatory we should deprecate all, but none the less leave at least 1 available indefinitely in future RHEL for sake of TCG which is needed in some CI/build scenarios

(In reply to Daniel Berrangé from comment #8)
> My suggestion for supported/deprecated status is as follows per architecture:
[...]
> * s390x
> 
>   Supported CPUs:
> 
>       gen15a-base
>       gen15a
>       gen15b-base
>       gen15b
>       gen16a-base
>       gen16a
>       gen16b-base
>       gen16b
>       max
>       z14.2-base
>       z14.2
>       z14-base
>       z14
>       z14ZR1-base
>       z14ZR1

Looking at https://access.redhat.com/solutions/certified-hypervisors , it seems like we generally support up to RHEL N-2 running on RHEL N on the host. I.e. in the RHEL version that comes after RHEL 9, we will likely still support RHEL 8 in the guest. Thus I think we shouldn't deprecate the z13 CPUs yet and add them to the list of supported CPUs instead?

>   Deprecated CPUs:
> 
>       qemu

The "qemu" CPU is different on s390x, compared to x86. It gets new features with newer machine types, and is required for running recent RHEL versions with TCG, so I think we should rather not deprecate it.

>       z10BC.2-base
>       z10BC.2
>       z10BC-base
>       z10BC
>       z10EC.2-base
>       z10EC.2
>       z10EC.3-base
>       z10EC.3
>       z10EC-base
>       z10EC
>       z114-base
>       z114
>       z13.2-base
>       z13.2
>       z13-base
>       z13s-base
>       z13s
>       z13
>       z196.2-base
>       z196.2
>       z196-base
>       z196
>       z800-base
>       z800
>       z890.2-base
>       z890.2
>       z890.3-base
>       z890.3
>       z890-base
>       z890
>       z900.2-base
>       z900.2
>       z900.3-base
>       z900.3
>       z900-base
>       z900
>       z990.2-base
>       z990.2
>       z990.3-base
>       z990.3
>       z990.4-base
>       z990.4
>       z990.5-base
>       z990.5
>       z990-base
>       z990
>       z9BC.2-base
>       z9BC.2
>       z9BC-base
>       z9BC
>       z9EC.2-base
>       z9EC.2
>       z9EC.3-base
>       z9EC.3
>       z9EC-base
>       z9EC

Note that all the z9* and z8* CPUs are already disabled in the downstream qemu-kvm binary already (they are still compiled in, but are not selectable).

>       zBC12-base
>       zBC12
>       zEC12.2-base
>       zEC12.2
>       zEC12-base
>       zEC12

(In reply to Thomas Huth from comment #12)
> (In reply to Daniel Berrangé from comment #8)
> > My suggestion for supported/deprecated status is as follows per architecture:
> [...]
> > * s390x
> > 
> >   Supported CPUs:
> > 
> >       gen15a-base
> >       gen15a
> >       gen15b-base
> >       gen15b
> >       gen16a-base
> >       gen16a
> >       gen16b-base
> >       gen16b
> >       max
> >       z14.2-base
> >       z14.2
> >       z14-base
> >       z14
> >       z14ZR1-base
> >       z14ZR1
> 
> Looking at https://access.redhat.com/solutions/certified-hypervisors , it
> seems like we generally support up to RHEL N-2 running on RHEL N on the
> host. I.e. in the RHEL version that comes after RHEL 9, we will likely still
> support RHEL 8 in the guest. Thus I think we shouldn't deprecate the z13
> CPUs yet and add them to the list of supported CPUs instead?

What CPU the guest is compiled for doesn't actually matter for this scenario.  A guest OS compiled against z13, can happily run in a VM exposing a z14 CPU model, there's no need to configure it with a z13 per comment #3.

> >   Deprecated CPUs:
> > 
> >       qemu
> 
> The "qemu" CPU is different on s390x, compared to x86. It gets new features
> with newer machine types, and is required for running recent RHEL versions
> with TCG, so I think we should rather not deprecate it.

While we do ship it, TCG is considered an unsupported feature from POV of production, so I think it is appropriate to deprecate any CPUs that are only intended for usage with TCG.

AFAIK though, this 'qemu' CPU model is obsolete even for TCG, since we introduced 'max' which sames same semantics as 'qemu' on TCG, while matching 'host' on KVM.


l> 
> >       z10BC.2-base
> >       z10BC.2
> >       z10BC-base
> >       z10BC
> >       z10EC.2-base
> >       z10EC.2
> >       z10EC.3-base
> >       z10EC.3
> >       z10EC-base
> >       z10EC
> >       z114-base
> >       z114
> >       z13.2-base
> >       z13.2
> >       z13-base
> >       z13s-base
> >       z13s
> >       z13
> >       z196.2-base
> >       z196.2
> >       z196-base
> >       z196
> >       z800-base
> >       z800
> >       z890.2-base
> >       z890.2
> >       z890.3-base
> >       z890.3
> >       z890-base
> >       z890
> >       z900.2-base
> >       z900.2
> >       z900.3-base
> >       z900.3
> >       z900-base
> >       z900
> >       z990.2-base
> >       z990.2
> >       z990.3-base
> >       z990.3
> >       z990.4-base
> >       z990.4
> >       z990.5-base
> >       z990.5
> >       z990-base
> >       z990
> >       z9BC.2-base
> >       z9BC.2
> >       z9BC-base
> >       z9BC
> >       z9EC.2-base
> >       z9EC.2
> >       z9EC.3-base
> >       z9EC.3
> >       z9EC-base
> >       z9EC
> 
> Note that all the z9* and z8* CPUs are already disabled in the downstream
> qemu-kvm binary already (they are still compiled in, but are not selectable).

Interesting. At least when I tested with libvirt 'virsh domcapabilities' still reported that all these CPUs exist. If its impossible to actually use them though, we could skip deprecation and go straight fully disabling them at compile time, to stop them appearing in 'query-cpu-definitions' so libvirt won't report them as available.

(In reply to Daniel Berrangé from comment #11)
...
> > > * aarch64
> > > 
> > >   Supported CPUs:
> > > 
> > >       a64fx
> > >       cortex-a53
> > >       cortex-a57
> > >       cortex-a72
> > >       max
> > > 
> > >   Deprecated CPUs:
> > > 
> > >       cortex-a15
> > > 
> > > 
> > > I'm most unclear about aarch64. cortex-a15 is 32-bit and I'm assuming we
> > > don't support 32-bit mode guests, not least because alot of hardware lacks
> > > ability to even run 32-bit IIUC. I'm furthuer unclear if RHEL aarch64
> > > supports all those 64-bit CPUs I've left supported.
> > 
> > a64fx I think is Fujitsu's supercomputer.
> > a53/a57 are the low end and A72 is the high end; you often get them paired
> > for big.little pairings.
> 
> (In reply to Cornelia Huck from comment #10)
> 
> > IIRC, qemu+kvm on aarch64 uses the host model (by default? or always?)
> > Hopefully @drjones can help here.
> 
> Hmm, if host model is mandatory we should deprecate all, but none the less
> leave at least 1 available indefinitely in future RHEL for sake of TCG which
> is needed in some CI/build scenarios


Yes, we should drop a64fx, cortex-a7, cortex-a53, cortex-a72, and cortex-a15 (cortex-a7 is coming with the latest rhel9 qemu rebase). By coincidence, I just brought this up while reviewing that rhel9 rebase (I'm not sure why we've never considered dropping extra CPU models before... Anyway, dropping them for 9.0 would be nice.)

I suggest we keep cortex-a57, as it serves as a "base" AArch64 cpu model that can be used for safer, basic TCG use, i.e. libguestfs should use that one. I suggest we keep 'max', because it allows us to write tests for KVM which will also run seamlessly without KVM ('max' works with both KVM and TCG). Also, if ever want to use our QEMU for a more featureful cpu model for TCG, then 'max' is the one we want. Of course we need 'host' for typical KVM use.

Thanks,
drew

(In reply to Daniel Berrangé from comment #13)
> (In reply to Thomas Huth from comment #12)
> > (In reply to Daniel Berrangé from comment #8)
> > > My suggestion for supported/deprecated status is as follows per architecture:

> > >   Deprecated CPUs:
> > > 
> > >       qemu
> > 
> > The "qemu" CPU is different on s390x, compared to x86. It gets new features
> > with newer machine types, and is required for running recent RHEL versions
> > with TCG, so I think we should rather not deprecate it.
> 
> While we do ship it, TCG is considered an unsupported feature from POV of
> production, so I think it is appropriate to deprecate any CPUs that are only
> intended for usage with TCG.
> 
> AFAIK though, this 'qemu' CPU model is obsolete even for TCG, since we
> introduced 'max' which sames same semantics as 'qemu' on TCG, while matching
> 'host' on KVM.

I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs) usage, as 'max' might contain experimental features for TCG. No idea what libguestfs currently uses, though.

We also need to answer the question what 'deprecated' actually means. Does it mean "If you are using this to run a KVM guest, you really want to switch to something else; it may be removed in the future.", or "It will be removed in the future."? Can a "deprecated" cpu model be unsupported, but still around for TCG usage (which is unsupported for normal use, but used by libguestfs?)

(In reply to Cornelia Huck from comment #15)
> (In reply to Daniel Berrangé from comment #13)
> > (In reply to Thomas Huth from comment #12)
> > > (In reply to Daniel Berrangé from comment #8)
> > > > My suggestion for supported/deprecated status is as follows per architecture:
> 
> > > >   Deprecated CPUs:
> > > > 
> > > >       qemu
> > > 
> > > The "qemu" CPU is different on s390x, compared to x86. It gets new features
> > > with newer machine types, and is required for running recent RHEL versions
> > > with TCG, so I think we should rather not deprecate it.
> > 
> > While we do ship it, TCG is considered an unsupported feature from POV of
> > production, so I think it is appropriate to deprecate any CPUs that are only
> > intended for usage with TCG.
> > 
> > AFAIK though, this 'qemu' CPU model is obsolete even for TCG, since we
> > introduced 'max' which sames same semantics as 'qemu' on TCG, while matching
> > 'host' on KVM.
> 
> I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> usage, as 'max' might contain experimental features for TCG. No idea what
> libguestfs currently uses, though.

What do you mean by "experimental" here ?  If we're not confident in the quality of the TCG impl of a CPU feature, IMHO, it shouldn't be enabled.

When QEMU added 'max' model across all targets, the intent was that this is something that applications can standardize on using by default across both TCG and KVM, when they don't need live migration compat.

IOW, 'max' needs to be reliably usable by any application.

> We also need to answer the question what 'deprecated' actually means. Does
> it mean "If you are using this to run a KVM guest, you really want to switch
> to something else; it may be removed in the future.", or "It will be removed
> in the future."? Can a "deprecated" cpu model be unsupported, but still
> around for TCG usage (which is unsupported for normal use, but used by
> libguestfs?)

I consider "deprecated" means this is a non-recommended configuration, and is liable (but not guaranteed) to be removed in the next major release, without further warning.

(In reply to Daniel Berrangé from comment #16)
> (In reply to Cornelia Huck from comment #15)
> > (In reply to Daniel Berrangé from comment #13)
> > > (In reply to Thomas Huth from comment #12)
> > > > (In reply to Daniel Berrangé from comment #8)
> > > > > My suggestion for supported/deprecated status is as follows per architecture:
> > 
> > > > >   Deprecated CPUs:
> > > > > 
> > > > >       qemu
> > > > 
> > > > The "qemu" CPU is different on s390x, compared to x86. It gets new features
> > > > with newer machine types, and is required for running recent RHEL versions
> > > > with TCG, so I think we should rather not deprecate it.
> > > 
> > > While we do ship it, TCG is considered an unsupported feature from POV of
> > > production, so I think it is appropriate to deprecate any CPUs that are only
> > > intended for usage with TCG.
> > > 
> > > AFAIK though, this 'qemu' CPU model is obsolete even for TCG, since we
> > > introduced 'max' which sames same semantics as 'qemu' on TCG, while matching
> > > 'host' on KVM.
> > 
> > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > usage, as 'max' might contain experimental features for TCG. No idea what
> > libguestfs currently uses, though.
> 
> What do you mean by "experimental" here ?  If we're not confident in the
> quality of the TCG impl of a CPU feature, IMHO, it shouldn't be enabled.
> 
> When QEMU added 'max' model across all targets, the intent was that this is
> something that applications can standardize on using by default across both
> TCG and KVM, when they don't need live migration compat.
> 
> IOW, 'max' needs to be reliably usable by any application.

IIRC, the 'max' model was added for s390x a bit later than for the other architectures.

The commit that introduced it (c61177881cbd ("s390x/cpumodel: Add "-cpu max" support")) explicitly talks about "experimental" CPU features. I'm not sure whether that still holds true; maybe @dhildenb knows more.

(In reply to Cornelia Huck from comment #17)
> (In reply to Daniel Berrangé from comment #16)
> > (In reply to Cornelia Huck from comment #15)
> > > (In reply to Daniel Berrangé from comment #13)
> > > > (In reply to Thomas Huth from comment #12)
> > > > > (In reply to Daniel Berrangé from comment #8)
> > > > > > My suggestion for supported/deprecated status is as follows per architecture:
> > > 
> > > > > >   Deprecated CPUs:
> > > > > > 
> > > > > >       qemu
> > > > > 
> > > > > The "qemu" CPU is different on s390x, compared to x86. It gets new features
> > > > > with newer machine types, and is required for running recent RHEL versions
> > > > > with TCG, so I think we should rather not deprecate it.
> > > > 
> > > > While we do ship it, TCG is considered an unsupported feature from POV of
> > > > production, so I think it is appropriate to deprecate any CPUs that are only
> > > > intended for usage with TCG.
> > > > 
> > > > AFAIK though, this 'qemu' CPU model is obsolete even for TCG, since we
> > > > introduced 'max' which sames same semantics as 'qemu' on TCG, while matching
> > > > 'host' on KVM.
> > > 
> > > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > > usage, as 'max' might contain experimental features for TCG. No idea what
> > > libguestfs currently uses, though.
> > 
> > What do you mean by "experimental" here ?  If we're not confident in the
> > quality of the TCG impl of a CPU feature, IMHO, it shouldn't be enabled.
> > 
> > When QEMU added 'max' model across all targets, the intent was that this is
> > something that applications can standardize on using by default across both
> > TCG and KVM, when they don't need live migration compat.
> > 
> > IOW, 'max' needs to be reliably usable by any application.
> 
> IIRC, the 'max' model was added for s390x a bit later than for the other
> architectures.
> 
> The commit that introduced it (c61177881cbd ("s390x/cpumodel: Add "-cpu max"
> support")) explicitly talks about "experimental" CPU features. I'm not sure
> whether that still holds true; maybe @dhildenb knows more.

We used it as a mechanism to test new TCG features we implemented before being able to bump the base model up to a z13. Nowadays, the only difference between "qemu" and "max" is one feature included in "max" that ends up triggering a QEMU warning. We could fix that for good and make "max" correspond exactly "qemu" under TCG.

(In reply to Cornelia Huck from comment #15)
> I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> usage, as 'max' might contain experimental features for TCG. No idea what
> libguestfs currently uses, though.

libguestfs will use 'max' for most architecture targets, including s390x.

(In reply to Daniel Berrangé from comment #19)
> (In reply to Cornelia Huck from comment #15)
> > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > usage, as 'max' might contain experimental features for TCG. No idea what
> > libguestfs currently uses, though.
> 
> libguestfs will use 'max' for most architecture targets, including s390x.

Why?

In my opinion, since we don't actively develop/test TCG, then we should be very conservative in our use of it. If libguestfs can fully function with the most basic CPU model TCG supports (which is likely named differently for different architectures), then it should use that basic model in order to reduce our need to debug TCG issues. The 'max' CPU type may not have "experimental" features, but it has most/all features, including the most recently added features, which means it has more potential for bugs. This is why I recommended in comment 14 to use the cortex-a57 for libguestfs for AArch64, but to keep 'max' CPU type as well for other reasons.

(In reply to Andrew Jones from comment #20)
> (In reply to Daniel Berrangé from comment #19)
> > (In reply to Cornelia Huck from comment #15)
> > > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > > usage, as 'max' might contain experimental features for TCG. No idea what
> > > libguestfs currently uses, though.
> > 
> > libguestfs will use 'max' for most architecture targets, including s390x.
> 
> Why?

Speculating the answer to my own question: A good 'why' may be that libguestfs can then create a single command line that will work on KVM and fallback to TCG. That's a reasonable thing to do, but also makes me wonder if we shouldn't have a 'min' CPU type that works on both KVM and TCG.

> 
> In my opinion, since we don't actively develop/test TCG, then we should be
> very conservative in our use of it. If libguestfs can fully function with
> the most basic CPU model TCG supports (which is likely named differently for
> different architectures), then it should use that basic model in order to
> reduce our need to debug TCG issues. The 'max' CPU type may not have
> "experimental" features, but it has most/all features, including the most
> recently added features, which means it has more potential for bugs. This is
> why I recommended in comment 14 to use the cortex-a57 for libguestfs for
> AArch64, but to keep 'max' CPU type as well for other reasons.

(In reply to Andrew Jones from comment #20)
> (In reply to Daniel Berrangé from comment #19)
> > (In reply to Cornelia Huck from comment #15)
> > > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > > usage, as 'max' might contain experimental features for TCG. No idea what
> > > libguestfs currently uses, though.
> > 
> > libguestfs will use 'max' for most architecture targets, including s390x.
> 
> Why?
> 
> In my opinion, since we don't actively develop/test TCG, then we should be
> very conservative in our use of it. If libguestfs can fully function with
> the most basic CPU model TCG supports (which is likely named differently for
> different architectures), then it should use that basic model in order to
> reduce our need to debug TCG issues. The 'max' CPU type may not have
> "experimental" features, but it has most/all features, including the most
> recently added features, which means it has more potential for bugs. This is
> why I recommended in comment 14 to use the cortex-a57 for libguestfs for
> AArch64, but to keep 'max' CPU type as well for other reasons.

Note, I said 'most' not 'all'  architectures. aarch64 is currently an outlier where libguestfs doesn't use max. This isn't a desirable situation though in general. We want a portable CPU model that applications can unconditionally use by default in all scenarios where live migration isn't needed, and 'max' is intended to fill that role from upstream POV. 

If 'max' has bugs, then so be it, those need to be found & fixed - if we never use it, we'll never find them. Libguestfs' test suite has a strong track record of finding bugs in all areas of QEMU, not just CPU models, so there's nothing special about CPUs in this regard. We'll usually find these bugs in Fedora rawhide first.

(In reply to Daniel Berrangé from comment #22)
> (In reply to Andrew Jones from comment #20)
> > (In reply to Daniel Berrangé from comment #19)
> > > (In reply to Cornelia Huck from comment #15)
> > > > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > > > usage, as 'max' might contain experimental features for TCG. No idea what
> > > > libguestfs currently uses, though.
> > > 
> > > libguestfs will use 'max' for most architecture targets, including s390x.
> > 
> > Why?
> > 
> > In my opinion, since we don't actively develop/test TCG, then we should be
> > very conservative in our use of it. If libguestfs can fully function with
> > the most basic CPU model TCG supports (which is likely named differently for
> > different architectures), then it should use that basic model in order to
> > reduce our need to debug TCG issues. The 'max' CPU type may not have
> > "experimental" features, but it has most/all features, including the most
> > recently added features, which means it has more potential for bugs. This is
> > why I recommended in comment 14 to use the cortex-a57 for libguestfs for
> > AArch64, but to keep 'max' CPU type as well for other reasons.
> 
> Note, I said 'most' not 'all'  architectures. aarch64 is currently an
> outlier where libguestfs doesn't use max. This isn't a desirable situation
> though in general. We want a portable CPU model that applications can
> unconditionally use by default in all scenarios where live migration isn't
> needed, and 'max' is intended to fill that role from upstream POV. 
> 
> If 'max' has bugs, then so be it, those need to be found & fixed - if we
> never use it, we'll never find them. Libguestfs' test suite has a strong
> track record of finding bugs in all areas of QEMU, not just CPU models, so
> there's nothing special about CPUs in this regard. We'll usually find these
> bugs in Fedora rawhide first.

I'll send patches to cleanup max vs. qemu in s390x.

For AArch64, we'll do the work under bug 2066824

(In reply to Andrew Jones from comment #24)
> For AArch64, we'll do the work under bug 2066824

I was intending to send a patch series to cover all architectures in one go, so we have a consistent approach across all.

(In reply to Daniel Berrangé from comment #25)
> (In reply to Andrew Jones from comment #24)
> > For AArch64, we'll do the work under bug 2066824
> 
> I was intending to send a patch series to cover all architectures in one go,
> so we have a consistent approach across all.

But AArch64 has the freedom to remove, rather than deprecate, CPU types that it doesn't want. Also, each architecture / machine type has different mechanisms for selecting supported CPU types, so I'm not sure how we can do this consistently across arch, unless we just want to "taint" a list of CPU types (but, as said, AArch64 doesn't want to taint/deprecate, it wants to remove).

Daniel, should I assign this BZ to you since it seems you are working on it?
Also, I think now we will get the change in 9.1.

(In reply to Thomas Huth from comment #12)
> (In reply to Daniel Berrangé from comment #8)
> > My suggestion for supported/deprecated status is as follows per architecture:
> [...]
> > * s390x
> > 
> >   Supported CPUs:
> > 
> >       gen15a-base
...
> >       z14ZR1
> 
> Looking at https://access.redhat.com/solutions/certified-hypervisors , it
> seems like we generally support up to RHEL N-2 running on RHEL N on the
> host. I.e. in the RHEL version that comes after RHEL 9, we will likely still
> support RHEL 8 in the guest. Thus I think we shouldn't deprecate the z13
> CPUs yet and add them to the list of supported CPUs instead?
> 
> >   Deprecated CPUs:
> > 
> >       qemu
> 
> The "qemu" CPU is different on s390x, compared to x86. It gets new features
> with newer machine types, and is required for running recent RHEL versions
> with TCG, so I think we should rather not deprecate it.
> 
> >       z10BC.2-base
...
> >       z9EC
> 
> Note that all the z9* and z8* CPUs are already disabled in the downstream
> qemu-kvm binary already (they are still compiled in, but are not selectable).
> 
> >       zBC12-base
...
> >       zEC12

Thomas, IIRC we recently discussed this with IBM, and given that we support RHEL 7.9 guests, it was decided that RH supports all CPU models since z196 and that's why we implemented the test suite here [1]. Maybe I'm missing something and this isn't about KVM but TCG?

[1]https://github.com/autotest/tp-qemu/pull/3271/files

(In reply to smitterl from comment #31)
> 
> Thomas, IIRC we recently discussed this with IBM, and given that we support
> RHEL 7.9 guests, it was decided that RH supports all CPU models since z196
> and that's why we implemented the test suite here [1]. Maybe I'm missing
> something and this isn't about KVM but TCG?

This is primarily about KVM. While it does also apply to TCG, any TCG usage was already considered unsupported in production.

My understanding is that the s390x architecture is forwards compatible, so even if RHEL-7 was released with z196 as a baseline, it will still work if run inside a guest with a newer model like z14, and so this doesn't impact supportability of RHEL-7 as a guest.

(In reply to David Hildenbrand from comment #23)
> (In reply to Daniel Berrangé from comment #22)
> > (In reply to Andrew Jones from comment #20)
> > > (In reply to Daniel Berrangé from comment #19)
> > > > (In reply to Cornelia Huck from comment #15)
> > > > > I'm not sure we want to use 'max' instead of 'qemu' for TCG (libguestfs)
> > > > > usage, as 'max' might contain experimental features for TCG. No idea what
> > > > > libguestfs currently uses, though.
> > > > 
> > > > libguestfs will use 'max' for most architecture targets, including s390x.
> > > 
> > > Why?
> > > 
> > > In my opinion, since we don't actively develop/test TCG, then we should be
> > > very conservative in our use of it. If libguestfs can fully function with
> > > the most basic CPU model TCG supports (which is likely named differently for
> > > different architectures), then it should use that basic model in order to
> > > reduce our need to debug TCG issues. The 'max' CPU type may not have
> > > "experimental" features, but it has most/all features, including the most
> > > recently added features, which means it has more potential for bugs. This is
> > > why I recommended in comment 14 to use the cortex-a57 for libguestfs for
> > > AArch64, but to keep 'max' CPU type as well for other reasons.
> > 
> > Note, I said 'most' not 'all'  architectures. aarch64 is currently an
> > outlier where libguestfs doesn't use max. This isn't a desirable situation
> > though in general. We want a portable CPU model that applications can
> > unconditionally use by default in all scenarios where live migration isn't
> > needed, and 'max' is intended to fill that role from upstream POV. 
> > 
> > If 'max' has bugs, then so be it, those need to be found & fixed - if we
> > never use it, we'll never find them. Libguestfs' test suite has a strong
> > track record of finding bugs in all areas of QEMU, not just CPU models, so
> > there's nothing special about CPUs in this regard. We'll usually find these
> > bugs in Fedora rawhide first.
> 
> I'll send patches to cleanup max vs. qemu in s390x.

FWIW, these patches are now upstream:

d98ed7d96e s390x/cpu_models: make "max" match the unmodified "qemu" CPU model under TCG
4756b106b3 s390x/cpu_models: drop "msa5" from the TCG "max" model

(In reply to smitterl from comment #31)
> (In reply to Thomas Huth from comment #12)
> > (In reply to Daniel Berrangé from comment #8)
... 
> > >   Deprecated CPUs:
...
> > >       zBC12-base
> ...
> > >       zEC12
> 
> Thomas, IIRC we recently discussed this with IBM, and given that we support
> RHEL 7.9 guests, it was decided that RH supports all CPU models since z196
> and that's why we implemented the test suite here [1]. Maybe I'm missing
> something and this isn't about KVM but TCG?

Sorry, I don't quite get your question. This BZ is about marking CPUs as deprecated, but not about disabling them yet (i.e. they'll continue to work as supported during the RHEL 9 cycle). By the time we release a new major version after RHEL 9, the z114 and z196 will be out of support since a couple of years (see https://www.ibm.com/support/pages/system/files/inline-files/IBM%20Mainframe%20Life%20Cycle%20History%20V2.8%20-%20April%205%202022.pdf ) so I think it should be OK to remove them in the next major release?

(In reply to Thomas Huth from comment #34)
> (In reply to smitterl from comment #31)
> > (In reply to Thomas Huth from comment #12)
> > > (In reply to Daniel Berrangé from comment #8)
> ... 
> > > >   Deprecated CPUs:
> ...
> > > >       zBC12-base
> > ...
> > > >       zEC12
> > 
> > Thomas, IIRC we recently discussed this with IBM, and given that we support
> > RHEL 7.9 guests, it was decided that RH supports all CPU models since z196
> > and that's why we implemented the test suite here [1]. Maybe I'm missing
> > something and this isn't about KVM but TCG?
> 
> Sorry, I don't quite get your question. This BZ is about marking CPUs as
> deprecated, but not about disabling them yet (i.e. they'll continue to work
> as supported during the RHEL 9 cycle). By the time we release a new major
> version after RHEL 9, the z114 and z196 will be out of support since a
> couple of years (see
> https://www.ibm.com/support/pages/system/files/inline-files/
> IBM%20Mainframe%20Life%20Cycle%20History%20V2.8%20-%20April%205%202022.pdf )
> so I think it should be OK to remove them in the next major release?

Thanks Thomas. I think I just misunderstood the term "deprecated" in this context. By Merriam-Webster [1] it can have two meanings here and I think I understood a) where you are using it with b):

a) to withdraw official support
b) to discourage the use in favor of a newer or better alternative

[1]https://www.merriam-webster.com/dictionary/deprecate

Hi Daniel,


Could you please give some suggestion about testing these deprecated CPU models?
Does QE need to test them on RHEL9.1? 
Or in other words, do we have a plan to fix the issues which are caused by deprecated CPU models in future?
 
Thanks.


Best regards
Liu Nana

Daniel, along with answering Nana's question, can you please update the DTM as well?
Thanks

(In reply to liunana from comment #36)
> Hi Daniel,
> 
> 
> Could you please give some suggestion about testing these deprecated CPU
> models?

'virsh domcapabilities' should report a deprecated flag against many CPUs

If you configure a guest VM to use a deprecated CPU, then /var/log/libvirt/qemu/$GUEST.log should get a warning message printed on startup. Also 'virsh dominfo' should report the VM as tainted and reported it using a deprecated CPU.

> Does QE need to test them on RHEL9.1? 

Well, test that deprecation flags/warnings are appearing.

> Or in other words, do we have a plan to fix the issues which are caused by
> deprecated CPU models in future?

There shouldn't be any functional problems caused by deprecating CPUS in RHEL-9. They remain fully functional.

It'll merely result in warning messages in log files and other places, as a way to alert users that their config is undesirable.

Mirek, Daniel, it seems the MR is still missing review Ack.
Can you please help by updating the DTM to accommodate the review time required?
Thanks

QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Test Env:
     5.14.0-127.el9.x86_64
     qemu-kvm-7.0.0-8.el9.x86_64
     libvirt-daemon-8.5.0-1.el9.x86_64
     Model name:            AMD EPYC 7302P 16-Core Processor


Test steps:
1. check CPU model by qemu command line, didn't see any deprecated info and the deprecated CPU models are still in Available CPUs.
# /usr/libexec/qemu-kvm -cpu help
Available CPUs:
x86 486                   (alias configured by machine type)
x86 486-v1                
x86 Broadwell             (alias configured by machine type)
x86 Broadwell-IBRS        (alias of Broadwell-v3)
......

x86 Nehalem               (alias configured by machine type)
x86 Nehalem-IBRS          (alias of Nehalem-v2)
x86 Nehalem-v1            Intel Core i7 9xx (Nehalem Class Core i7)
x86 Nehalem-v2            Intel Core i7 9xx (Nehalem Core i7, IBRS update)
x86 Opteron_G1            (alias configured by machine type)
x86 Opteron_G1-v1         AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2            (alias configured by machine type)
x86 Opteron_G2-v1         AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3            (alias configured by machine type)
x86 Opteron_G3-v1         AMD Opteron 23xx (Gen 3 Class Opteron)
........


2. check deprecated flag with libvirt. results are expected.
# virsh domcapabilities | grep model | grep deprecated
      <model usable='yes' deprecated='yes'>qemu64</model>
      <model usable='yes' deprecated='yes'>qemu32</model>
      <model usable='no' deprecated='yes'>phenom</model>
      <model usable='yes' deprecated='yes'>pentium3</model>
      <model usable='yes' deprecated='yes'>pentium2</model>
      <model usable='yes' deprecated='yes'>pentium</model>
      <model usable='no' deprecated='yes'>n270</model>
      <model usable='yes' deprecated='yes'>kvm64</model>
      <model usable='yes' deprecated='yes'>kvm32</model>
      <model usable='no' deprecated='yes'>coreduo</model>
      <model usable='no' deprecated='yes'>core2duo</model>
      <model usable='no' deprecated='yes'>athlon</model>
      <model usable='yes' deprecated='yes'>Penryn</model>
      <model usable='yes' deprecated='yes'>Opteron_G3</model>
      <model usable='yes' deprecated='yes'>Opteron_G2</model>
      <model usable='yes' deprecated='yes'>Opteron_G1</model>
      <model usable='no' deprecated='yes'>Icelake-Client-noTSX</model>
      <model usable='no' deprecated='yes'>Icelake-Client</model>
      <model usable='yes' deprecated='yes'>Conroe</model>
      <model usable='yes' deprecated='yes'>486</model>


3. Boot qemu with deprecated CPU model, can get expected warning info:
# /usr/libexec/qemu-kvm -M q35 -monitor stdio -cpu Opteron_G3
QEMU 7.0.0 monitor - type 'help' for more information
(qemu) qemu-kvm: warning: CPU model Opteron_G3-x86_64-cpu is deprecated -- use at least 'Nehalem' / 'Opteron_G4', or 'host' / 'max'


4.  Boot RHEL9.1 guest with deprecated CPU model, guest will crash with unspported log.
[    1.220625] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00007f00
Fatal glibc error: CPU does not support x86-64-v2
[    1.222200] CPU: 3 PID: 1 Comm: init Not tainted 5.14.0-125.el9.x86_64 #1


5. Boot RHEL8.7 guest with deprecated CPU model 'Opteron_G1', guest can boot up sccessfully.
Guest kernel: 4.18.0-408.el8.x86_64


6. Boot Win2022 guest with deprecated CPU model 'Opteron_G2', guest can boot up sccessfully.



Hi Daniel,

Could you please help to check the step 1? Seems the results aren't the expected.
And for booting a VM with deprecated CPU model, are the steps 4,5,6 the expected results?

Please check this, thanks.


Best regards
Nana Liu

Test environment:
Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
qemu version: qemu-kvm-7.0.0-8.el9.s390x
libvirt version: libvirt-8.3.0-1.el9.s390x

1. check CPU model by qemu command line, didn't see any deprecated info
@dhildenb 
please double check if we don't have deprecated info with qemu on s390x

[root@l42 ~]# /usr/libexec/qemu-kvm -cpu help
s390 qemu            QEMU Virtual CPU version 2.5+       (migration-safe)
s390 host            KVM only: All recognized features   
s390 max             Enables all features supported by the accelerator in the current host 
s390 z900-base       IBM zSeries 900 GA1                 (static, migration-safe)
s390 z900            IBM zSeries 900 GA1                 (migration-safe)
s390 z900.2-base     IBM zSeries 900 GA2                 (static, migration-safe)
s390 z900.2          IBM zSeries 900 GA2                 (migration-safe)
s390 z900.3-base     IBM zSeries 900 GA3                 (static, migration-safe)
s390 z900.3          IBM zSeries 900 GA3                 (migration-safe)
s390 z800-base       IBM zSeries 800 GA1                 (static, migration-safe)
s390 z800            IBM zSeries 800 GA1                 (migration-safe)
s390 z990-base       IBM zSeries 990 GA1                 (static, migration-safe)
s390 z990            IBM zSeries 990 GA1                 (migration-safe)
s390 z990.2-base     IBM zSeries 990 GA2                 (static, migration-safe)
s390 z990.2          IBM zSeries 990 GA2                 (migration-safe)
s390 z990.3-base     IBM zSeries 990 GA3                 (static, migration-safe)
s390 z990.3          IBM zSeries 990 GA3                 (migration-safe)
s390 z890-base       IBM zSeries 880 GA1                 (static, migration-safe)
s390 z890            IBM zSeries 880 GA1                 (migration-safe)
s390 z990.4-base     IBM zSeries 990 GA4                 (static, migration-safe)
s390 z990.4          IBM zSeries 990 GA4                 (migration-safe)
s390 z890.2-base     IBM zSeries 880 GA2                 (static, migration-safe)
s390 z890.2          IBM zSeries 880 GA2                 (migration-safe)
s390 z990.5-base     IBM zSeries 990 GA5                 (static, migration-safe)
s390 z990.5          IBM zSeries 990 GA5                 (migration-safe)
s390 z890.3-base     IBM zSeries 880 GA3                 (static, migration-safe)
s390 z890.3          IBM zSeries 880 GA3                 (migration-safe)
s390 z9EC-base       IBM System z9 EC GA1                (static, migration-safe)
s390 z9EC            IBM System z9 EC GA1                (migration-safe)
s390 z9EC.2-base     IBM System z9 EC GA2                (static, migration-safe)
s390 z9EC.2          IBM System z9 EC GA2                (migration-safe)
s390 z9BC-base       IBM System z9 BC GA1                (static, migration-safe)
s390 z9BC            IBM System z9 BC GA1                (migration-safe)
s390 z9EC.3-base     IBM System z9 EC GA3                (static, migration-safe)
s390 z9EC.3          IBM System z9 EC GA3                (migration-safe)
s390 z9BC.2-base     IBM System z9 BC GA2                (static, migration-safe)
s390 z9BC.2          IBM System z9 BC GA2                (migration-safe)
s390 z10EC-base      IBM System z10 EC GA1               (static, migration-safe)
s390 z10EC           IBM System z10 EC GA1               (migration-safe)
s390 z10EC.2-base    IBM System z10 EC GA2               (static, migration-safe)
s390 z10EC.2         IBM System z10 EC GA2               (migration-safe)
s390 z10BC-base      IBM System z10 BC GA1               (static, migration-safe)
s390 z10BC           IBM System z10 BC GA1               (migration-safe)
s390 z10EC.3-base    IBM System z10 EC GA3               (static, migration-safe)
s390 z10EC.3         IBM System z10 EC GA3               (migration-safe)
s390 z10BC.2-base    IBM System z10 BC GA2               (static, migration-safe)
s390 z10BC.2         IBM System z10 BC GA2               (migration-safe)
s390 z196-base       IBM zEnterprise 196 GA1             (static, migration-safe)
s390 z196            IBM zEnterprise 196 GA1             (migration-safe)
s390 z196.2-base     IBM zEnterprise 196 GA2             (static, migration-safe)
s390 z196.2          IBM zEnterprise 196 GA2             (migration-safe)
s390 z114-base       IBM zEnterprise 114 GA1             (static, migration-safe)
s390 z114            IBM zEnterprise 114 GA1             (migration-safe)
s390 zEC12-base      IBM zEnterprise EC12 GA1            (static, migration-safe)
s390 zEC12           IBM zEnterprise EC12 GA1            (migration-safe)
s390 zEC12.2-base    IBM zEnterprise EC12 GA2            (static, migration-safe)
s390 zEC12.2         IBM zEnterprise EC12 GA2            (migration-safe)
s390 zBC12-base      IBM zEnterprise BC12 GA1            (static, migration-safe)
s390 zBC12           IBM zEnterprise BC12 GA1            (migration-safe)
s390 z13-base        IBM z13 GA1                         (static, migration-safe)
s390 z13             IBM z13 GA1                         (migration-safe)
s390 z13.2-base      IBM z13 GA2                         (static, migration-safe)
s390 z13.2           IBM z13 GA2                         (migration-safe)
s390 z13s-base       IBM z13s GA1                        (static, migration-safe)
s390 z13s            IBM z13s GA1                        (migration-safe)
s390 z14-base        IBM z14 GA1                         (static, migration-safe)
s390 z14             IBM z14 GA1                         (migration-safe)
s390 z14.2-base      IBM z14 GA2                         (static, migration-safe)
s390 z14.2           IBM z14 GA2                         (migration-safe)
s390 z14ZR1-base     IBM z14 Model ZR1 GA1               (static, migration-safe)
s390 z14ZR1          IBM z14 Model ZR1 GA1               (migration-safe)
s390 gen15a-base     IBM z15 T01 GA1                     (static, migration-safe)
s390 gen15a          IBM z15 T01 GA1                     (migration-safe)
s390 gen15b-base     IBM z15 T02 GA1                     (static, migration-safe)
s390 gen15b          IBM z15 T02 GA1                     (migration-safe)
s390 gen16a-base     IBM 3931 GA1                        (static, migration-safe)
s390 gen16a          IBM 3931 GA1                        (migration-safe)
s390 gen16b-base     IBM 3932 GA1                        (static, migration-safe)
s390 gen16b          IBM 3932 GA1                        (migration-safe)


2. check deprecated flag with libvirt. results are unexpected
@dan I think we miss the model of "qemu" here or maybe we give wrong comment info with the MR above?

[root@l42 ~]# virsh domcapabilities | grep model | grep deprecated
      <model usable='no' deprecated='yes'>z890.2-base</model>
      <model usable='no' deprecated='yes'>z800-base</model>
      <model usable='no' deprecated='yes'>z9EC.2</model>
      <model usable='yes' deprecated='yes'>z13.2</model>
      <model usable='no' deprecated='yes'>z990.5-base</model>
      <model usable='no' deprecated='yes'>z9BC-base</model>
      <model usable='no' deprecated='yes'>z890.2</model>
      <model usable='no' deprecated='yes'>z890</model>
      <model usable='no' deprecated='yes'>z9BC</model>
      <model usable='yes' deprecated='yes'>z13</model>
      <model usable='yes' deprecated='yes'>z196</model>
      <model usable='yes' deprecated='yes'>z13s</model>
      <model usable='no' deprecated='yes'>z990.3</model>
      <model usable='yes' deprecated='yes'>z13s-base</model>
      <model usable='no' deprecated='yes'>z9EC</model>
      <model usable='no' deprecated='yes'>z900.3-base</model>
      <model usable='yes' deprecated='yes'>z13.2-base</model>
      <model usable='yes' deprecated='yes'>z196.2-base</model>
      <model usable='yes' deprecated='yes'>zBC12-base</model>
      <model usable='no' deprecated='yes'>z9BC.2-base</model>
      <model usable='no' deprecated='yes'>z900.2-base</model>
      <model usable='no' deprecated='yes'>z9EC.3</model>
      <model usable='yes' deprecated='yes'>zEC12</model>
      <model usable='no' deprecated='yes'>z900</model>
      <model usable='yes' deprecated='yes'>z114-base</model>
      <model usable='yes' deprecated='yes'>zEC12-base</model>
      <model usable='no' deprecated='yes'>z10EC.2</model>
      <model usable='no' deprecated='yes'>z10EC-base</model>
      <model usable='no' deprecated='yes'>z900.3</model>
      <model usable='no' deprecated='yes'>z10BC</model>
      <model usable='no' deprecated='yes'>z10BC.2-base</model>
      <model usable='no' deprecated='yes'>z9BC.2</model>
      <model usable='no' deprecated='yes'>z990.2</model>
      <model usable='no' deprecated='yes'>z990</model>
      <model usable='no' deprecated='yes'>z990.4</model>
      <model usable='no' deprecated='yes'>z10EC.2-base</model>
      <model usable='no' deprecated='yes'>z990.2-base</model>
      <model usable='no' deprecated='yes'>z800</model>
      <model usable='no' deprecated='yes'>z10EC</model>
      <model usable='yes' deprecated='yes'>zEC12.2</model>
      <model usable='no' deprecated='yes'>z900-base</model>
      <model usable='no' deprecated='yes'>z10BC.2</model>
      <model usable='no' deprecated='yes'>z9EC-base</model>
      <model usable='no' deprecated='yes'>z9EC.3-base</model>
      <model usable='yes' deprecated='yes'>z114</model>
      <model usable='no' deprecated='yes'>z890.3</model>
      <model usable='yes' deprecated='yes'>z196-base</model>
      <model usable='no' deprecated='yes'>z9EC.2-base</model>
      <model usable='yes' deprecated='yes'>z196.2</model>
      <model usable='no' deprecated='yes'>z990-base</model>
      <model usable='no' deprecated='yes'>z900.2</model>
      <model usable='no' deprecated='yes'>z890-base</model>
      <model usable='no' deprecated='yes'>z10EC.3</model>
      <model usable='no' deprecated='yes'>z990.4-base</model>
      <model usable='no' deprecated='yes'>z10EC.3-base</model>
      <model usable='no' deprecated='yes'>z10BC-base</model>
      <model usable='yes' deprecated='yes'>z13-base</model>
      <model usable='no' deprecated='yes'>z990.3-base</model>
      <model usable='yes' deprecated='yes'>zEC12.2-base</model>
      <model usable='yes' deprecated='yes'>zBC12</model>
      <model usable='no' deprecated='yes'>z890.3-base</model>
      <model usable='no' deprecated='yes'>z990.5</model>

3. Boot RHEL7.9 guest with deprecated CPU model 'z196', guest can boot up successfully without deprecated info.
[root@l42 home]# sh 7.sh 
QEMU 7.0.0 monitor - type 'help' for more information
(qemu) 
kernel-3.10.0-1160.el7.s390x

4. Boot RHEL8.7 guest with deprecated CPU model 'z13', guest can boot up successfully without deprecated info.
[root@l42 home]# sh 8.sh 
QEMU 7.0.0 monitor - type 'help' for more information
(qemu) 
guest kernel: kernel-4.18.0-408.el8.s390x

5. Boot RHEL9.1 guest with deprecated CPU model, guest will crash with deprecated info. But we don't really supported z13 with RHEL.9.1 guest
QEMU 7.0.0 monitor - type 'help' for more information
(qemu) qemu-kvm: warning: CPU model z13-s390x-cpu is deprecated -- use at least 'z14', or 'host' / 'qemu' / 'max'


@thuth 

@smitterl FYI, I think we should modify our test cases related to CPU model

(In reply to bfu from comment #55)
> Test environment:
...
> 
> @smitterl FYI, I think we should modify our test cases related to CPU model
OK, let's discuss this. I had understood 'deprecated' as "on it's way to being disabled but still supported" - see comment#34 and comment#35 and that if supported -> tested.

(In reply to liunana from comment #52)
> Test Env:
>      5.14.0-127.el9.x86_64
>      qemu-kvm-7.0.0-8.el9.x86_64
>      libvirt-daemon-8.5.0-1.el9.x86_64
>      Model name:            AMD EPYC 7302P 16-Core Processor
> 
> 
> Test steps:
> 1. check CPU model by qemu command line, didn't see any deprecated info and
> the deprecated CPU models are still in Available CPUs.
> # /usr/libexec/qemu-kvm -cpu help
> Available CPUs:

Sorry, the merged patches don't include changes to '-cpu help', as they're still pending upstream.


> Could you please help to check the step 1? Seems the results aren't the
> expected.

We'll address -cpu help at a later time.
 
> And for booting a VM with deprecated CPU model, are the steps 4,5,6 the
> expected results?

Yes, this is correct.

(In reply to bfu from comment #55)
> Test environment:
> Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
> qemu version: qemu-kvm-7.0.0-8.el9.s390x
> libvirt version: libvirt-8.3.0-1.el9.s390x
> 
> 1. check CPU model by qemu command line, didn't see any deprecated info
> @dhildenb 
> please double check if we don't have deprecated info with qemu on s390x
> 
> [root@l42 ~]# /usr/libexec/qemu-kvm -cpu help

THis is expected right now. Displayging a deprecation message in -cpu help is still pending upstream


> 
> 
> 2. check deprecated flag with libvirt. results are unexpected
> @dan I think we miss the model of "qemu" here or maybe we give
> wrong comment info with the MR above?

The 'qemu'  CPU is *NOT* deprecated. This is a change (fix) since my original patch mistakenly deprecated 'qemu'

(In reply to Daniel Berrangé from comment #58)
> (In reply to bfu from comment #55)
> > Test environment:
> > Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
> > qemu version: qemu-kvm-7.0.0-8.el9.s390x
> > libvirt version: libvirt-8.3.0-1.el9.s390x
> > 
> > 1. check CPU model by qemu command line, didn't see any deprecated info
> > @dhildenb 
> > please double check if we don't have deprecated info with qemu on s390x
> > 
> > [root@l42 ~]# /usr/libexec/qemu-kvm -cpu help
> 
> THis is expected right now. Displayging a deprecation message in -cpu help
> is still pending upstream
> 
> 
> > 
> > 
> > 2. check deprecated flag with libvirt. results are unexpected
> > @dan I think we miss the model of "qemu" here or maybe we give
> > wrong comment info with the MR above?
> 
> The 'qemu'  CPU is *NOT* deprecated. This is a change (fix) since my
> original patch mistakenly deprecated 'qemu'

Could you please double confirm that test step 3,4 in comment 55 is what we expected as well?
@dan

(In reply to Daniel Berrangé from comment #58)
> (In reply to bfu from comment #55)
> > Test environment:
> > Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
> > qemu version: qemu-kvm-7.0.0-8.el9.s390x
> > libvirt version: libvirt-8.3.0-1.el9.s390x
> > 
> > 1. check CPU model by qemu command line, didn't see any deprecated info
> > @dhildenb 
> > please double check if we don't have deprecated info with qemu on s390x
> > 
> > [root@l42 ~]# /usr/libexec/qemu-kvm -cpu help
> 
> THis is expected right now. Displayging a deprecation message in -cpu help
> is still pending upstream

Clearing needinfo because Dan answered the question.

> 
> 3. Boot RHEL7.9 guest with deprecated CPU model 'z196', guest can boot up
> successfully without deprecated info.
> [root@l42 home]# sh 7.sh 
> QEMU 7.0.0 monitor - type 'help' for more information
> (qemu) 
> kernel-3.10.0-1160.el7.s390x
> 
> 4. Boot RHEL8.7 guest with deprecated CPU model 'z13', guest can boot up
> successfully without deprecated info.
> [root@l42 home]# sh 8.sh 
> QEMU 7.0.0 monitor - type 'help' for more information
> (qemu) 
> guest kernel: kernel-4.18.0-408.el8.s390x

Those two cases are weird. AFAIU, the deprecation-behaviour shouldn't be guest-dependant.

The warning ("warning: CPU model ... is deprecated") should be printed in any case where a deprecated CPU model is provided, even when starting a legacy guest.

Can you double-check that your scripts don't accidentally swallow that warning?

(In reply to Daniel Berrangé from comment #58)
> (In reply to bfu from comment #55)
> > Test environment:
> > Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
> > qemu version: qemu-kvm-7.0.0-8.el9.s390x
> > libvirt version: libvirt-8.3.0-1.el9.s390x
> > 
> > 1. check CPU model by qemu command line, didn't see any deprecated info
> > @dhildenb 
> > please double check if we don't have deprecated info with qemu on s390x
> > 
> > [root@l42 ~]# /usr/libexec/qemu-kvm -cpu help
> 
> THis is expected right now. Displayging a deprecation message in -cpu help
> is still pending upstream
>

Hi Daniel,

Do we need a separate bz to track this patch?
If so I will create one for it, thanks.


Best regards
Nana Liu

> 
> > 
> > 
> > 2. check deprecated flag with libvirt. results are unexpected
> > @dan I think we miss the model of "qemu" here or maybe we give
> > wrong comment info with the MR above?
> 
> The 'qemu'  CPU is *NOT* deprecated. This is a change (fix) since my
> original patch mistakenly deprecated 'qemu'

(In reply to bfu from comment #55)
[...]
> 3. Boot RHEL7.9 guest with deprecated CPU model 'z196', guest can boot up
> successfully without deprecated info.
> [root@l42 home]# sh 7.sh 
> QEMU 7.0.0 monitor - type 'help' for more information
> (qemu) 
> kernel-3.10.0-1160.el7.s390x

I just tried to run a RHEL7 guest with qemu-kvm-7.0.0-8 and the z196 CPU type, and I can see the deprecation message there:

$ /usr/libexec/qemu-kvm -cpu z196 -nographic -hda rhel7_7.qcow2
qemu-kvm: warning: CPU model z196-s390x-cpu is deprecated -- use at least 'z14', or 'host' / 'qemu' / 'max'
LOADPARM=[        ]
Using virtio-blk.
Using SCSI scheme.
.....
[    0.077263] Initializing cgroup subsys cpuset
[    0.077266] Initializing cgroup subsys cpu
[    0.077267] Initializing cgroup subsys cpuacct
[    0.077271] Linux version 3.10.0-1014.el7.s390x (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Fri Mar 1 16:40:37 EST 2019
[    0.077277] setup: Linux is running under KVM in 64-bit mode
...

So this seems to work as expected. Are you sure that the deprecation notice isn't suppressed by your script?

(In reply to liunana from comment #62)
> (In reply to Daniel Berrangé from comment #58)
> > (In reply to bfu from comment #55)
> > > Test environment:
> > > Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
> > > qemu version: qemu-kvm-7.0.0-8.el9.s390x
> > > libvirt version: libvirt-8.3.0-1.el9.s390x
> > > 
> > > 1. check CPU model by qemu command line, didn't see any deprecated info
> > > @dhildenb 
> > > please double check if we don't have deprecated info with qemu on s390x
> > > 
> > > [root@l42 ~]# /usr/libexec/qemu-kvm -cpu help
> > 
> > THis is expected right now. Displayging a deprecation message in -cpu help
> > is still pending upstream
> >
> 
> Hi Daniel,
> 
> Do we need a separate bz to track this patch?
> If so I will create one for it, thanks.

Yes, I think it will be something for 9.2, given it is not yet upstream.

(In reply to Daniel Berrangé from comment #64)
> (In reply to liunana from comment #62)
> > (In reply to Daniel Berrangé from comment #58)
> > > (In reply to bfu from comment #55)
> > > > Test environment:
> > > > Kernel version: kernel-5.14.0-110.mr981_220614_0752.el9.s390x
> > > > qemu version: qemu-kvm-7.0.0-8.el9.s390x
> > > > libvirt version: libvirt-8.3.0-1.el9.s390x
> > > > 
> > > > 1. check CPU model by qemu command line, didn't see any deprecated info
> > > > @dhildenb 
> > > > please double check if we don't have deprecated info with qemu on s390x
> > > > 
> > > > [root@l42 ~]# /usr/libexec/qemu-kvm -cpu help
> > > 
> > > THis is expected right now. Displayging a deprecation message in -cpu help
> > > is still pending upstream
> > >
> > 
> > Hi Daniel,
> > 
> > Do we need a separate bz to track this patch?
> > If so I will create one for it, thanks.
> 
> Yes, I think it will be something for 9.2, given it is not yet upstream.

Ok, create one new bug to track this on 9.2, Could you please help to take this? Thanks.
Bug 2108923 - [RHEL.9.2] Display a deprecation message in '-cpu help' for deprecated CPU models


Best regards
Nana Liu

Hi Boqiao,

Could you plese help to answer Thomas's question in Comment 63?
And if you have got the expected results please help to add a comment in this bug so that I can move this bug to verified.
Thanks.


Best regards
Nana Liu

(In reply to Thomas Huth from comment #63)
> (In reply to bfu from comment #55)
> [...]
> > 3. Boot RHEL7.9 guest with deprecated CPU model 'z196', guest can boot up
> > successfully without deprecated info.
> > [root@l42 home]# sh 7.sh 
> > QEMU 7.0.0 monitor - type 'help' for more information
> > (qemu) 
> > kernel-3.10.0-1160.el7.s390x
> 
> I just tried to run a RHEL7 guest with qemu-kvm-7.0.0-8 and the z196 CPU
> type, and I can see the deprecation message there:
> 
> $ /usr/libexec/qemu-kvm -cpu z196 -nographic -hda rhel7_7.qcow2
> qemu-kvm: warning: CPU model z196-s390x-cpu is deprecated -- use at least
> 'z14', or 'host' / 'qemu' / 'max'
> LOADPARM=[        ]
> Using virtio-blk.
> Using SCSI scheme.
> .....
> [    0.077263] Initializing cgroup subsys cpuset
> [    0.077266] Initializing cgroup subsys cpu
> [    0.077267] Initializing cgroup subsys cpuacct
> [    0.077271] Linux version 3.10.0-1014.el7.s390x
> (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623
> (Red Hat 4.8.5-36) (GCC) ) #1 SMP Fri Mar 1 16:40:37 EST 2019
> [    0.077277] setup: Linux is running under KVM in 64-bit mode
> ...
> 
> So this seems to work as expected. Are you sure that the deprecation notice
> isn't suppressed by your script?

I re-test with the qemu package, and they could provide the right deprecated information as you do


@nanliu I think you could set this bz as verified if there's no other issues reported by other platform except s390x

(In reply to bfu from comment #67)
> (In reply to Thomas Huth from comment #63)
> > (In reply to bfu from comment #55)
> > [...]
> > > 3. Boot RHEL7.9 guest with deprecated CPU model 'z196', guest can boot up
> > > successfully without deprecated info.
> > > [root@l42 home]# sh 7.sh 
> > > QEMU 7.0.0 monitor - type 'help' for more information
> > > (qemu) 
> > > kernel-3.10.0-1160.el7.s390x
> > 
> > I just tried to run a RHEL7 guest with qemu-kvm-7.0.0-8 and the z196 CPU
> > type, and I can see the deprecation message there:
> > 
> > $ /usr/libexec/qemu-kvm -cpu z196 -nographic -hda rhel7_7.qcow2
> > qemu-kvm: warning: CPU model z196-s390x-cpu is deprecated -- use at least
> > 'z14', or 'host' / 'qemu' / 'max'
> > LOADPARM=[        ]
> > Using virtio-blk.
> > Using SCSI scheme.
> > .....
> > [    0.077263] Initializing cgroup subsys cpuset
> > [    0.077266] Initializing cgroup subsys cpu
> > [    0.077267] Initializing cgroup subsys cpuacct
> > [    0.077271] Linux version 3.10.0-1014.el7.s390x
> > (mockbuild.eng.bos.redhat.com) (gcc version 4.8.5 20150623
> > (Red Hat 4.8.5-36) (GCC) ) #1 SMP Fri Mar 1 16:40:37 EST 2019
> > [    0.077277] setup: Linux is running under KVM in 64-bit mode
> > ...
> > 
> > So this seems to work as expected. Are you sure that the deprecation notice
> > isn't suppressed by your script?
> 
> I re-test with the qemu package, and they could provide the right deprecated
> information as you do
> 
> 
> @nanliu I think you could set this bz as verified if there's no
> other issues reported by other platform except s390x

Thank you, move this bug to verified now.

Hi Daniel,


I have two questions here.
1. According to test scenario 4 in Comment 52, booting a RHEL9.1 guest with deprecated CPU model, guest will crash with unspported log.
   So do we need to add this to Doc Text to avoid that bugs like 1925384 and 2065023 are created in future?

2. Since the default cpu model 'qemu64' is deprecated, is the cpu model 'max' the default cpu model on rhel9 now? 
    And on RHEL8, I guess 'qemu64' is still the default cpu model, nothing change, right?

Could you please help to check the two questions? Thanks for your time.


Best regards
Nana Liu

Sorry for that, I add the bug link in here for the question 1.

Bug 1925384 - RHEL9 KVM guest kernel panic during system boot up with default cpu mode
Bug 2065023 - [RHEL9] guest Kernel panic while booting with cpu model 'Opteron_G3'

(In reply to liunana from comment #69)
> Hi Daniel,
> 
> 
> I have two questions here.
> 1. According to test scenario 4 in Comment 52, booting a RHEL9.1 guest with
> deprecated CPU model, guest will crash with unspported log.
>    So do we need to add this to Doc Text to avoid that bugs like 1925384 and
> 2065023 are created in future?

That's because RHEL-9 has been compiled to require a new minimum featureset and some of the deprecated CPUs can't meet that.  The guest would crash regardless of whether we marked them as deprecated. In terms of documentation, this is a generic RHEL problem - RHEL needs to document minimum required hardware levels, and i believe it has already done so, though I dno't have a link handy.

> 
> 2. Since the default cpu model 'qemu64' is deprecated, is the cpu model
> 'max' the default cpu model on rhel9 now? 
>     And on RHEL8, I guess 'qemu64' is still the default cpu model, nothing
> change, right?

We're not changing the default CPU model in QEMU, despite it being deprecated. All management apps using libvirt should be explicitly choosing a CPU model. if the default is ever used, that is a bug in the mgmt app for forgetting to set a CPU model.

(In reply to Daniel Berrangé from comment #71)
> (In reply to liunana from comment #69)
> > Hi Daniel,
> > 
> > 
> > I have two questions here.
> > 1. According to test scenario 4 in Comment 52, booting a RHEL9.1 guest with
> > deprecated CPU model, guest will crash with unspported log.
> >    So do we need to add this to Doc Text to avoid that bugs like 1925384 and
> > 2065023 are created in future?
> 
> That's because RHEL-9 has been compiled to require a new minimum featureset
> and some of the deprecated CPUs can't meet that.  The guest would crash
> regardless of whether we marked them as deprecated. In terms of
> documentation, this is a generic RHEL problem - RHEL needs to document
> minimum required hardware levels, and i believe it has already done so,
> though I dno't have a link handy.
> 
OK, thank you, do you know who I can ask for these documentaion?


> > 
> > 2. Since the default cpu model 'qemu64' is deprecated, is the cpu model
> > 'max' the default cpu model on rhel9 now? 
> >     And on RHEL8, I guess 'qemu64' is still the default cpu model, nothing
> > change, right?
> 
> We're not changing the default CPU model in QEMU, despite it being
> deprecated. All management apps using libvirt should be explicitly choosing
> a CPU model. if the default is ever used, that is a bug in the mgmt app for
> forgetting to set a CPU model.

Got it, so on RHEL9 which cpu model can regard as the 'minimum featureset'?
Is that different between AMD host and Intel host?

Thanks for your time.


Best regards
Nana Liu

(In reply to liunana from comment #72)
> (In reply to Daniel Berrangé from comment #71)

> > That's because RHEL-9 has been compiled to require a new minimum featureset
> > and some of the deprecated CPUs can't meet that.  The guest would crash
> > regardless of whether we marked them as deprecated. In terms of
> > documentation, this is a generic RHEL problem - RHEL needs to document
> > minimum required hardware levels, and i believe it has already done so,
> > though I dno't have a link handy.
> > 
> OK, thank you, do you know who I can ask for these documentaion?

The minimum hardware baseline is documented in the release notes:

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html-single/9.0_release_notes/index#architectures

> 
> 
> > > 
> > > 2. Since the default cpu model 'qemu64' is deprecated, is the cpu model
> > > 'max' the default cpu model on rhel9 now? 
> > >     And on RHEL8, I guess 'qemu64' is still the default cpu model, nothing
> > > change, right?
> > 
> > We're not changing the default CPU model in QEMU, despite it being
> > deprecated. All management apps using libvirt should be explicitly choosing
> > a CPU model. if the default is ever used, that is a bug in the mgmt app for
> > forgetting to set a CPU model.
> 
> Got it, so on RHEL9 which cpu model can regard as the 'minimum featureset'?
> Is that different between AMD host and Intel host?

On x86_64, the x86_64-v2 ABI is the baseline. You can see all CPUs satisfying that in this table:

https://www.qemu.org/docs/master/system/i386/cpu.html

Of those 'Nehalem' is the oldest CPU and happens to be compatible with AMD hosts, as well as Intel.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7967