Bug 1732726

Summary: Failed to boot a guest on power9 witherspoon DD2.2 with "pseries-rhel7.6.0-sxxm" machine type with error "qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs"
Product: Red Hat Enterprise Linux 8 Reporter: xianwang <xianwang>
Component: doc-Release_Notes-8-en-USAssignee: Jiri Herrmann <jherrman>
Status: CLOSED CURRENTRELEASE QA Contact: RHEL DPM <rhel-docs>
Severity: medium Docs Contact: Jiri Herrmann <jherrman>
Priority: medium    
Version: 8.1CC: abologna, dgibson, dzheng, jherrman, lmanasko, mdeng, ngu, qzhang, rhel-docs, tcapek, yihyu
Target Milestone: rc   
Target Release: 8.0   
Hardware: ppc64le   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
.RHEL 8 virtual machines sometimes cannot boot on Witherspoon hosts RHEL 8 virtual machines (VMs) that use the `pseries-rhel7.6.0-sxxm` machine type in some cases fail to boot on _Power9 S922LC for HPC_ hosts (also known as Witherspoon) that use the DD2.2 or DD2.3 CPU. Attempting to boot such a VM instead generates the following error message: ---- qemu-kvm: Requested safe indirect branch capability level not supported by kvm ---- To work around this problem, configure the virtual machine's XML configuration as follows: ---- <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <qemu:commandline> <qemu:arg value='-machine'/> <qemu:arg value='cap-ibs=workaround'/> </qemu:commandline> ----
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-02-01 07:43:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description xianwang 2019-07-24 08:38:12 UTC
Document URL: 

Section Number and Name: 

Describe the issue: 
On power9  witherspoon DD2.2 host, boot a guest with qemu cli:
Host:
4.14.0-115.8.2.el7a.ppc64le
qemu-kvm-rhev-2.12.0-18.el7_6.7.ppc64le
SLOF-20171214-2.gitfa98132.el7.noarch

# /usr/libexec/qemu-kvm -M pseries-rhel7.6.0-sxxm -monitor stdio -nodefaults
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs

 


Suggestions for improvement: 
This issue is also hit on rhelav8.0.1 on witherspoon, it only exists on witherspoon host, it seems has something wrong with hardware.

Additional information:

Comment 1 xianwang 2019-07-24 08:45:05 UTC
I:
On the same host(rhel-alt-7.6):
# /usr/libexec/qemu-kvm -M pseries-rhel7.6.0
VNC server running on ::1:5900

if qemu cli without "sxxm", it will be ok.

II.
on rhelav8.0.1 host:
# /usr/libexec/qemu-kvm -M pseries-rhel7.6.0-sxxm
VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs

# /usr/libexec/qemu-kvm -M pseries-rhel8.0.0
works well.

Comment 2 xianwang 2019-07-24 08:57:21 UTC
Hi, Jiri,
could you help to check this issue, thanks

Comment 20 David Gibson 2019-11-18 03:35:13 UTC
ngu,

I see you set needinfo? for me, but I don't see a question.  What is it you want to know?

Comment 21 Gu Nini 2019-11-18 04:32:25 UTC
(In reply to David Gibson from comment #20)
> ngu,
> 
> I see you set needinfo? for me, but I don't see a question.  What is it you
> want to know?

Hi David,

It's in comment 18, to follow up the action in comment 16. Do you think it's meaningful to add support for parameter 'ibs' in libvirt xm so we can recommend users to use cap-ibs=workaround?

Comment 22 David Gibson 2019-11-19 03:12:19 UTC
Jiri, update text looks better to me.  One final revision:

I don't think there's any point suggesting different options for the different CPU versions.  I'd just suggest cap-ibs=workaround in all cases.  That should work for both CPUs, and is the currently recommended mitigation for this bug.

Comment 23 Andrea Bolognani 2019-11-19 11:06:47 UTC
(In reply to David Gibson from comment #22)
> Jiri, update text looks better to me.  One final revision:
> 
> I don't think there's any point suggesting different options for the
> different CPU versions.  I'd just suggest cap-ibs=workaround in all cases. 
> That should work for both CPUs, and is the currently recommended mitigation
> for this bug.

Hold on. The doc text reads

  To work around this problem, add one of the following strings to the
  kernel command line of the VM, depending on the CPU firmware:

  * For DD2.2: `pseries-rhel7.6.0-sxxm,cap-ibs=fixed-ibs`
  * For DD2.3: `pseries-rhel7.6.0-sxxm,cap-ibs=workaround`

which is clearly not correct, as it's not the *kernel* command line
that needs tweaking but rather the *QEMU* command line.


Additionally, since we don't support running QEMU directly, we need
to document the solution in a libvirt-compatible format: either by
implementing the flag properly, as suggested earlier by Gu Nini, or
at the very least by providing an equivalent XML snippet such as
(untested)

  <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <qemu:commandline>
      <qemu:arg value='-machine'/>
      <qemu:arg value='pseries-rhel7.6.0-sxxm,cap-ibs=workaround'/>
    </qemu:commandline>
    ...

Comment 24 David Gibson 2019-11-20 01:33:52 UTC
Duh.  Good point, I totally missed that Andrea.

Yes, this is about the qemu command line, not the kernel command line.

Jiři, yes I think we're going to need the XML workaround instead.

I guess for now we can just use the hack that puts the qemu command line argument into the XML, though longer term it might be worth adding proper support for the parameter to libvirt.  We shouldn't need to repeat the machine type name in there, though.  So we could just use:

<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <qemu:commandline>
    <qemu:arg value='-machine'/>
    <qemu:arg value='cap-ibs=workaround'/>
  </qemu:commandline>
</domain>

Comment 26 Andrea Bolognani 2019-12-12 08:30:21 UTC
According to what David said in Comment 22 and Comment 24, the XML
snippet doesn't need to be different for the two CPU models and the
machine type doesn't need to be included, so we should be able to
apply the following diff:

--- doctext.orig	2019-12-12 09:27:19.132159938 +0100
+++ doctext	2019-12-12 09:27:46.482350262 +0100
@@ -10,22 +10,10 @@
 
 To work around this problem, configure the virtual machine's XML configuration as follows:
 
-* For DD2.2:
-
-----
-  <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
-    <qemu:commandline>
-      <qemu:arg value='-machine'/>
-      <qemu:arg value='pseries-rhel7.6.0-sxxm,cap-ibs=fixed-ibs'/>
-    </qemu:commandline>
-----
-
-* For DD2.3:
-
 ----
   <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
     <qemu:commandline>
       <qemu:arg value='-machine'/>
-      <qemu:arg value='pseries-rhel7.6.0-sxxm,cap-ibs=workaround'/>
+      <qemu:arg value='cap-ibs=workaround'/>
     </qemu:commandline>
 ----

Comment 27 Jiri Herrmann 2019-12-12 10:31:42 UTC
Understood and updated :-) Thank you again folks!

Comment 30 RHEL Program Management 2021-02-01 07:43:56 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.