1732726 – Failed to boot a guest on power9 witherspoon DD2.2 with "pseries-rhel7.6.0-sxxm" machine type with error "qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs"

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1732726 - Failed to boot a guest on power9 witherspoon DD2.2 with "pseries-rhel7.6.0-sxxm" machine type with error "qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs"

Summary: Failed to boot a guest on power9 witherspoon DD2.2 with "pseries-rhel7.6.0-s...

Keywords:
Status:	CLOSED CURRENTRELEASE
Alias:	None
Product:	Red Hat Enterprise Linux 8
Classification:	Red Hat
Component:	doc-Release_Notes-8-en-US
Sub Component:
Version:	8.1
Hardware:	ppc64le
OS:	Linux
Priority:	medium
Severity:	medium
Target Milestone:	rc
Target Release:	8.0
Assignee:	Jiri Herrmann
QA Contact:	RHEL DPM
Docs Contact:	Jiri Herrmann
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2019-07-24 08:38 UTC by xianwang
Modified:	2021-02-03 09:32 UTC (History)
CC List:	11 users (show)
Fixed In Version:
Doc Type:	Known Issue
Doc Text:	.RHEL 8 virtual machines sometimes cannot boot on Witherspoon hosts RHEL 8 virtual machines (VMs) that use the `pseries-rhel7.6.0-sxxm` machine type in some cases fail to boot on _Power9 S922LC for HPC_ hosts (also known as Witherspoon) that use the DD2.2 or DD2.3 CPU. Attempting to boot such a VM instead generates the following error message: ---- qemu-kvm: Requested safe indirect branch capability level not supported by kvm ---- To work around this problem, configure the virtual machine's XML configuration as follows: ---- <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <qemu:commandline> <qemu:arg value='-machine'/> <qemu:arg value='cap-ibs=workaround'/> </qemu:commandline> ----
Clone Of:
Environment:
Last Closed:	2021-02-01 07:43:56 UTC
Type:	Bug
Target Upstream Version:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)

Description xianwang 2019-07-24 08:38:12 UTC

Document URL: 

Section Number and Name: 

Describe the issue: 
On power9  witherspoon DD2.2 host, boot a guest with qemu cli:
Host:
4.14.0-115.8.2.el7a.ppc64le
qemu-kvm-rhev-2.12.0-18.el7_6.7.ppc64le
SLOF-20171214-2.gitfa98132.el7.noarch

# /usr/libexec/qemu-kvm -M pseries-rhel7.6.0-sxxm -monitor stdio -nodefaults
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs

 


Suggestions for improvement: 
This issue is also hit on rhelav8.0.1 on witherspoon, it only exists on witherspoon host, it seems has something wrong with hardware.

Additional information:

Comment 1 xianwang 2019-07-24 08:45:05 UTC

I:
On the same host(rhel-alt-7.6):
# /usr/libexec/qemu-kvm -M pseries-rhel7.6.0
VNC server running on ::1:5900

if qemu cli without "sxxm", it will be ok.

II.
on rhelav8.0.1 host:
# /usr/libexec/qemu-kvm -M pseries-rhel7.6.0-sxxm
VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ibs

# /usr/libexec/qemu-kvm -M pseries-rhel8.0.0
works well.

Comment 2 xianwang 2019-07-24 08:57:21 UTC

Hi, Jiri,
could you help to check this issue, thanks

Comment 20 David Gibson 2019-11-18 03:35:13 UTC

ngu,

I see you set needinfo? for me, but I don't see a question.  What is it you want to know?

Comment 21 Gu Nini 2019-11-18 04:32:25 UTC

(In reply to David Gibson from comment #20)
> ngu,
> 
> I see you set needinfo? for me, but I don't see a question.  What is it you
> want to know?

Hi David,

It's in comment 18, to follow up the action in comment 16. Do you think it's meaningful to add support for parameter 'ibs' in libvirt xm so we can recommend users to use cap-ibs=workaround?

Comment 22 David Gibson 2019-11-19 03:12:19 UTC

Jiri, update text looks better to me.  One final revision:

I don't think there's any point suggesting different options for the different CPU versions.  I'd just suggest cap-ibs=workaround in all cases.  That should work for both CPUs, and is the currently recommended mitigation for this bug.

Comment 23 Andrea Bolognani 2019-11-19 11:06:47 UTC

(In reply to David Gibson from comment #22)
> Jiri, update text looks better to me.  One final revision:
> 
> I don't think there's any point suggesting different options for the
> different CPU versions.  I'd just suggest cap-ibs=workaround in all cases. 
> That should work for both CPUs, and is the currently recommended mitigation
> for this bug.

Hold on. The doc text reads

  To work around this problem, add one of the following strings to the
  kernel command line of the VM, depending on the CPU firmware:

  * For DD2.2: `pseries-rhel7.6.0-sxxm,cap-ibs=fixed-ibs`
  * For DD2.3: `pseries-rhel7.6.0-sxxm,cap-ibs=workaround`

which is clearly not correct, as it's not the *kernel* command line
that needs tweaking but rather the *QEMU* command line.


Additionally, since we don't support running QEMU directly, we need
to document the solution in a libvirt-compatible format: either by
implementing the flag properly, as suggested earlier by Gu Nini, or
at the very least by providing an equivalent XML snippet such as
(untested)

  <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
    <qemu:commandline>
      <qemu:arg value='-machine'/>
      <qemu:arg value='pseries-rhel7.6.0-sxxm,cap-ibs=workaround'/>
    </qemu:commandline>
    ...

Comment 24 David Gibson 2019-11-20 01:33:52 UTC

Duh.  Good point, I totally missed that Andrea.

Yes, this is about the qemu command line, not the kernel command line.

Jiři, yes I think we're going to need the XML workaround instead.

I guess for now we can just use the hack that puts the qemu command line argument into the XML, though longer term it might be worth adding proper support for the parameter to libvirt.  We shouldn't need to repeat the machine type name in there, though.  So we could just use:

<domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <qemu:commandline>
    <qemu:arg value='-machine'/>
    <qemu:arg value='cap-ibs=workaround'/>
  </qemu:commandline>
</domain>

Comment 26 Andrea Bolognani 2019-12-12 08:30:21 UTC

According to what David said in Comment 22 and Comment 24, the XML
snippet doesn't need to be different for the two CPU models and the
machine type doesn't need to be included, so we should be able to
apply the following diff:

--- doctext.orig	2019-12-12 09:27:19.132159938 +0100
+++ doctext	2019-12-12 09:27:46.482350262 +0100
@@ -10,22 +10,10 @@
 
 To work around this problem, configure the virtual machine's XML configuration as follows:
 
-* For DD2.2:
-
-----
-  <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
-    <qemu:commandline>
-      <qemu:arg value='-machine'/>
-      <qemu:arg value='pseries-rhel7.6.0-sxxm,cap-ibs=fixed-ibs'/>
-    </qemu:commandline>
-----
-
-* For DD2.3:
-
 ----
   <domain type='qemu' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
     <qemu:commandline>
       <qemu:arg value='-machine'/>
-      <qemu:arg value='pseries-rhel7.6.0-sxxm,cap-ibs=workaround'/>
+      <qemu:arg value='cap-ibs=workaround'/>
     </qemu:commandline>
 ----

Comment 27 Jiri Herrmann 2019-12-12 10:31:42 UTC

Understood and updated :-) Thank you again folks!

Comment 30 RHEL Program Management 2021-02-01 07:43:56 UTC

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Note You need to log in before you can comment on or make changes to this bug.