Bug 1751054

Summary: Forward migration for guest with '-machine pseries-rhel7.6.0-sxxm'(using the default 'cap-ibs') from RHEL(ALT) 7.6 host to RHEL(AV) 8.1.0 host(with DD2.3 cpu revision) can't be done
Product: Red Hat Enterprise Linux 7 Reporter: Gu Nini <ngu>
Component: Red_Hat_Enterprise_Linux-Release_Notes-7-en-USAssignee: Lenka Špačková <lkuprova>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: medium Docs Contact: Jiri Herrmann <jherrman>
Priority: medium    
Version: 7.6CC: dgibson, jherrman, knoel, lkuprova, lvivier, mdeng, micai, pkovar, qzhang, rhel-docs, xianwang, xuma, yihyu, zhenyzha
Target Milestone: rcKeywords: Documentation
Target Release: ---   
Hardware: ppc64le   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
.RHEL 7 virtual machines sometimes cannot boot on and migrate to Witherspoon hosts RHEL 7 virtual machines (VMs) that use the `pseries-rhel7.6.0-sxxm` machine type in some cases fail to boot on _Power9 S922LC for HPC_ hosts (also known as Witherspoon) that use the DD2.3 CPU. Attempting to boot such a VM instead generates the following error message: ---- qemu-kvm: Requested safe indirect branch capability level not supported by kvm ---- In addition, migrating VMs that use the `pseries-rhel7.6.0-sxxm` machine type to Witherspoon hosts from other hosts fails.
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-01-28 15:25:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Gu Nini 2019-09-11 06:06:25 UTC
Document URL: 

Section Number and Name: 

Describe the issue: 
For the migration test from RHEL 7.6 or RHEL ALT 7.6(POWER8 or POWER9) to RHEL 8.1.0 or RHELAV 8.1.0(POWER9 with DD2.3 firmware), when '-machine pseries-rhel7.6.0-sxxm'(using the default 'cap-ibs=fixed-ccd') is used to start up the src guest, the migration can't be carried out since '-machine pseries-rhel7.6.0-sxxm,cap-ibs=workaround' or '-machine pseries-rhel7.6.0-sxxm,cap-ibs=broken' must be used to start up the dst guest(on POWER9 DD2.3 host).

To do the migration, the src guest should have been booted up with '-machine pseries-rhel7.6.0-sxxm,cap-ibs=broken' since 'cap-ibs=workaround' is not supported on the host qemu.

For details, please refer to:
https://bugzilla.redhat.com/show_bug.cgi?id=1700699#c1
https://bugzilla.redhat.com/show_bug.cgi?id=1710663#c17


####SRC HOST Info:
[root@ibm-p8-08 ipa]# /usr/libexec/qemu-kvm -machine pseries-rhel7.6.0-sxxm,cap-ibs=?
qemu-kvm: broken - no protection, fixed-ibs - indirect branch serialisation, fixed-ccd - cache count disabled
[root@ibm-p8-08 ipa]# /usr/libexec/qemu-kvm -machine pseries-rhel7.6.0-sxxm,cap-ibs=fixed-ibs -monitor stdio
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=fixed-ccd
[root@ibm-p8-08 ipa]# /usr/libexec/qemu-kvm -machine pseries-rhel7.6.0-sxxm,cap-ibs=fixed-ccd -monitor stdio
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900
(qemu)
[root@ibm-p8-08 ipa]# 
[root@ibm-p8-08 ipa]# uname -r
3.10.0-957.27.2.el7.ppc64le
[root@ibm-p8-08 ipa]# 
[root@ibm-p8-08 ipa]# rpm -qa|grep qemu
qemu-img-rhev-2.12.0-18.el7_6.7.ppc64le
qemu-kvm-common-rhev-2.12.0-18.el7_6.7.ppc64le
qemu-kvm-tools-rhev-2.12.0-18.el7_6.7.ppc64le
ipxe-roms-qemu-20170123-1.git4e85b27.el7_4.1.noarch
qemu-kvm-rhev-2.12.0-18.el7_6.7.ppc64le
[root@ibm-p8-08 ipa]# 

####DST Host Info:
[root@ibm-p9wr-06 ~]# /usr/libexec/qemu-kvm -machine pseries,cap-ibs=?
qemu-kvm: broken - no protection, workaround - count cache flush, fixed-ibs - indirect branch serialisation, fixed-ccd - cache count disabled, fixed-na - fixed in hardware (no longer applicable)
[root@ibm-p9wr-06 ~]# 
[root@ibm-p9wr-06 ipa]# /usr/libexec/qemu-kvm -machine pseries,cap-ibs=fixed-ibs -monitor stdio
QEMU 4.1.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=workaround
[root@ibm-p9wr-06 ipa]# 
[root@ibm-p9wr-06 ipa]# /usr/libexec/qemu-kvm -machine pseries,cap-ibs=fixed-ccd -monitor stdio
QEMU 4.1.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900
qemu-kvm: Requested safe indirect branch capability level not supported by kvm, try cap-ibs=workaround
[root@ibm-p9wr-06 ipa]# 
[root@ibm-p9wr-06 ipa]# /usr/libexec/qemu-kvm -machine pseries,cap-ibs=workaround -monitor stdio
QEMU 4.1.0 monitor - type 'help' for more information
(qemu) VNC server running on ::1:5900

(qemu) 
[root@ibm-p9wr-06 ipa]# uname -r
4.18.0-141.el8.ppc64le
[root@ibm-p9wr-06 ipa]# rpm -qa|grep qemu
ipxe-roms-qemu-20181214-3.git133f4c47.el8.noarch
qemu-kvm-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-ssh-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-iscsi-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-img-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-ssh-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-iscsi-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-tests-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-common-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-core-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-debugsource-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-common-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-rbd-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-curl-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-core-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-tests-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-rbd-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-kvm-block-curl-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-guest-agent-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-img-debuginfo-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
qemu-guest-agent-4.1.0-8.module+el8.1.0+4199+446e40fc.ppc64le
[root@ibm-p9wr-06 ipa]# 
[root@ibm-p9wr-06 ipa]# lsprop /proc/device-tree/ibm,firmware-versions/
skiboot          "v6.3-rc1-p1ce8930"
bmc-firmware-version
		 "2.03"
occ              "58e422d"
hostboot         "e5622fb"
buildroot        "2019.02.1-16-ge01dcd0"
capp-ucode       "p9-dd2-v4"
machine-xml      "e3e9aef"
hostboot-binaries
		 "hw021419a.930"
sbe              "1410677"
hcode            "hw040319a.940"
petitboot        "v1.10.3"
phandle          00000799 (1945)
version          "witherspoon-ibm-OP9-v2.2-3.5"
linux            "5.0.5-openpower1-p4b42b5c"
name             "ibm,firmware-versions"


Suggestions for improvement: 
Suggest to add above limitation to the doc to inform customers.

Additional information:
When the DST host is with qemu-kvm-2.12.0-86.module+el8.1.0+4146+4ed2d185.ppc64le, there is the same limitation.

Comment 18 Michal Bocek 2019-10-01 11:02:08 UTC
> I'm not sure that RHEL-ALT-7.6 will even run on a DD2.3 host at all.
I've just tried that before commenting. I chose one of the Beaker machines from the link you provided above (ibm-p9wr-08) and the RHEL-ALT-7.6 installed fine, and I could log in. But I haven't tried running a VM with pseries-rhel7.6.0-sxxm, that's true.

So from what you write, there's really nothing related to in-place upgrades at all.

Feel free to change the bugzilla component.

Comment 19 David Gibson 2019-10-02 01:09:18 UTC
> > I'm not sure that RHEL-ALT-7.6 will even run on a DD2.3 host at all.
> I've just tried that before commenting. I chose one of the Beaker machines from the link you provided above (ibm-p9wr-08) and the RHEL-ALT-7.6 installed fine, and I could log in. But I haven't tried running a VM with pseries-rhel7.6.0-sxxm, that's true.

Ok.  If that does start, it's a security bug and we need to fix it immediately.

> So from what you write, there's really nothing related to in-place upgrades at all.

Correct - that's what I've been trying to say all along here.  I'm not sure how we got down this wrong track.

> Feel free to change the bugzilla component.

I'm not sure who that's addressed to.  I certainly don't know enough about RHV architecture to know what the right component is.

Comment 20 Petr Stodulka 2019-10-02 07:28:39 UTC
As discusseed with David, this is probably just thing for documentation. I do not know, who is doc guy for RHV (Gu Nini? Lenka?).

Comment 33 Lenka Špačková 2020-01-28 15:25:09 UTC
The release note has been published with RHEL 7.8 Beta:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7-beta/html-single/7.8_release_notes/index#known-issue_virtualization

and will be updated if it is fixed in the future (as any other known issues documented in the Release Notes).

Thank you all!