Bug 1662228

Summary:	[RFE] Allow HPT (POWER8 compat) nested guests (bottom level only) [qemu changes]
Product:	Red Hat Enterprise Linux Advanced Virtualization	Reporter:	Yihuang Yu <yihyu>
Component:	qemu-kvm	Assignee:	Virtualization Maintenance <virt-maint>
qemu-kvm sub component:	Devices	QA Contact:	Zhenyu Zhang <zhenyzha>
Status:	CLOSED WONTFIX	Docs Contact:
Severity:	medium
Priority:	medium	CC:	dgibson, knoel, mdeng, ngu, qzhang, rbalakri, virt-maint, xianwang, xuma, zhenyzha
Version:	8.1	Keywords:	FutureFeature, Triaged
Target Milestone:	rc
Target Release:	---
Hardware:	ppc64le
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-11-26 00:36:45 UTC	Type:	Feature Request
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1661474
Bug Blocks:

Description Yihuang Yu 2018-12-27 06:43:08 UTC

Description of problem:
qemu will report "qemu-kvm: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory" when I want to launch an L2 guest if I set "max-cpu-compat=power8" and memory > 2G. When memory = 2G, see BZ 1661474

Version-Release number of selected component (if applicable):
host kernel version: 4.18.0-57.el8.ppc64le
host qemu version: qemu-kvm-2.12.0-51.module+el8+2608+a17c4bfe.ppc64le
L1 guest kernel version: 4.18.0-57.el8.ppc64le
L1 guest qemu version: qemu-kvm-2.12.0-51.module+el8+2608+a17c4bfe.ppc64le

How reproducible:
100%

Steps to Reproduce:
1. Launch an L1 guest
/usr/libexec/qemu-kvm \
-name 'nested-L1' \
-sandbox off \
-machine pseries,cap-nested-hv=on \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-chardev socket,id=serial_id_serial0,path=/var/tmp/serial,server,nowait \
-device spapr-vty,reg=0x30000000,chardev=serial_id_serial0,id=serial0 \
-device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
-blockdev driver=file,filename=/home/nested/L1new.qcow2,cache.direct=off,cache.no-flush=on,node-name=L1disk \
-blockdev driver=qcow2,file=L1disk,node-name=drive_image1 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4 \
-device scsi-hd,id=image1,drive=drive_image1,werror=stop,rerror=stop \
-device virtio-net-pci,mac=9a:4a:4b:4c:4d:4e,id=nic0,mq=on,vectors=4,netdev=virtio_nic,bus=pci.0,addr=0x5 \
-netdev tap,id=virtio_nic,vhost=on,script=/etc/qemu-ifup \
-m 128G,slots=4,maxmem=192G \
-smp 64,maxcpus=64,cores=16,threads=2,sockets=2 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-device usb-kbd,id=usb-kbdppc1,bus=usb1.0,port=2  \
-vnc :0 \
-rtc base=utc,clock=host \
-boot order=cdn,once=c,menu=off,strict=off \
-enable-kvm \
-monitor stdio

2. Launch an L2 guest with Power8 compat mode
/usr/libexec/qemu-kvm \
-name 'nested-L2' \
-sandbox off \
-machine pseries,max-cpu-compat=power8 \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-chardev socket,id=serial_id_serial0,path=/var/tmp/serial,server,nowait \
-device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \
-device nec-usb-xhci,id=usb2,bus=pci.0,addr=0x3 \
-object iothread,id=iothread0 \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/nested/L2.qcow2,node-name=L2disk \
-blockdev driver=qcow2,file=L2disk,node-name=drive_image2 \
-device virtio-blk-pci,drive=drive_image2,id=virtio-L2disk,bus=pci.0,addr=0x6,werror=stop,rerror=stop \
-device virtio-net-pci,mac=9a:6a:6b:6c:6d:6e,id=nic0,vectors=4,netdev=virtio_nic,bus=pci.0,addr=0x5 \
-netdev tap,id=virtio_nic,vhost=on,script=/etc/qemu-ifup \
-m 4G \
-smp 8,maxcpus=16,cores=4,threads=1,sockets=2 \
-device usb-tablet,id=usb-tablet2,bus=usb2.0,port=1 \
-device usb-kbd,id=usb-kbdppc2,bus=usb2.0,port=2  \
-vnc :0 \
-qmp tcp:0:4441,server,nowait \
-rtc base=utc,clock=host \
-boot menu=off,strict=off,order=cdn,once=c \
-enable-kvm \
-monitor stdio

Actual results:
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) qemu-kvm: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory

Expected results:
L2 guest can be launched normally.

Additional info:

Comment 1 Yihuang Yu 2018-12-27 06:45:00 UTC

This is a ppc only bug.

Comment 2 Serhii Popovych 2018-12-28 12:29:38 UTC

We have problems with THP support for power8 compatibility mode on power9.
More details can be found in bz1629531#c18 where problem is tracked.

I suspect we need to retest after bz1629531 is fixed.

Comment 4 David Gibson 2019-01-03 02:29:25 UTC

HPT (POWER8 compat) guests are not supported with nesting, at any level.  We might eventually allow it for the bottom level, but that will essentially be a new feature.  Refiling accordingly.

Comment 5 David Gibson 2019-07-29 05:09:34 UTC

Since this is essentially for testing setups, I think we only want to aim for the qemu parts in RHEL-AV, rather than backport to base RHEL.

Comment 6 David Gibson 2019-09-02 05:20:33 UTC

Suraj has left IBM.

Comment 8 Ademar Reis 2020-02-05 22:53:24 UTC

QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 10 David Gibson 2020-11-26 00:36:45 UTC

I've discussed this with Paul Mackerras, and it doesn't look like it's going to be worthwhile to implement this:
  * It's quite a lot of work, and we don't have any good candidates to do it
  * HPT guests are becoming increasingly irrelevant
  * With POWER10 we'll be moving to nested KVM-under-PowerVM only, and PowerVM aren't intending to allow HPT based L2 guests, so this feature would have no long term future.

Therefore, closing.