Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1662228 - [RFE] Allow HPT (POWER8 compat) nested guests (bottom level only) [qemu changes]
Summary: [RFE] Allow HPT (POWER8 compat) nested guests (bottom level only) [qemu changes]
Alias: None
Product: Red Hat Enterprise Linux Advanced Virtualization
Classification: Red Hat
Component: qemu-kvm
Version: 8.1
Hardware: ppc64le
OS: Linux
Target Milestone: rc
: ---
Assignee: Virtualization Maintenance
QA Contact: Zhenyu Zhang
Depends On: 1661474
TreeView+ depends on / blocked
Reported: 2018-12-27 06:43 UTC by Yihuang Yu
Modified: 2020-11-26 00:36 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2020-11-26 00:36:45 UTC
Type: Feature Request
Target Upstream Version:

Attachments (Terms of Use)

Description Yihuang Yu 2018-12-27 06:43:08 UTC
Description of problem:
qemu will report "qemu-kvm: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory" when I want to launch an L2 guest if I set "max-cpu-compat=power8" and memory > 2G. When memory = 2G, see BZ 1661474

Version-Release number of selected component (if applicable):
host kernel version: 4.18.0-57.el8.ppc64le
host qemu version: qemu-kvm-2.12.0-51.module+el8+2608+a17c4bfe.ppc64le
L1 guest kernel version: 4.18.0-57.el8.ppc64le
L1 guest qemu version: qemu-kvm-2.12.0-51.module+el8+2608+a17c4bfe.ppc64le

How reproducible:

Steps to Reproduce:
1. Launch an L1 guest
/usr/libexec/qemu-kvm \
-name 'nested-L1' \
-sandbox off \
-machine pseries,cap-nested-hv=on \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-chardev socket,id=serial_id_serial0,path=/var/tmp/serial,server,nowait \
-device spapr-vty,reg=0x30000000,chardev=serial_id_serial0,id=serial0 \
-device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
-blockdev driver=file,filename=/home/nested/L1new.qcow2,cache.direct=off,cache.no-flush=on,node-name=L1disk \
-blockdev driver=qcow2,file=L1disk,node-name=drive_image1 \
-device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4 \
-device scsi-hd,id=image1,drive=drive_image1,werror=stop,rerror=stop \
-device virtio-net-pci,mac=9a:4a:4b:4c:4d:4e,id=nic0,mq=on,vectors=4,netdev=virtio_nic,bus=pci.0,addr=0x5 \
-netdev tap,id=virtio_nic,vhost=on,script=/etc/qemu-ifup \
-m 128G,slots=4,maxmem=192G \
-smp 64,maxcpus=64,cores=16,threads=2,sockets=2 \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-device usb-kbd,id=usb-kbdppc1,bus=usb1.0,port=2  \
-vnc :0 \
-rtc base=utc,clock=host \
-boot order=cdn,once=c,menu=off,strict=off \
-enable-kvm \
-monitor stdio

2. Launch an L2 guest with Power8 compat mode
/usr/libexec/qemu-kvm \
-name 'nested-L2' \
-sandbox off \
-machine pseries,max-cpu-compat=power8 \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-chardev socket,id=serial_id_serial0,path=/var/tmp/serial,server,nowait \
-device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \
-device nec-usb-xhci,id=usb2,bus=pci.0,addr=0x3 \
-object iothread,id=iothread0 \
-blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/nested/L2.qcow2,node-name=L2disk \
-blockdev driver=qcow2,file=L2disk,node-name=drive_image2 \
-device virtio-blk-pci,drive=drive_image2,id=virtio-L2disk,bus=pci.0,addr=0x6,werror=stop,rerror=stop \
-device virtio-net-pci,mac=9a:6a:6b:6c:6d:6e,id=nic0,vectors=4,netdev=virtio_nic,bus=pci.0,addr=0x5 \
-netdev tap,id=virtio_nic,vhost=on,script=/etc/qemu-ifup \
-m 4G \
-smp 8,maxcpus=16,cores=4,threads=1,sockets=2 \
-device usb-tablet,id=usb-tablet2,bus=usb2.0,port=1 \
-device usb-kbd,id=usb-kbdppc2,bus=usb2.0,port=2  \
-vnc :0 \
-qmp tcp:0:4441,server,nowait \
-rtc base=utc,clock=host \
-boot menu=off,strict=off,order=cdn,once=c \
-enable-kvm \
-monitor stdio

Actual results:
QEMU 2.12.0 monitor - type 'help' for more information
(qemu) qemu-kvm: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory

Expected results:
L2 guest can be launched normally.

Additional info:

Comment 1 Yihuang Yu 2018-12-27 06:45:00 UTC
This is a ppc only bug.

Comment 2 Serhii Popovych 2018-12-28 12:29:38 UTC
We have problems with THP support for power8 compatibility mode on power9.
More details can be found in bz1629531#c18 where problem is tracked.

I suspect we need to retest after bz1629531 is fixed.

Comment 4 David Gibson 2019-01-03 02:29:25 UTC
HPT (POWER8 compat) guests are not supported with nesting, at any level.  We might eventually allow it for the bottom level, but that will essentially be a new feature.  Refiling accordingly.

Comment 5 David Gibson 2019-07-29 05:09:34 UTC
Since this is essentially for testing setups, I think we only want to aim for the qemu parts in RHEL-AV, rather than backport to base RHEL.

Comment 6 David Gibson 2019-09-02 05:20:33 UTC
Suraj has left IBM.

Comment 8 Ademar Reis 2020-02-05 22:53:24 UTC
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks

Comment 10 David Gibson 2020-11-26 00:36:45 UTC
I've discussed this with Paul Mackerras, and it doesn't look like it's going to be worthwhile to implement this:
  * It's quite a lot of work, and we don't have any good candidates to do it
  * HPT guests are becoming increasingly irrelevant
  * With POWER10 we'll be moving to nested KVM-under-PowerVM only, and PowerVM aren't intending to allow HPT based L2 guests, so this feature would have no long term future.

Therefore, closing.

Note You need to log in before you can comment on or make changes to this bug.