Description of problem: qemu will report "qemu-kvm: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory" when I want to launch an L2 guest if I set "max-cpu-compat=power8" and memory > 2G. When memory = 2G, see BZ 1661474 Version-Release number of selected component (if applicable): host kernel version: 4.18.0-57.el8.ppc64le host qemu version: qemu-kvm-2.12.0-51.module+el8+2608+a17c4bfe.ppc64le L1 guest kernel version: 4.18.0-57.el8.ppc64le L1 guest qemu version: qemu-kvm-2.12.0-51.module+el8+2608+a17c4bfe.ppc64le How reproducible: 100% Steps to Reproduce: 1. Launch an L1 guest /usr/libexec/qemu-kvm \ -name 'nested-L1' \ -sandbox off \ -machine pseries,cap-nested-hv=on \ -nodefaults \ -device VGA,bus=pci.0,addr=0x2 \ -chardev socket,id=serial_id_serial0,path=/var/tmp/serial,server,nowait \ -device spapr-vty,reg=0x30000000,chardev=serial_id_serial0,id=serial0 \ -device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \ -blockdev driver=file,filename=/home/nested/L1new.qcow2,cache.direct=off,cache.no-flush=on,node-name=L1disk \ -blockdev driver=qcow2,file=L1disk,node-name=drive_image1 \ -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=0x4 \ -device scsi-hd,id=image1,drive=drive_image1,werror=stop,rerror=stop \ -device virtio-net-pci,mac=9a:4a:4b:4c:4d:4e,id=nic0,mq=on,vectors=4,netdev=virtio_nic,bus=pci.0,addr=0x5 \ -netdev tap,id=virtio_nic,vhost=on,script=/etc/qemu-ifup \ -m 128G,slots=4,maxmem=192G \ -smp 64,maxcpus=64,cores=16,threads=2,sockets=2 \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -device usb-kbd,id=usb-kbdppc1,bus=usb1.0,port=2 \ -vnc :0 \ -rtc base=utc,clock=host \ -boot order=cdn,once=c,menu=off,strict=off \ -enable-kvm \ -monitor stdio 2. Launch an L2 guest with Power8 compat mode /usr/libexec/qemu-kvm \ -name 'nested-L2' \ -sandbox off \ -machine pseries,max-cpu-compat=power8 \ -nodefaults \ -device VGA,bus=pci.0,addr=0x2 \ -chardev socket,id=serial_id_serial0,path=/var/tmp/serial,server,nowait \ -device spapr-vty,reg=0x30000000,chardev=serial_id_serial0 \ -device nec-usb-xhci,id=usb2,bus=pci.0,addr=0x3 \ -object iothread,id=iothread0 \ -blockdev driver=file,cache.direct=off,cache.no-flush=on,filename=/home/nested/L2.qcow2,node-name=L2disk \ -blockdev driver=qcow2,file=L2disk,node-name=drive_image2 \ -device virtio-blk-pci,drive=drive_image2,id=virtio-L2disk,bus=pci.0,addr=0x6,werror=stop,rerror=stop \ -device virtio-net-pci,mac=9a:6a:6b:6c:6d:6e,id=nic0,vectors=4,netdev=virtio_nic,bus=pci.0,addr=0x5 \ -netdev tap,id=virtio_nic,vhost=on,script=/etc/qemu-ifup \ -m 4G \ -smp 8,maxcpus=16,cores=4,threads=1,sockets=2 \ -device usb-tablet,id=usb-tablet2,bus=usb2.0,port=1 \ -device usb-kbd,id=usb-kbdppc2,bus=usb2.0,port=2 \ -vnc :0 \ -qmp tcp:0:4441,server,nowait \ -rtc base=utc,clock=host \ -boot menu=off,strict=off,order=cdn,once=c \ -enable-kvm \ -monitor stdio Actual results: QEMU 2.12.0 monitor - type 'help' for more information (qemu) qemu-kvm: Failed to allocate KVM HPT of order 25 (try smaller maxmem?): Cannot allocate memory Expected results: L2 guest can be launched normally. Additional info:
This is a ppc only bug.
We have problems with THP support for power8 compatibility mode on power9. More details can be found in bz1629531#c18 where problem is tracked. I suspect we need to retest after bz1629531 is fixed.
HPT (POWER8 compat) guests are not supported with nesting, at any level. We might eventually allow it for the bottom level, but that will essentially be a new feature. Refiling accordingly.
Since this is essentially for testing setups, I think we only want to aim for the qemu parts in RHEL-AV, rather than backport to base RHEL.
Suraj has left IBM.
QEMU has been recently split into sub-components and as a one-time operation to avoid breakage of tools, we are setting the QEMU sub-component of this BZ to "General". Please review and change the sub-component if necessary the next time you review this BZ. Thanks
I've discussed this with Paul Mackerras, and it doesn't look like it's going to be worthwhile to implement this: * It's quite a lot of work, and we don't have any good candidates to do it * HPT guests are becoming increasingly irrelevant * With POWER10 we'll be moving to nested KVM-under-PowerVM only, and PowerVM aren't intending to allow HPT based L2 guests, so this feature would have no long term future. Therefore, closing.