Hide Forgot
Description of problem: Boot one guest with hugepage binding to one node which doesn't have enough hugepage, when run dd cmd in guest, guest hangs, and hmp prints "(qemu) error: kvm run failed Bad address". QE think qemu should prevent guest booting when no enough hugepages in the binded node. Version-Release number of selected component (if applicable): qemu-kvm-rhev-2.5.0-4.el7 kernel-3.10.0-373.el7.x86_64 How reproducible: always Steps to Reproduce: 1. In host(4 nodes), allocate 512M hugepage on each node. # cat /proc/meminfo | grep -i huge AnonHugePages: 6144 kB HugePages_Total: 1024 HugePages_Free: 1024 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB # numastat -m | grep -i huge AnonHugePages 0.00 0.00 6.00 0.00 6.00 HugePages_Total 512.00 512.00 512.00 512.00 2048.00 HugePages_Free 512.00 512.00 512.00 512.00 2048.00 HugePages_Surp 0.00 0.00 0.00 0.00 0.00 2. boot guest with 1G hugepage binding to node0 /usr/libexec/qemu-kvm -m 1G,slots=4,maxmem=32G -smp 4 \ -object memory-backend-file,policy=bind,mem-path=/mnt/kvm_hugepage,host-nodes=0,size=1G,id=mem-mem1 -device pc-dimm,id=dimm-mem1,memdev=mem-mem1 \ -drive file=/home/guest/RHEL-Server-7.3-64-virtio.qcow2,id=drive-virtio-disk1,media=disk,cache=none,snapshot=off,format=qcow2,aio=native,if=none -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,bootindex=0 -monitor stdio -vnc :0 3. run dd cmd in guest to allocate memory # mount -t tmpfs none /mnt # dd if=/dev/zero of=/mnt/test bs=1M count=1000 Actual results: After step2, guest can boot successfully. After step3, guest hangs, and hmp prints: " (qemu) error: kvm run failed Bad address RAX=0000000000000000 RBX=ffff88003e001b00 RCX=0000000000000000 RDX=0000000000000040 RSI=ffff880122800000 RDI=ffff88003e001b00 RBP=ffff88012feb3a60 RSP=ffff88012feb3a38 R8 =0000000000000004 R9 =0000000000000000 R10=0000000000000003 R11=ffffffffffffffe2 R12=ffff880122800000 R13=ffffea00048a0000 R14=ffff880122800000 R15=ffff880122800000 RIP=ffffffff811be8d5 RFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 ffffffff 00800000 CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] SS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =0000 0000000000000000 ffffffff 00800000 FS =0000 00007f9ae55c8740 ffffffff 00800000 GS =0000 ffff88003fd80000 ffffffff 00800000 LDT=0000 0000000000000000 0000ffff 00000000 TR =0040 ffff88003fd91b40 00002087 00008b00 DPL=0 TSS64-busy GDT= ffff88003fd89000 0000007f IDT= ffffffffff529000 00000fff CR0=8005003b CR2=000000000bcf8003 CR3=000000003af47000 CR4=000007e0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000d01 Code=89 e7 49 89 cc 4c 89 fe 48 89 df e8 5f fc ff ff 48 63 43 20 <4d> 89 24 07 41 0f b7 55 1a 8b 43 18 81 e2 ff 7f 00 00 48 63 c8 0f af d0 4c 01 e1 48 63 d2 " Expected results: After step2, qemu should quit and print "unable to map backing store for hugepages: Cannot allocate memory" Additional info:
Correct comment0 step1, add mount step: # mount -t hugetlbfs none /mnt/kvm_hugepage
Please, use the -mem-prealloc option. QEMU's default behavior by design is to continue when hugepage allocation fails.
(In reply to Luiz Capitulino from comment #3) > Please, use the -mem-prealloc option. QEMU's default behavior by design is > to continue when hugepage allocation fails. Actually, when memory-backend-file(mem-mem1)'s size is 2G and without -mem-prealloc option, qemu quits directly and prints "unable to map backing store for hugepages: Cannot allocate memory". Should I file a new bug for this ?
I guess this may have something to do with the numa memory policy you're using, but it's better to check with the memory backend device maintainer. Igor?
(In reply to Yumei Huang from comment #4) > (In reply to Luiz Capitulino from comment #3) > > Please, use the -mem-prealloc option. QEMU's default behavior by design is > > to continue when hugepage allocation fails. > > Actually, when memory-backend-file(mem-mem1)'s size is 2G and without > -mem-prealloc option, qemu quits directly and prints "unable to map backing > store for hugepages: Cannot allocate memory". > > Should I file a new bug for this ? memory is allocated before 'bind' policy comes in to effect, so it's the same for 1Gb or 2Gb case. Allocation may fail at early depending on available resources or may fail later at runtime when accessing not yet allocated page. I'd say what you see here is normal. To ensure that guest won't hang/crash for not enough memory reason, one needs to use mem-prealloc option.
*** Bug 1455450 has been marked as a duplicate of this bug. ***