Bug 1296800

Summary: -mem-prealloc option does not take effect when no huge page is allocated
Product: Red Hat Enterprise Linux 7 Reporter: Dan Zheng <dzheng>
Component: qemu-kvm-rhevAssignee: Luiz Capitulino <lcapitulino>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.3CC: chayang, dyuan, hhuang, huding, jherrman, jsuchane, juzhang, knoel, lcapitulino, lhuang, lmiksik, mkolaja, mrezanin, mzhan, sgordon, sherold, snagar, tlavigne, virt-maint, xfu, yafu, yuhuang, zpeng
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Fixed In Version: qemu-kvm-rhev-2.6.0-1.el7 Doc Type: Bug Fix
Doc Text:
Prior to this update, when the qemu-kvm service was used with the -mem-prealloc option to allocate huge pages but the operation failed, qemu-kvm incorrectly reverted to regular RAM usage. Now, qemu-kvm exits in the described situation as expected.
Story Points: ---
Clone Of:
: 1324523 (view as bug list) Environment:
Last Closed: 2016-11-07 20:53:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 1324523    

Description Dan Zheng 2016-01-08 06:45:21 UTC
Description of problem:

Guest can start with hugepage setting when no huge pages are allocated.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. There is no huge page allocated.
# cat /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
# ll /dev/hugepages/libvirt/qemu 
total 0

2. Guest XML is with below setting:


3. The guest can start up using system page instead of huge page.
# virsh start d1
Domain d1 started

# numastat -p `pidof qemu-kvm`

Per-node process memory usage (in MBs) for PID 12087 (qemu-kvm)
                           Node 0           Total
                  --------------- ---------------
Huge                         0.00            0.00
Heap                        13.29           13.29
Stack                        2.07            2.07
Private                    171.72          171.72
----------------  --------------- ---------------
Total                      187.07          187.07

# cat /proc/meminfo|grep Huge
AnonHugePages:    718848 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

4.The qemu command line is as below:
qemu     12087     1 85 14:27 ?        00:00:05 /usr/libexec/qemu-kvm -name d1 -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu SandyBridge -m 1024 -mem-prealloc -mem-path /dev/hugepages/libvirt/qemu -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid 5d7c2231-5f46-4b94-bbd8-e46f245e1402 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-d1/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x4 -drive file=/var/lib/libvirt/migrate/jeos-19-64.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:52:53:54,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-d1/org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -device usb-tablet,id=input0 -vnc -device cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on

5. If use qemu-kvm-rhev-2.3.0-31.el7.x86_64, above guest can not start up.
# virsh start d1
error: Failed to start domain d1
error: internal error: process exited while connecting to monitor: 2016-01-08T05:40:50.562987Z qemu-kvm: unable to map backing store for hugepages: Cannot allocate memory

Or using above qemu command can not start the qemu process, either.

6. Remove -mem-prealloc on qemu command line, then the qemu process can start up with qemu-kvm-rhev-2.3.0-31.el7.x86_64.

Actual results:
The qemu process can start up.

Expected results:
The qemu process should not start up with -mem-prealloc without hugepages allocated.

Additional info:
This problem also exists in qemu-kvm-rhev-2.3.0-31_2.2.el7.x86_64

Comment 2 Luiz Capitulino 2016-01-22 14:51:03 UTC
Posted fix upstream. will backport it soon.

Comment 7 Mike McCune 2016-03-28 22:55:34 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions

Comment 12 Yumei Huang 2016-05-18 03:23:58 UTC

1. # echo 0 > /proc/sys/vm/nr_hugepages

# cat /proc/meminfo | grep -i huge
AnonHugePages:      6144 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

# mount -t hugetlbfs hugetlbfs /mnt/kvm_hugepage

2. boot guest with -mem-prealloc 
# /usr/libexec/qemu-kvm  -m 1G,slots=8,maxmem=32G -smp 4  \

-mem-prealloc -mem-path /mnt/kvm_hugepage    \

-drive file=/home/guest/RHEL-Server-7.3-64-virtio.qcow2,id=drive-virtio-disk1,media=disk,cache=none,snapshot=off,format=qcow2,aio=native,if=none  -device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk1,bootindex=0  -netdev tap,id=idw97X5F  -device virtio-net-pci,netdev=idw97X5F,mac='9a:37:37:37:37:8e',bus=pci.0,addr=0x5,id='idMqFTXq' -vga cirrus   -monitor stdio  -vnc :0  

(qemu) qemu-kvm: unable to map backing store for hugepages: Cannot allocate memory
Guest starts up and works well. 
So the bug is reproduced.


With same steps as above, after step 2, 
(qemu) qemu-kvm: unable to map backing store for guest RAM: Cannot allocate memory
and qemu process quits directly. 
So the bug is fixed.

Comment 15 errata-xmlrpc 2016-11-07 20:53:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.