Bug 1162989

Summary: qemu-kvm-rhev core dump when guest numa node hugepage size < host hugepage size
Product: Red Hat Enterprise Linux 7 Reporter: FuXiangChun <xfu>
Component: qemu-kvm-rhevAssignee: Eduardo Habkost <ehabkost>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.1CC: hhuang, jmiao, juzhang, lcapitulino, linchen, michen, virt-maint, ypu
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-12 15:22:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description FuXiangChun 2014-11-12 07:22:52 UTC
Description of problem:
First, add hugepagesz=1G default_hugepagesz=1G to host kernel line.  make sure host support 1G hugepage.  Then starting qemu-kvm process with "-object memory-backend-file,.....size=512M".  size < 1G hugepage.  qemu-kvm-rhev core dump.  QE tested libvirt as well.  libvirt show error message as below.

error: internal error: process exited while connecting to monitor: Bad ram offset fffffffffffff000


Version-Release number of selected component (if applicable):
3.10.0-195.el7.x86_64
qemu-kvm-rhev-2.1.2-6.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.make sure host support 1G hugepage
# cat /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages 
1

2.starting qemu-kvm-rhev process with cli

/usr/libexec/qemu-kvm -m 512 \

-object memory-backend-file,prealloc=yes,mem-path=/mnt/kvm_hugepage1/libvirt/qemu,size=512M,id=ram-node0,host-nodes=0,policy=bind  \

-numa node,nodeid=0,cpus=0-1,memdev=ram-node0 \

3.

Actual results:
(gdb) bt
#0  0x00007ffff1a075e9 in raise () from /lib64/libc.so.6
#1  0x00007ffff1a08cf8 in abort () from /lib64/libc.so.6
#2  0x0000555555622eca in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.1.2/exec.c:797
#3  0x0000555555625119 in qemu_get_ram_ptr (addr=18446744073709547520) at /usr/src/debug/qemu-2.1.2/exec.c:1512
#4  0x0000555555658ce7 in memory_region_get_ram_ptr (mr=mr@entry=0x555556192290) at /usr/src/debug/qemu-2.1.2/memory.c:1458
#5  0x00005555557627ec in host_memory_backend_memory_complete (uc=<optimized out>, errp=0x7fffffffdf90) at backends/hostmem.c:291
#6  0x0000555555736185 in object_add (type=<optimized out>, id=0x5555561904c0 "ram-node0", qdict=qdict@entry=0x555556190d50, v=0x555556190c50, 
    errp=errp@entry=0x7fffffffdfe8) at qmp.c:630
#7  0x0000555555724174 in object_create (opts=<optimized out>, opaque=<optimized out>) at vl.c:2899
#8  0x000055555589079b in qemu_opts_foreach (list=<optimized out>, func=func@entry=0x555555723fd0 <object_create>, opaque=opaque@entry=0x0, 
    abort_on_failure=abort_on_failure@entry=0) at util/qemu-option.c:1072
#9  0x000055555561d08c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4240
(gdb) 
#0  0x00007ffff1a075e9 in raise () from /lib64/libc.so.6
#1  0x00007ffff1a08cf8 in abort () from /lib64/libc.so.6
#2  0x0000555555622eca in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.1.2/exec.c:797
#3  0x0000555555625119 in qemu_get_ram_ptr (addr=18446744073709547520) at /usr/src/debug/qemu-2.1.2/exec.c:1512
#4  0x0000555555658ce7 in memory_region_get_ram_ptr (mr=mr@entry=0x555556192290) at /usr/src/debug/qemu-2.1.2/memory.c:1458
#5  0x00005555557627ec in host_memory_backend_memory_complete (uc=<optimized out>, errp=0x7fffffffdf90) at backends/hostmem.c:291
#6  0x0000555555736185 in object_add (type=<optimized out>, id=0x5555561904c0 "ram-node0", qdict=qdict@entry=0x555556190d50, v=0x555556190c50, 
    errp=errp@entry=0x7fffffffdfe8) at qmp.c:630
#7  0x0000555555724174 in object_create (opts=<optimized out>, opaque=<optimized out>) at vl.c:2899
#8  0x000055555589079b in qemu_opts_foreach (list=<optimized out>, func=func@entry=0x555555723fd0 <object_create>, opaque=opaque@entry=0x0, 
    abort_on_failure=abort_on_failure@entry=0) at util/qemu-option.c:1072
#9  0x000055555561d08c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4240
(gdb) 
#0  0x00007ffff1a075e9 in raise () from /lib64/libc.so.6
#1  0x00007ffff1a08cf8 in abort () from /lib64/libc.so.6
#2  0x0000555555622eca in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.1.2/exec.c:797
#3  0x0000555555625119 in qemu_get_ram_ptr (addr=18446744073709547520) at /usr/src/debug/qemu-2.1.2/exec.c:1512
#4  0x0000555555658ce7 in memory_region_get_ram_ptr (mr=mr@entry=0x555556192290) at /usr/src/debug/qemu-2.1.2/memory.c:1458
#5  0x00005555557627ec in host_memory_backend_memory_complete (uc=<optimized out>, errp=0x7fffffffdf90) at backends/hostmem.c:291
#6  0x0000555555736185 in object_add (type=<optimized out>, id=0x5555561904c0 "ram-node0", qdict=qdict@entry=0x555556190d50, v=0x555556190c50, 
    errp=errp@entry=0x7fffffffdfe8) at qmp.c:630
#7  0x0000555555724174 in object_create (opts=<optimized out>, opaque=<optimized out>) at vl.c:2899
#8  0x000055555589079b in qemu_opts_foreach (list=<optimized out>, func=func@entry=0x555555723fd0 <object_create>, opaque=opaque@entry=0x0, 
    abort_on_failure=abort_on_failure@entry=0) at util/qemu-option.c:1072
#9  0x000055555561d08c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4240
(gdb) 
#0  0x00007ffff1a075e9 in raise () from /lib64/libc.so.6
#1  0x00007ffff1a08cf8 in abort () from /lib64/libc.so.6
#2  0x0000555555622eca in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.1.2/exec.c:797
#3  0x0000555555625119 in qemu_get_ram_ptr (addr=18446744073709547520) at /usr/src/debug/qemu-2.1.2/exec.c:1512
#4  0x0000555555658ce7 in memory_region_get_ram_ptr (mr=mr@entry=0x555556192290) at /usr/src/debug/qemu-2.1.2/memory.c:1458
#5  0x00005555557627ec in host_memory_backend_memory_complete (uc=<optimized out>, errp=0x7fffffffdf90) at backends/hostmem.c:291
#6  0x0000555555736185 in object_add (type=<optimized out>, id=0x5555561904c0 "ram-node0", qdict=qdict@entry=0x555556190d50, v=0x555556190c50, 
    errp=errp@entry=0x7fffffffdfe8) at qmp.c:630
#7  0x0000555555724174 in object_create (opts=<optimized out>, opaque=<optimized out>) at vl.c:2899
#8  0x000055555589079b in qemu_opts_foreach (list=<optimized out>, func=func@entry=0x555555723fd0 <object_create>, opaque=opaque@entry=0x0, 
    abort_on_failure=abort_on_failure@entry=0) at util/qemu-option.c:1072
#9  0x000055555561d08c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4240

Expected results:
qemu-kvm-rhev show correctly and friendly error message. shouldn't core dump.

Additional info:
libivrt hit the same issue.

Comment 2 FuXiangChun 2014-11-12 07:50:15 UTC
Sorry,There are some redundant(repeat gdb) information in comment 0. This is only core trace information as below.

(gdb) bt
#0  0x00007ffff1a075e9 in raise () from /lib64/libc.so.6
#1  0x00007ffff1a08cf8 in abort () from /lib64/libc.so.6
#2  0x0000555555622eca in qemu_get_ram_block (addr=<optimized out>) at /usr/src/debug/qemu-2.1.2/exec.c:797
#3  0x0000555555625119 in qemu_get_ram_ptr (addr=18446744073709547520) at /usr/src/debug/qemu-2.1.2/exec.c:1512
#4  0x0000555555658ce7 in memory_region_get_ram_ptr (mr=mr@entry=0x555556192290) at /usr/src/debug/qemu-2.1.2/memory.c:1458
#5  0x00005555557627ec in host_memory_backend_memory_complete (uc=<optimized out>, errp=0x7fffffffdf90) at backends/hostmem.c:291
#6  0x0000555555736185 in object_add (type=<optimized out>, id=0x5555561904c0 "ram-node0", qdict=qdict@entry=0x555556190d50, v=0x555556190c50, 
    errp=errp@entry=0x7fffffffdfe8) at qmp.c:630
#7  0x0000555555724174 in object_create (opts=<optimized out>, opaque=<optimized out>) at vl.c:2899
#8  0x000055555589079b in qemu_opts_foreach (list=<optimized out>, func=func@entry=0x555555723fd0 <object_create>, opaque=opaque@entry=0x0, 
    abort_on_failure=abort_on_failure@entry=0) at util/qemu-option.c:1072
#9  0x000055555561d08c in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at vl.c:4240

Comment 3 Luiz Capitulino 2014-11-12 14:02:14 UTC
I think this one has been fixed as part of the fix for bug 1147354. I can provide a brew build for testing.

Comment 5 Eduardo Habkost 2014-11-12 15:22:52 UTC
I just confirmed that the fix for bug 1147354 fixes this one. Closing as duplicate.

*** This bug has been marked as a duplicate of bug 1147354 ***

Comment 6 Eduardo Habkost 2014-11-12 15:23:38 UTC
For reference, this is the error message that's expected after applying the fix for bug 1147354:
# /usr/libexec/qemu-kvm -m 512 -object memory-backend-file,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu,size=512M,id=ram-node0,host-nodes=0,policy=bind  -numa node,nodeid=0,cpus=0-1,memdev=ram-node0
qemu-kvm: -object memory-backend-file,prealloc=yes,mem-path=/dev/hugepages/libvirt/qemu,size=512M,id=ram-node0,host-nodes=0,policy=bind: memory size 0x20000000 must be equal to or larger than huge page size 0x40000000