Bug 1032874 - qemu-kvm with vfio Killed when boot with "numactl -m 1"
qemu-kvm with vfio Killed when boot with "numactl -m 1"
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm (Show other bugs)
7.0
Unspecified Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: Alex Williamson
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-11-21 00:49 EST by ShupingCui
Modified: 2016-05-16 00:07 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-22 14:31:21 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
strace debug file (134.53 KB, text/plain)
2013-11-21 00:50 EST, ShupingCui
no flags Details

  None (edit)
Description ShupingCui 2013-11-21 00:49:46 EST
Description of problem:
qemu-kvm with vfio Killed when boot with "numactl -m 1"

Version-Release number of selected component (if applicable):
qemu-kvm-1.5.3-19.el7.x86_64
kernel-3.10.0-50.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. boot guest with numactl
numactl \
    -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
    -name 'virt-tests-vm1'  \
    -sandbox on \
    -M pc  \
    -nodefaults  \
    -vga cirrus  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20131119-170324-eboEDTcp,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20131119-170324-eboEDTcp,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20131119-170324-eboEDTcp,path=/tmp/seabios-20131119-170324-eboEDTcp,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20131119-170324-eboEDTcp,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=03 \
    -drive id=drive_image1,if=none,cache=none,snapshot=on,aio=native,file=/root/staf-kvm-devel/autotest/client/tests/virt/shared/data/images/RHEL-Server-6.5-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04 \
    -device vfio-pci,host=0000:05:00.0,id=id_0000.05.00.0,bus=pci.0,addr=05 \
    -device vfio-pci,host=0000:05:00.1,id=id_0000.05.00.1,bus=pci.0,addr=06  \
    -m 4096  \
    -smp 8,maxcpus=8,cores=4,threads=1,sockets=2  \
    -cpu 'Nehalem' \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off  \
    -no-kvm-pit-reinjection \
    -enable-kvm \
    -monitor stdio

2.
3.

Actual results:
(qemu) Killed

Expected results:
guest boot up successful

Additional info:
processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           E5530  @ 2.40GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2393.991
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 4787.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-50.el7.x86_64 root=/dev/mapper/rhel_intel--e5530--8--2-root ro console=tty0 vconsole.keymap=us crashkernel=auto reboot=pci console=ttyS0,115200 rd.lvm.lv=rhel_intel-e5530-8-2/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_intel-e5530-8-2/root biosdevname=0 crashkernel=256M rhgb quiet intel_iommu=on

# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14
node 0 size: 4096 MB
node 0 free: 3880 MB
node 1 cpus: 1 3 5 7 9 11 13 15
node 1 size: 4083 MB
node 1 free: 3622 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 

trace the qemu-kvm:
# strace numactl -m 1  /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu     -name 'virt-tests-vm1'      -sandbox off     -M pc      -nodefaults      -vga cirrus      -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20131119-170324-eboEDTcp,server,nowait     -mon chardev=qmp_id_qmpmonitor1,mode=control      -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20131119-170324-eboEDTcp,server,nowait     -device isa-serial,chardev=serial_id_serial0      -chardev socket,id=seabioslog_id_20131119-170324-eboEDTcp,path=/tmp/seabios-20131119-170324-eboEDTcp,server,nowait     -device isa-debugcon,chardev=seabioslog_id_20131119-170324-eboEDTcp,iobase=0x402     -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=03     -drive id=drive_image1,if=none,cache=none,snapshot=on,aio=native,file=/root/staf-kvm-devel/autotest/client/tests/virt/shared/data/images/RHEL-Server-6.5-64-virtio.qcow2     -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04     -device vfio-pci,host=0000:05:00.0,id=id_0000.05.00.0,bus=pci.0,addr=05     -device vfio-pci,host=0000:05:00.1,id=id_0000.05.00.1,bus=pci.0,addr=06      -m 4096      -smp 8,maxcpus=8,cores=4,threads=1,sockets=2      -cpu 'Nehalem'     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1      -vnc :0      -rtc base=utc,clock=host,driftfix=slew      -boot order=cdn,once=c,menu=off      -no-kvm-pit-reinjection     -enable-kvm     -monitor stdio

...
ioctl(23, VFIO_GET_API_VERSION, 0x17)   = 0
ioctl(23, VFIO_CHECK_EXTENSION, 0x1)    = 1
ioctl(22, VFIO_GROUP_SET_CONTAINER, 0x7fff988bf2a0) = 0
ioctl(23, VFIO_SET_IOMMU, 0x1)          = 0
ioctl(23, VFIO_IOMMU_MAP_DMA, 0x7fff988bf110) = 0
ioctl(23, VFIO_IOMMU_MAP_DMA, 0x7fff988bf110) = 0
ioctl(23, VFIO_IOMMU_MAP_DMA, 0x7fff988bf110) = 0
ioctl(23, VFIO_IOMMU_MAP_DMA <unfinished ...>
+++ killed by SIGKILL +++
Killed
Comment 1 ShupingCui 2013-11-21 00:50:36 EST
Created attachment 827004 [details]
strace debug file
Comment 3 Alex Williamson 2013-11-22 14:31:21 EST
(In reply to ShupingCui from comment #0)
> Steps to Reproduce:
> 1. boot guest with numactl
> numactl \
>     -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
...
>     -m 4096  \
...
> # numactl --hardware
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6 8 10 12 14
> node 0 size: 4096 MB
> node 0 free: 3880 MB
> node 1 cpus: 1 3 5 7 9 11 13 15
> node 1 size: 4083 MB
> node 1 free: 3622 MB
               ^^^^

From numactl man page:

       --membind=nodes, -m nodes
              Only allocate memory from  nodes.   Allocation  will  fail  when
              there  is not enough memory available on these nodes.  nodes may
              be specified as noted above.

So you're trying to start a 4G VM, forcing memory to be allocated from a node that does not have 4G available and the allocation fails as the numactl man page says it will... sounds like notabug.  Re-open if there's something I'm missing.  I've tested using membind within the size available on my host and it works fine.
Comment 4 Xiaoqing Wei 2013-11-27 03:48:57 EST
(In reply to Alex Williamson from comment #3)
> (In reply to ShupingCui from comment #0)
> > Steps to Reproduce:
> > 1. boot guest with numactl
> > numactl \
> >     -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
> ...
> >     -m 4096  \
> ...
> > # numactl --hardware
> > available: 2 nodes (0-1)
> > node 0 cpus: 0 2 4 6 8 10 12 14
> > node 0 size: 4096 MB
> > node 0 free: 3880 MB
> > node 1 cpus: 1 3 5 7 9 11 13 15
> > node 1 size: 4083 MB
> > node 1 free: 3622 MB
>                ^^^^
> 
> From numactl man page:
> 
>        --membind=nodes, -m nodes
>               Only allocate memory from  nodes.   Allocation  will  fail 
> when
>               there  is not enough memory available on these nodes.  nodes
> may
>               be specified as noted above.
> 
> So you're trying to start a 4G VM, forcing memory to be allocated from a
> node that does not have 4G available and the allocation fails as the numactl
> man page says it will... sounds like notabug.  Re-open if there's something
> I'm missing.  I've tested using membind within the size available on my host
> and it works fine.

scui, pls retry w/ lower ram, and update the result here, thx.
Comment 5 ShupingCui 2013-11-27 03:54:07 EST
(In reply to Xiaoqing Wei from comment #4)
> (In reply to Alex Williamson from comment #3)
> > (In reply to ShupingCui from comment #0)
> > > Steps to Reproduce:
> > > 1. boot guest with numactl
> > > numactl \
> > >     -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
> > ...
> > >     -m 4096  \
> > ...
> > > # numactl --hardware
> > > available: 2 nodes (0-1)
> > > node 0 cpus: 0 2 4 6 8 10 12 14
> > > node 0 size: 4096 MB
> > > node 0 free: 3880 MB
> > > node 1 cpus: 1 3 5 7 9 11 13 15
> > > node 1 size: 4083 MB
> > > node 1 free: 3622 MB
> >                ^^^^
> > 
> > From numactl man page:
> > 
> >        --membind=nodes, -m nodes
> >               Only allocate memory from  nodes.   Allocation  will  fail 
> > when
> >               there  is not enough memory available on these nodes.  nodes
> > may
> >               be specified as noted above.
> > 
> > So you're trying to start a 4G VM, forcing memory to be allocated from a
> > node that does not have 4G available and the allocation fails as the numactl
> > man page says it will... sounds like notabug.  Re-open if there's something
> > I'm missing.  I've tested using membind within the size available on my host
> > and it works fine.
> 
> scui, pls retry w/ lower ram, and update the result here, thx.

tried with "-m 2048", no killed.

Thanks,
Shuping

Note You need to log in before you can comment on or make changes to this bug.