Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1032874

Summary: qemu-kvm with vfio Killed when boot with "numactl -m 1"
Product: Red Hat Enterprise Linux 7 Reporter: ShupingCui <scui>
Component: qemu-kvmAssignee: Alex Williamson <alex.williamson>
Status: CLOSED NOTABUG QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.0CC: acathrow, bdas, chayang, hhuang, juzhang, michen, scui, shuang, virt-maint, xuhan, xwei
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-22 19:31:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
strace debug file none

Description ShupingCui 2013-11-21 05:49:46 UTC
Description of problem:
qemu-kvm with vfio Killed when boot with "numactl -m 1"

Version-Release number of selected component (if applicable):
qemu-kvm-1.5.3-19.el7.x86_64
kernel-3.10.0-50.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1. boot guest with numactl
numactl \
    -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
    -name 'virt-tests-vm1'  \
    -sandbox on \
    -M pc  \
    -nodefaults  \
    -vga cirrus  \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20131119-170324-eboEDTcp,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20131119-170324-eboEDTcp,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20131119-170324-eboEDTcp,path=/tmp/seabios-20131119-170324-eboEDTcp,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20131119-170324-eboEDTcp,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=03 \
    -drive id=drive_image1,if=none,cache=none,snapshot=on,aio=native,file=/root/staf-kvm-devel/autotest/client/tests/virt/shared/data/images/RHEL-Server-6.5-64-virtio.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04 \
    -device vfio-pci,host=0000:05:00.0,id=id_0000.05.00.0,bus=pci.0,addr=05 \
    -device vfio-pci,host=0000:05:00.1,id=id_0000.05.00.1,bus=pci.0,addr=06  \
    -m 4096  \
    -smp 8,maxcpus=8,cores=4,threads=1,sockets=2  \
    -cpu 'Nehalem' \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=utc,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off  \
    -no-kvm-pit-reinjection \
    -enable-kvm \
    -monitor stdio

2.
3.

Actual results:
(qemu) Killed

Expected results:
guest boot up successful

Additional info:
processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           E5530  @ 2.40GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2393.991
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 4787.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

# cat /proc/cmdline 
BOOT_IMAGE=/vmlinuz-3.10.0-50.el7.x86_64 root=/dev/mapper/rhel_intel--e5530--8--2-root ro console=tty0 vconsole.keymap=us crashkernel=auto reboot=pci console=ttyS0,115200 rd.lvm.lv=rhel_intel-e5530-8-2/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=rhel_intel-e5530-8-2/root biosdevname=0 crashkernel=256M rhgb quiet intel_iommu=on

# numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 2 4 6 8 10 12 14
node 0 size: 4096 MB
node 0 free: 3880 MB
node 1 cpus: 1 3 5 7 9 11 13 15
node 1 size: 4083 MB
node 1 free: 3622 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 

trace the qemu-kvm:
# strace numactl -m 1  /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu     -name 'virt-tests-vm1'      -sandbox off     -M pc      -nodefaults      -vga cirrus      -chardev socket,id=qmp_id_qmpmonitor1,path=/tmp/monitor-qmpmonitor1-20131119-170324-eboEDTcp,server,nowait     -mon chardev=qmp_id_qmpmonitor1,mode=control      -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20131119-170324-eboEDTcp,server,nowait     -device isa-serial,chardev=serial_id_serial0      -chardev socket,id=seabioslog_id_20131119-170324-eboEDTcp,path=/tmp/seabios-20131119-170324-eboEDTcp,server,nowait     -device isa-debugcon,chardev=seabioslog_id_20131119-170324-eboEDTcp,iobase=0x402     -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=03     -drive id=drive_image1,if=none,cache=none,snapshot=on,aio=native,file=/root/staf-kvm-devel/autotest/client/tests/virt/shared/data/images/RHEL-Server-6.5-64-virtio.qcow2     -device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=0,bus=pci.0,addr=04     -device vfio-pci,host=0000:05:00.0,id=id_0000.05.00.0,bus=pci.0,addr=05     -device vfio-pci,host=0000:05:00.1,id=id_0000.05.00.1,bus=pci.0,addr=06      -m 4096      -smp 8,maxcpus=8,cores=4,threads=1,sockets=2      -cpu 'Nehalem'     -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1      -vnc :0      -rtc base=utc,clock=host,driftfix=slew      -boot order=cdn,once=c,menu=off      -no-kvm-pit-reinjection     -enable-kvm     -monitor stdio

...
ioctl(23, VFIO_GET_API_VERSION, 0x17)   = 0
ioctl(23, VFIO_CHECK_EXTENSION, 0x1)    = 1
ioctl(22, VFIO_GROUP_SET_CONTAINER, 0x7fff988bf2a0) = 0
ioctl(23, VFIO_SET_IOMMU, 0x1)          = 0
ioctl(23, VFIO_IOMMU_MAP_DMA, 0x7fff988bf110) = 0
ioctl(23, VFIO_IOMMU_MAP_DMA, 0x7fff988bf110) = 0
ioctl(23, VFIO_IOMMU_MAP_DMA, 0x7fff988bf110) = 0
ioctl(23, VFIO_IOMMU_MAP_DMA <unfinished ...>
+++ killed by SIGKILL +++
Killed

Comment 1 ShupingCui 2013-11-21 05:50:36 UTC
Created attachment 827004 [details]
strace debug file

Comment 3 Alex Williamson 2013-11-22 19:31:21 UTC
(In reply to ShupingCui from comment #0)
> Steps to Reproduce:
> 1. boot guest with numactl
> numactl \
>     -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
...
>     -m 4096  \
...
> # numactl --hardware
> available: 2 nodes (0-1)
> node 0 cpus: 0 2 4 6 8 10 12 14
> node 0 size: 4096 MB
> node 0 free: 3880 MB
> node 1 cpus: 1 3 5 7 9 11 13 15
> node 1 size: 4083 MB
> node 1 free: 3622 MB
               ^^^^

From numactl man page:

       --membind=nodes, -m nodes
              Only allocate memory from  nodes.   Allocation  will  fail  when
              there  is not enough memory available on these nodes.  nodes may
              be specified as noted above.

So you're trying to start a 4G VM, forcing memory to be allocated from a node that does not have 4G available and the allocation fails as the numactl man page says it will... sounds like notabug.  Re-open if there's something I'm missing.  I've tested using membind within the size available on my host and it works fine.

Comment 4 Xiaoqing Wei 2013-11-27 08:48:57 UTC
(In reply to Alex Williamson from comment #3)
> (In reply to ShupingCui from comment #0)
> > Steps to Reproduce:
> > 1. boot guest with numactl
> > numactl \
> >     -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
> ...
> >     -m 4096  \
> ...
> > # numactl --hardware
> > available: 2 nodes (0-1)
> > node 0 cpus: 0 2 4 6 8 10 12 14
> > node 0 size: 4096 MB
> > node 0 free: 3880 MB
> > node 1 cpus: 1 3 5 7 9 11 13 15
> > node 1 size: 4083 MB
> > node 1 free: 3622 MB
>                ^^^^
> 
> From numactl man page:
> 
>        --membind=nodes, -m nodes
>               Only allocate memory from  nodes.   Allocation  will  fail 
> when
>               there  is not enough memory available on these nodes.  nodes
> may
>               be specified as noted above.
> 
> So you're trying to start a 4G VM, forcing memory to be allocated from a
> node that does not have 4G available and the allocation fails as the numactl
> man page says it will... sounds like notabug.  Re-open if there's something
> I'm missing.  I've tested using membind within the size available on my host
> and it works fine.

scui, pls retry w/ lower ram, and update the result here, thx.

Comment 5 ShupingCui 2013-11-27 08:54:07 UTC
(In reply to Xiaoqing Wei from comment #4)
> (In reply to Alex Williamson from comment #3)
> > (In reply to ShupingCui from comment #0)
> > > Steps to Reproduce:
> > > 1. boot guest with numactl
> > > numactl \
> > >     -m 1 /root/staf-kvm-devel/autotest/client/tests/virt/qemu/qemu \
> > ...
> > >     -m 4096  \
> > ...
> > > # numactl --hardware
> > > available: 2 nodes (0-1)
> > > node 0 cpus: 0 2 4 6 8 10 12 14
> > > node 0 size: 4096 MB
> > > node 0 free: 3880 MB
> > > node 1 cpus: 1 3 5 7 9 11 13 15
> > > node 1 size: 4083 MB
> > > node 1 free: 3622 MB
> >                ^^^^
> > 
> > From numactl man page:
> > 
> >        --membind=nodes, -m nodes
> >               Only allocate memory from  nodes.   Allocation  will  fail 
> > when
> >               there  is not enough memory available on these nodes.  nodes
> > may
> >               be specified as noted above.
> > 
> > So you're trying to start a 4G VM, forcing memory to be allocated from a
> > node that does not have 4G available and the allocation fails as the numactl
> > man page says it will... sounds like notabug.  Re-open if there's something
> > I'm missing.  I've tested using membind within the size available on my host
> > and it works fine.
> 
> scui, pls retry w/ lower ram, and update the result here, thx.

tried with "-m 2048", no killed.

Thanks,
Shuping