Hide Forgot
Description of problem: Numa node it null when boot up 3 numa together Version-Release number of selected component (if applicable): qemu-kvm-tools-rhev-2.6.0-27.el7.ppc64le kernel-3.10.0-514.el7.ppc64le How reproducible: 2/2 Steps to Reproduce: 1.boot up guest with cli and attach three numas in total /usr/libexec/qemu-kvm -name avocado-vt-vm1 -sandbox off -machine pseries -nodefaults -vga std -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=04 -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/root/test_home/mdeng/staf-kvm-devel/workspace/usr/share/avocado/data/avocado-vt/images/RHEL-Server-7.3-ppc64le-virtio-scsi.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -m 1536 -smp 6,maxcpus=6,cores=3,threads=1,sockets=2 -vnc :2 -rtc base=utc,clock=host -boot order=cdn,once=c,menu=off,strict=off -numa node -numa node -numa node -enable-kvm -monitor stdio Actual results: (qemu) info numa 3 nodes node 0 cpus: 0 1 2 node 0 size: 512 MB node 1 cpus: 3 4 5 node 1 size: 512 MB node 2 cpus: --- null node 2 size: 512 MB It seemed that the cpus only were assigned to two nodes.Does it work as designed ? Expected results: Compared to results from x86 it is more reasonable in some degrees. (qemu) info numa 3 nodes node 0 cpus: 0 3 node 0 size: 512 MB node 1 cpus: 1 4 node 1 size: 512 MB node 2 cpus: 2 5 node 2 size: 512 MB Additional info: It will effect on automation scripts so please give a hand to look over it,thanks a lot.
This seems normal as you ask for 3 nodes with 3 cores per socket for a total of 6 cores, so you have 3 cores on node 0, 3 cores on node 1, and as there is no more core available, 0 core on the last one. If you want the same result as you have with x86, use: -numa node -numa node -numa node \ -smp 6,maxcpus=6,cores=2,threads=1,sockets=3
Right. In short, qemu tries to place whole sockets into NUMA nodes, because splitting a socket across nodes would be implausible on real hardware.
(In reply to Laurent Vivier from comment #2) > This seems normal as you ask for 3 nodes with 3 cores per socket for a total > of 6 cores, so you have 3 cores on node 0, 3 cores on node 1, and as there > is no more core available, 0 core on the last one. > > If you want the same result as you have with x86, use: > > -numa node -numa node -numa node \ > -smp 6,maxcpus=6,cores=2,threads=1,sockets=3 Hi,it seems that the results is not same to what you said On x86 ,the output still be (qemu) info numa 3 nodes node 0 cpus: 0 3 node 0 size: 512 MB node 1 cpus: 1 4 node 1 size: 512 MB node 2 cpus: 2 5 node 2 size: 512 MB
(In reply to dengmin from comment #4) > (In reply to Laurent Vivier from comment #2) > > This seems normal as you ask for 3 nodes with 3 cores per socket for a total > > of 6 cores, so you have 3 cores on node 0, 3 cores on node 1, and as there > > is no more core available, 0 core on the last one. > > > > If you want the same result as you have with x86, use: > > > > -numa node -numa node -numa node \ > > -smp 6,maxcpus=6,cores=2,threads=1,sockets=3 > > Hi,it seems that the results is not same to what you said > > On x86 ,the output still be > > (qemu) info numa > 3 nodes > node 0 cpus: 0 3 > node 0 size: 512 MB > node 1 cpus: 1 4 > node 1 size: 512 MB > node 2 cpus: 2 5 > node 2 size: 512 MB Please ignore my comments,thanks
Draw a conclusion for this bug finally. On x86, if use the following commands, a....-numa node -numa node -numa node -smp 6,maxcpus=6,*cores=2*,threads=1,*sockets=3* ... b....-numa node -numa node -numa node -smp 6,maxcpus=6,*cores=3*,threads=1,*sockets=2* ... The outputs are the same (qemu) info numa 3 nodes node 0 cpus: 0 3 node 0 size: 512 MB node 1 cpus: 1 4 node 1 size: 512 MB node 2 cpus: 2 5 node 2 size: 512 MB On ppc a....-numa node -numa node -numa node -smp 6,maxcpus=6,*cores=2*,threads=1,*sockets=3* ... b....-numa node -numa node -numa node -smp 6,maxcpus=6,*cores=3*,threads=1,*sockets=2* ... The output are different with each other The output of command a, (qemu) info numa 3 nodes node 0 cpus: 0 1 node 0 size: 512 MB node 1 cpus: 2 3 node 1 size: 512 MB node 2 cpus: 4 5 node 2 size: 512 MB The output of command b, (qemu) info numa 3 nodes node 0 cpus: 0 1 2 node 0 size: 512 MB node 1 cpus: 3 4 5 node 1 size: 512 MB node 2 cpus: --- null node 2 size: 512 MB
looks like a bug on x86.
Debateable whether this is really a bug on x86. Basically x86 allows cores to be split between NUMA nodes, whereas ppc does not. That's... weird, but not necessarily a problem.
Hi dengmin, unfortunately I didn't reproduce the same issue you. could you provide the qemu version you used under x86 platform/RHEL7.3. I got the result as below: [root@hp-z800-06 liuzt]# sh test.sh QEMU 2.6.0 monitor - type 'help' for more information (qemu) VNC server running on '::1;5900' (qemu) (qemu) info numa 3 nodes node 0 cpus: 0 1 2 node 0 size: 40 MB node 1 cpus: 3 4 5 node 1 size: 40 MB node 2 cpus: node 2 size: 48 MB [root@hp-z800-06 liuzt]# cat test.sh /usr/libexec/qemu-kvm \ -smp 6,maxcpus=6,cores=3,threads=1,sockets=2 \ -numa node \ -numa node \ -numa node \ -monitor stdio \ -enable-kvm [root@hp-z800-06 liuzt]# /usr/libexec/qemu-kvm --version QEMU emulator version 2.6.0 (qemu-kvm-rhev-2.6.0-28.el7_3.2), Copyright (c) 2003-2008 Fabrice Bellard
The issue only occurs on the build any issues please let me know,thanks a lot! /usr/libexec/qemu-kvm --version QEMU emulator version 1.5.3 (qemu-kvm-1.5.3-126.el7), Copyright (c) 2003-2008 Fabrice Bellard
(In reply to Zhengtong from comment #9) > [root@hp-z800-06 liuzt]# /usr/libexec/qemu-kvm --version > QEMU emulator version 2.6.0 (qemu-kvm-rhev-2.6.0-28.el7_3.2), Copyright (c) > 2003-2008 Fabrice Bellard (In reply to dengmin from comment #10) > /usr/libexec/qemu-kvm --version > QEMU emulator version 1.5.3 (qemu-kvm-1.5.3-126.el7), Copyright (c) > 2003-2008 Fabrice Bellard You are testing different QEMU versions. The behavior on qemu-kvm-1.5.3 is the old one (assigning threads from the same socket to different nodes). qemu-kvm-rhev-2.6.0 has the new behavior that is similar to power. qemu-kvm-rhev has no problem at all, and the default on qemu-kvm-1.5.3 won't be changed (as it is not a bug).