Bug 1433193
| Summary: | Guest could not boot up when attached numa nodes with ram on ppc64le | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Min Deng <mdeng> |
| Component: | qemu-kvm-rhev | Assignee: | Laurent Vivier <lvivier> |
| Status: | CLOSED ERRATA | QA Contact: | Min Deng <mdeng> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.4 | CC: | eblake, knoel, lvivier, mdeng, mrezanin, qzhang, virt-maint |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | ppc64le | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | qemu-kvm-rhev-2.9.0-1.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-02 03:39:56 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Min Deng
2017-03-17 03:31:58 UTC
(In reply to dengmin from comment #0) > 1.boot up guest with the following parameters > -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 > -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 "size=" is not a valid option of "-numa", use "mem=" instead. Could you retry with "mem=" and provide the full command line: I'm not able to reproduce. Provide also from the qemu HMP the result of "info numa". (In reply to Laurent Vivier from comment #2) > (In reply to dengmin from comment #0) > > > 1.boot up guest with the following parameters > > -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 > > -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 > > "size=" is not a valid option of "-numa", use "mem=" instead. According to current test results I am afraid that I cannot agree with you since it works with "size="on x86 platform but it has issue on ppc64le. X86, builds, kernel-3.10.0-600.el7.x86_64 qemu-kvm-tools-rhev-2.8.0-6.el7.x86_64 [root@intel-5205-32-2]/usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pc -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9 -monitor stdio QEMU 2.8.0 monitor - type 'help' for more information (qemu) info numa 3 nodes node 0 cpus: 0 2 node 0 size: 1704 MB node 1 cpus: 1 3 node 1 size: 1704 MB node 2 cpus: node 2 size: 1712 MB (qemu) ppc64le, 1.[root@ibm-p8-rhevm-17 home]# /usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9 qemu-kvm: Node 0 memory size 0x55000000 is not aligned to 256 MiB 2./usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -chardev socket,id=hmp_id_humanmonitor1,path=/tmp/monitor-humanmonitor1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_humanmonitor1,mode=readline -chardev socket,id=qmp_id_qmp1,path=/tmp/monitor-qmp1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=qmp_id_qmp1,mode=control -chardev socket,id=hmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_catch_monitor,mode=readline -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151207-185515-CKlGrjUv,server,nowait -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=03,disable-legacy=off,disable-modern=off -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=rhel74-ppc64le-virtio-scsi-latest.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -qmp tcp:0:4444,server,nowait -vnc :1 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -netdev tap,script=/etc/qemu-ifup,downscript=/etc/qemu-down,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:52:11:36:3f:00 -m 5G -smp 4,maxcpus=10 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9 QEMU 2.8.0 monitor - type 'help' for more information (qemu) qemu-kvm: Node 0 memory size 0x6a800000 is not aligned to 256 MiB > > Could you retry with "mem=" and provide the full command line: I'm not able > to reproduce. > > Provide also from the qemu HMP the result of "info numa". With the "mem=" it do work on ppc64le 1./usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa node,mem=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,mem=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,mem=1G,cpus=8,cpus=9 -monitor stdio QEMU 2.8.0 monitor - type 'help' for more information (qemu) info numa 3 nodes node 0 cpus: 0 2 node 0 size: 2048 MB node 1 cpus: 1 3 node 1 size: 2048 MB node 2 cpus: node 2 size: 1024 MB (qemu) or 2./usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -chardev socket,id=hmp_id_humanmonitor1,path=/tmp/monitor-humanmonitor1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_humanmonitor1,mode=readline -chardev socket,id=qmp_id_qmp1,path=/tmp/monitor-qmp1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=qmp_id_qmp1,mode=control -chardev socket,id=hmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_catch_monitor,mode=readline -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151207-185515-CKlGrjUv,server,nowait -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=03,disable-legacy=off,disable-modern=off -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=rhel74-ppc64le-virtio-scsi-latest.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -qmp tcp:0:4444,server,nowait -vnc :1 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -netdev tap,script=/etc/qemu-ifup,downscript=/etc/qemu-down,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:52:11:36:3f:00 -m 5G -smp 4,maxcpus=10 -numa node,mem=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,mem=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,mem=1G,cpus=8,cpus=9 QEMU 2.8.0 monitor - type 'help' for more information (qemu) info numa 3 nodes node 0 cpus: 0 2 node 0 size: 2048 MB node 1 cpus: 1 3 node 1 size: 2048 MB node 2 cpus: node 2 size: 1024 MB (qemu) Draw a conclusion,for both "size=" and "mem=" work on x86 but only "mem=" work for ppc64le.There is somewhat difference between x86 and ppc64le.Any issues please let me know,thanks a lot. Best Regrads, Min (In reply to dengmin from comment #3) > (In reply to Laurent Vivier from comment #2) > > (In reply to dengmin from comment #0) > > > > > 1.boot up guest with the following parameters > > > -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 > > > -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 > > > > "size=" is not a valid option of "-numa", use "mem=" instead. > According to current test results I am afraid that I cannot agree with > you since it works with "size="on x86 platform but it has issue on ppc64le. > X86, > builds, > kernel-3.10.0-600.el7.x86_64 > qemu-kvm-tools-rhev-2.8.0-6.el7.x86_64 > [root@intel-5205-32-2]/usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox > off -machine pc -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa > node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa > node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9 > -monitor stdio > QEMU 2.8.0 monitor - type 'help' for more information > (qemu) info numa > 3 nodes > node 0 cpus: 0 2 > node 0 size: 1704 MB > node 1 cpus: 1 3 > node 1 size: 1704 MB > node 2 cpus: > node 2 size: 1712 MB > (qemu) ... > Draw a conclusion,for both "size=" and "mem=" work on x86 but only "mem=" > work for ppc64le.There is somewhat difference between x86 and ppc64le.Any > issues please let me know,thanks a lot. "size" is ignored on x86_64 and ppc64le, and then it uses a default value: 1704MB is not 2GB and 1712MB is not 1GB. qemu-kvm(1) -numa node[,mem=size][,cpus=cpu[-cpu]][,nodeid=node] -numa node[,memdev=id][,cpus=cpu[-cpu]][,nodeid=node] Simulate a multi node NUMA system. If mem, memdev and cpus are omitted, resources are split equally. Also, note that the -numa option doesn't allocate any of the specified resources. That is, it just assigns existing resources to NUMA nodes. This means that one still has to use the -m, -smp options to allocate RAM and VCPUs respectively, and possibly -object to specify the memory backend for the memdev suboption. mem and memdev are mutually exclusive. Furthermore, if one node uses memdev, all of them have to use it. I think there are 2 bugs here:
- "-numa" silently accepts an invalid parameter ("size"),
- the default value for "mem" can be invalid.
I'm able to reproduce the problem with:
$ qemu-kvm -m 4G -numa node -numa node -numa node
qemu-kvm: Node 0 memory size 0x55000000 is not aligned to 256 MiB
The default node memory size is:
"Equally divide total memory among nodes if both "mem" and "memdev" are omitted."
In the previous example, memory is divided as:
-node 0: 1368MB
-node 1: 1368MB
-node 2: 1360MB (0x55000000)
None of them are aligned to 256MB.
We should have:
-node 0: 1280MB
-node 2: 1280MB
-node 3: 1536MB
(In reply to Laurent Vivier from comment #5) > I think there are 2 bugs here: > - "-numa" silently accepts an invalid parameter ("size"), This part is my fault, and I've posted a 2-patch series upstream to beef up the testsuite and fix the one-liner off-by-one bug that broke invalid parameter detection in the QemuOpts visitor. (In reply to Laurent Vivier from comment #5) > - the default value for "mem" can be invalid. I've sent a patch upstream: http://patchwork.ozlabs.org/patch/741488/ (In reply to Eric Blake from comment #6) > (In reply to Laurent Vivier from comment #5) > > I think there are 2 bugs here: > > - "-numa" silently accepts an invalid parameter ("size"), > > This part is my fault, and I've posted a 2-patch series upstream to beef up > the testsuite and fix the one-liner off-by-one bug that broke invalid > parameter detection in the QemuOpts visitor QE will file a bug for trace the issue. (In reply to dengmin from comment #8) > (In reply to Eric Blake from comment #6) > > (In reply to Laurent Vivier from comment #5) > > > I think there are 2 bugs here: > > > - "-numa" silently accepts an invalid parameter ("size"), > > > > This part is my fault, and I've posted a 2-patch series upstream to beef up > > the testsuite and fix the one-liner off-by-one bug that broke invalid > > parameter detection in the QemuOpts visitor > > QE will file a bug for trace the issue. Bug 1434666 "-numa" should not silently accept an invalid parameter ("size") The fix for pseries part is now upstream: 5564121 numa,spapr: align default numa node memory size to 256MB Verified the bug on the following builds kernel-3.10.0-655.el7.ppc64le (guest and host) qemu-kvm-rhev-2.9.0-1.el7.ppc64le SLOF-20170303-1.git66d250e.el7.noarch Detail steps and cli please refer to comment0 Actual results, For "size" shouldn't be used any more here.(Fixed bug 1434666) qemu-kvm: -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6: Invalid parameter 'size' For "mem" the guest worked correctly Expected results, 1."size" can not be used any more 2."mem" can work well As a result,the original issue has been fixed already,so move it to verified status,thanks a lot. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:2392 |