Bug 1433193 - Guest could not boot up when attached numa nodes with ram on ppc64le
Summary: Guest could not boot up when attached numa nodes with ram on ppc64le
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.4
Hardware: ppc64le
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Laurent Vivier
QA Contact: Min Deng
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-03-17 03:31 UTC by Min Deng
Modified: 2017-08-02 03:39 UTC (History)
7 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2017-08-02 03:39:56 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2017:2392 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2017-08-01 20:04:36 UTC

Description Min Deng 2017-03-17 03:31:58 UTC
Description of problem:
Guest could not boot up when attached 2 numa nodes with 2G ram on ppc64le

Version-Release number of selected component (if applicable):
kernel-3.10.0-600.el7.ppc64le
qemu-kvm-rhev-2.8.0-6.el7.ppc64le
SLOF-20160223-6.gitdbbfda4.el7.noarch

How reproducible:
3/3

Steps to Reproduce:
1.boot up guest with the following parameters
  -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7
2.
3.

Actual results:
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) qemu-kvm: Node 0 memory size 0x55000000 is not aligned to 256 MiB


Expected results:
The 2G is a multiple of 256MB

Additional info:

Comment 2 Laurent Vivier 2017-03-17 11:25:22 UTC
(In reply to dengmin from comment #0)

> 1.boot up guest with the following parameters
>   -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6
> -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7

"size=" is not a valid option of "-numa", use "mem=" instead.

Could you retry with "mem=" and provide the full command line: I'm not able to reproduce.

Provide also from the qemu HMP the result of "info numa".

Comment 3 Min Deng 2017-03-20 05:29:38 UTC
(In reply to Laurent Vivier from comment #2)
> (In reply to dengmin from comment #0)
> 
> > 1.boot up guest with the following parameters
> >   -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6
> > -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7
> 
> "size=" is not a valid option of "-numa", use "mem=" instead.
   According to current test results I am afraid that I cannot agree with you since it works with "size="on x86 platform but it has issue on ppc64le.
X86,
   builds,
   kernel-3.10.0-600.el7.x86_64
   qemu-kvm-tools-rhev-2.8.0-6.el7.x86_64
   [root@intel-5205-32-2]/usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off  -machine pc -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9 -monitor stdio
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) info numa
3 nodes
node 0 cpus: 0 2
node 0 size: 1704 MB
node 1 cpus: 1 3
node 1 size: 1704 MB
node 2 cpus:
node 2 size: 1712 MB
(qemu) 

ppc64le,
1.[root@ibm-p8-rhevm-17 home]# /usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9
qemu-kvm: Node 0 memory size 0x55000000 is not aligned to 256 MiB

2./usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -chardev socket,id=hmp_id_humanmonitor1,path=/tmp/monitor-humanmonitor1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_humanmonitor1,mode=readline -chardev socket,id=qmp_id_qmp1,path=/tmp/monitor-qmp1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=qmp_id_qmp1,mode=control -chardev socket,id=hmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_catch_monitor,mode=readline -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151207-185515-CKlGrjUv,server,nowait -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=03,disable-legacy=off,disable-modern=off -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=rhel74-ppc64le-virtio-scsi-latest.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -qmp tcp:0:4444,server,nowait -vnc :1 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -netdev tap,script=/etc/qemu-ifup,downscript=/etc/qemu-down,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:52:11:36:3f:00 -m 5G -smp 4,maxcpus=10 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) qemu-kvm: Node 0 memory size 0x6a800000 is not aligned to 256 MiB

> 
> Could you retry with "mem=" and provide the full command line: I'm not able
> to reproduce.
> 
> Provide also from the qemu HMP the result of "info numa".

With the "mem=" it do work on ppc64le

1./usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa node,mem=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,mem=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,mem=1G,cpus=8,cpus=9 -monitor stdio
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) info numa
3 nodes
node 0 cpus: 0 2
node 0 size: 2048 MB
node 1 cpus: 1 3
node 1 size: 2048 MB
node 2 cpus:
node 2 size: 1024 MB
(qemu) 

or 
2./usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox off -machine pseries-rhel7.4.0 -nodefaults -vga std -chardev socket,id=hmp_id_humanmonitor1,path=/tmp/monitor-humanmonitor1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_humanmonitor1,mode=readline -chardev socket,id=qmp_id_qmp1,path=/tmp/monitor-qmp1-20151207-185515-CKlGrjUv,server,nowait -mon chardev=qmp_id_qmp1,mode=control -chardev socket,id=hmp_id_catch_monitor,path=/tmp/monitor-catch_monitor-20151207-185515-CKlGrjUv,server,nowait -mon chardev=hmp_id_catch_monitor,mode=readline -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20151207-185515-CKlGrjUv,server,nowait -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=03,disable-legacy=off,disable-modern=off -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,format=qcow2,file=rhel74-ppc64le-virtio-scsi-latest.qcow2 -device scsi-hd,id=image1,drive=drive_image1 -qmp tcp:0:4444,server,nowait -vnc :1 -rtc base=utc,clock=host,driftfix=slew -boot order=cdn,once=c,menu=off,strict=off -enable-kvm -monitor stdio -netdev tap,script=/etc/qemu-ifup,downscript=/etc/qemu-down,id=hostnet1,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:52:11:36:3f:00 -m 5G -smp 4,maxcpus=10 -numa node,mem=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa node,mem=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,mem=1G,cpus=8,cpus=9
QEMU 2.8.0 monitor - type 'help' for more information
(qemu) info numa
3 nodes
node 0 cpus: 0 2
node 0 size: 2048 MB
node 1 cpus: 1 3
node 1 size: 2048 MB
node 2 cpus:
node 2 size: 1024 MB
(qemu) 


 Draw a conclusion,for both "size=" and "mem=" work on x86 but only "mem=" work for ppc64le.There is somewhat difference between x86 and ppc64le.Any issues please let me know,thanks a lot.

Best Regrads,
Min

Comment 4 Laurent Vivier 2017-03-20 07:53:25 UTC
(In reply to dengmin from comment #3)
> (In reply to Laurent Vivier from comment #2)
> > (In reply to dengmin from comment #0)
> > 
> > > 1.boot up guest with the following parameters
> > >   -m 4G -smp 4,maxcpus=8 -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6
> > > -numa node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7
> > 
> > "size=" is not a valid option of "-numa", use "mem=" instead.
>    According to current test results I am afraid that I cannot agree with
> you since it works with "size="on x86 platform but it has issue on ppc64le.
> X86,
>    builds,
>    kernel-3.10.0-600.el7.x86_64
>    qemu-kvm-tools-rhev-2.8.0-6.el7.x86_64
>    [root@intel-5205-32-2]/usr/libexec/qemu-kvm -name virt-tests-vm1 -sandbox
> off  -machine pc -nodefaults -vga std -m 5G -smp 4,maxcpus=10 -numa
> node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6 -numa
> node,size=2G,cpus=1,cpus=3,cpus=5,cpus=7 -numa node,size=1G,cpus=8,cpus=9
> -monitor stdio
> QEMU 2.8.0 monitor - type 'help' for more information
> (qemu) info numa
> 3 nodes
> node 0 cpus: 0 2
> node 0 size: 1704 MB
> node 1 cpus: 1 3
> node 1 size: 1704 MB
> node 2 cpus:
> node 2 size: 1712 MB
> (qemu) 
...
>  Draw a conclusion,for both "size=" and "mem=" work on x86 but only "mem="
> work for ppc64le.There is somewhat difference between x86 and ppc64le.Any
> issues please let me know,thanks a lot.

"size" is ignored on x86_64 and ppc64le, and then it uses a default value: 1704MB is not 2GB and 1712MB is not 1GB.

qemu-kvm(1)

       -numa node[,mem=size][,cpus=cpu[-cpu]][,nodeid=node]
       -numa node[,memdev=id][,cpus=cpu[-cpu]][,nodeid=node]
           Simulate a multi node NUMA system. If mem, memdev and cpus are
           omitted, resources are split equally. Also, note that the -numa
           option doesn't allocate any of the specified resources. That is, it
           just assigns existing resources to NUMA nodes. This means that one
           still has to use the -m, -smp options to allocate RAM and VCPUs
           respectively, and possibly -object to specify the memory backend
           for the memdev suboption.

           mem and memdev are mutually exclusive.  Furthermore, if one node
           uses memdev, all of them have to use it.

Comment 5 Laurent Vivier 2017-03-20 09:45:57 UTC
I think there are 2 bugs here:
- "-numa" silently accepts an invalid parameter ("size"),
- the default value for "mem" can be invalid.

I'm able to reproduce the problem with:

$ qemu-kvm -m 4G -numa node -numa node -numa node
qemu-kvm: Node 0 memory size 0x55000000 is not aligned to 256 MiB

The default node memory size is:

"Equally divide total memory among nodes if both "mem" and "memdev" are omitted."

In the previous example, memory is divided as:

-node 0: 1368MB
-node 1: 1368MB
-node 2: 1360MB (0x55000000)

None of them are aligned to 256MB.

We should have:
-node 0: 1280MB
-node 2: 1280MB
-node 3: 1536MB

Comment 6 Eric Blake 2017-03-21 03:22:24 UTC
(In reply to Laurent Vivier from comment #5)
> I think there are 2 bugs here:
> - "-numa" silently accepts an invalid parameter ("size"),

This part is my fault, and I've posted a 2-patch series upstream to beef up the testsuite and fix the one-liner off-by-one bug that broke invalid parameter detection in the QemuOpts visitor.

Comment 7 Laurent Vivier 2017-03-21 12:17:54 UTC
(In reply to Laurent Vivier from comment #5)
> - the default value for "mem" can be invalid.

I've sent a patch upstream:

http://patchwork.ozlabs.org/patch/741488/

Comment 8 Min Deng 2017-03-22 04:11:34 UTC
(In reply to Eric Blake from comment #6)
> (In reply to Laurent Vivier from comment #5)
> > I think there are 2 bugs here:
> > - "-numa" silently accepts an invalid parameter ("size"),
> 
> This part is my fault, and I've posted a 2-patch series upstream to beef up
> the testsuite and fix the one-liner off-by-one bug that broke invalid
> parameter detection in the QemuOpts visitor

  QE will file a bug for trace the issue.

Comment 9 Min Deng 2017-03-22 04:22:20 UTC
(In reply to dengmin from comment #8)
> (In reply to Eric Blake from comment #6)
> > (In reply to Laurent Vivier from comment #5)
> > > I think there are 2 bugs here:
> > > - "-numa" silently accepts an invalid parameter ("size"),
> > 
> > This part is my fault, and I've posted a 2-patch series upstream to beef up
> > the testsuite and fix the one-liner off-by-one bug that broke invalid
> > parameter detection in the QemuOpts visitor
> 
>   QE will file a bug for trace the issue.
    Bug 1434666 "-numa" should not silently accept an invalid parameter ("size")

Comment 10 Laurent Vivier 2017-03-23 17:04:47 UTC
The fix for pseries part is now upstream:

5564121 numa,spapr: align default numa node memory size to 256MB

Comment 11 Min Deng 2017-04-25 07:50:12 UTC
Verified the bug on the following builds
kernel-3.10.0-655.el7.ppc64le (guest and host)
qemu-kvm-rhev-2.9.0-1.el7.ppc64le
SLOF-20170303-1.git66d250e.el7.noarch

Detail steps and cli
please refer to comment0

Actual results,
For "size" shouldn't be used any more here.(Fixed bug 1434666)
qemu-kvm: -numa node,size=2G,cpus=0,cpus=2,cpus=4,cpus=6: Invalid parameter 'size'
For "mem" the guest worked correctly
Expected results,
1."size" can not be used any more
2."mem" can work well
As a result,the original issue has been fixed already,so move it to verified status,thanks a lot.

Comment 13 errata-xmlrpc 2017-08-02 03:39:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:2392


Note You need to log in before you can comment on or make changes to this bug.