Bug 1263064 - [PowerKVM] Guest cannot boot up successful with "Can't support memory configuration" when boot up guest with multi -numa node
[PowerKVM] Guest cannot boot up successful with "Can't support memory configu...
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev (Show other bugs)
7.2
ppc64le Unspecified
unspecified Severity unspecified
: rc
: ---
Assigned To: David Gibson
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-15 01:10 EDT by Shuang Yu
Modified: 2016-01-14 22:00 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-15 01:50:19 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Shuang Yu 2015-09-15 01:10:41 EDT
Description of problem:
Boot up the guest with -nuam node and set up the node number to 8,the guest can boot up successfully,set up the node number >8,the guest cannot boot up successfully with "qemu-kvm: Can't support memory configuration where memory size e3800000(this value will change when the node number is different) of node 0 isn't aligned to 256 MB "

Version-Release number of selected component (if applicable):
kernel-3.10.0-313.el7.ppc64le
qemu-kvm-rhev-2.3.0-22.el7.ppc64le
SLOF-20150313-3.gitc89b0df.el7.noarch
Guest version:
RHEL-6.7-20150304.0-Server-ppc64-dvd1.iso

How reproducible:
100%

Steps to Reproduce:

1.
Check the host mem:

# free -m
              total        used        free      shared  buff/cache   available
Mem:          31636         552       23438          27        7645       30233
Swap:         16255           0       16255

Check the host numa structure:

# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 8 16 24 32
node 0 size: 32768 MB
node 0 free: 23440 MB
node 1 cpus: 40 48 56 64 72
node 1 size: 0 MB
node 1 free: 0 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 

Check the host cpu:
# lscpu
Architecture:          ppc64le
Byte Order:            Little Endian
CPU(s):                80
On-line CPU(s) list:   0,8,16,24,32,40,48,56,64,72
Off-line CPU(s) list:  1-7,9-15,17-23,25-31,33-39,41-47,49-55,57-63,65-71,73-79
Thread(s) per core:    1
Core(s) per socket:    5
Socket(s):             2
NUMA node(s):          2
Model:                 8247-21L
L1d cache:             64K
L1i cache:             32K
L2 cache:              512K
L3 cache:              8192K
NUMA node0 CPU(s):     0,8,16,24,32
NUMA node1 CPU(s):     40,48,56,64,72



2.Boot up the guest with "-m 32G -smp 64"

# /usr/libexec/qemu-kvm -name numa-test -machine pseries,accel=kvm,usb=off -realtime mlock=off -nodefaults -monitor stdio -rtc base=utc -device virtio-scsi-pci,id=scsi -drive file=RHEL-6.7-20150304.0-Server-ppc64-2.qcow2,format=qcow2,if=none,id=drive-scsi0,cache=none -device scsi-hd,drive=drive-scsi0,id=disk0,bus=scsi.0 -vnc :10 -msg timestamp=on -usb -device usb-tablet,id=tablet1 -vga std -netdev tap,id=hostnet1,script=/etc/qemu-ifup,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:54:5a:5f:5b:5c -drive file=RHEL-6.7-20150304.0-Server-ppc64-dvd1.iso,if=none,media=cdrom,format=raw,rerror=stop,werror=stop,id=scsi-cdrom0 -device virtio-scsi-pci,id=bus2,addr=0x5 -device scsi-cd,bus=bus2.0,drive=scsi-cdrom0,id=cdrom0 -uuid f17c415d-e0b1-4771-b5da-87fb46bc73fd -m 32G -smp 64


3.Boot up the guest with "-m 32G -smp 64" and 8 "-numa node"

# /usr/libexec/qemu-kvm -name numa-test -machine pseries,accel=kvm,usb=off -realtime mlock=off -nodefaults -monitor stdio -rtc base=utc -device virtio-scsi-pci,id=scsi -drive file=RHEL-6.7-20150304.0-Server-ppc64-2.qcow2,format=qcow2,if=none,id=drive-scsi0,cache=none -device scsi-hd,drive=drive-scsi0,id=disk0,bus=scsi.0 -vnc :10 -msg timestamp=on -usb -device usb-tablet,id=tablet1 -vga std -netdev tap,id=hostnet1,script=/etc/qemu-ifup,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:54:5a:5f:5b:5c -drive file=RHEL-6.7-20150304.0-Server-ppc64-dvd1.iso,if=none,media=cdrom,format=raw,rerror=stop,werror=stop,id=scsi-cdrom0 -device virtio-scsi-pci,id=bus2,addr=0x5 -device scsi-cd,bus=bus2.0,drive=scsi-cdrom0,id=cdrom0 -uuid f17c415d-e0b1-4771-b5da-87fb46bc73fd -m 32G -smp 64 -numa node -numa node -numa node -numa node -numa node -numa node -numa node -numa node

4.Boot up the guest with "-m 32G -smp 64" and 9 "-numa node"

/usr/libexec/qemu-kvm -name numa-test -machine pseries,accel=kvm,usb=off -realtime mlock=off -nodefaults -monitor stdio -rtc base=utc -device virtio-scsi-pci,id=scsi -drive file=RHEL-6.7-20150304.0-Server-ppc64-2.qcow2,format=qcow2,if=none,id=drive-scsi0,cache=none -device scsi-hd,drive=drive-scsi0,id=disk0,bus=scsi.0 -vnc :10 -msg timestamp=on -usb -device usb-tablet,id=tablet1 -vga std -netdev tap,id=hostnet1,script=/etc/qemu-ifup,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:54:5a:5f:5b:5c -drive file=RHEL-6.7-20150304.0-Server-ppc64-dvd1.iso,if=none,media=cdrom,format=raw,rerror=stop,werror=stop,id=scsi-cdrom0 -device virtio-scsi-pci,id=bus2,addr=0x5 -device scsi-cd,bus=bus2.0,drive=scsi-cdrom0,id=cdrom0 -uuid f17c415d-e0b1-4771-b5da-87fb46bc73fd -m 32G -smp 64 -numa node -numa node -numa node -numa node -numa node -numa node -numa node -numa node -numa node

5.Boot up the guest  with "-m 32G -smp 64" and 10 "-numa node"

# /usr/libexec/qemu-kvm -name numa-test -machine pseries,accel=kvm,usb=off -realtime mlock=off -nodefaults -monitor stdio -rtc base=utc -device virtio-scsi-pci,id=scsi -drive file=RHEL-6.7-20150304.0-Server-ppc64-2.qcow2,format=qcow2,if=none,id=drive-scsi0,cache=none -device scsi-hd,drive=drive-scsi0,id=disk0,bus=scsi.0 -vnc :10 -msg timestamp=on -usb -device usb-tablet,id=tablet1 -vga std -netdev tap,id=hostnet1,script=/etc/qemu-ifup,vhost=on -device virtio-net-pci,netdev=hostnet1,id=net1,mac=00:54:5a:5f:5b:5c -drive file=RHEL-6.7-20150304.0-Server-ppc64-dvd1.iso,if=none,media=cdrom,format=raw,rerror=stop,werror=stop,id=scsi-cdrom0 -device virtio-scsi-pci,id=bus2,addr=0x5 -device scsi-cd,bus=bus2.0,drive=scsi-cdrom0,id=cdrom0 -uuid f17c415d-e0b1-4771-b5da-87fb46bc73fd -m 32G -smp 64 -numa node -numa node -numa node -numa node -numa node -numa node -numa node -numa node -numa node -numa node

Actual results:

After step 2,the guest can boot up successful.

After step 3,the guest can boot up successful.

After step 4,the guest cannot boot up with 
"(qemu) 2015-09-15T02:32:12.156172Z qemu-kvm: Can't support memory configuration where memory size e3800000 of node 0 isn't aligned to 256 MB"

After step 5,the guest cannot boot up with 
"(qemu) 2015-09-15T02:54:06.902887Z qemu-kvm: Can't support memory configuration where memory size cc800000 of node 0 isn't aligned to 256 MB"

Expected results:
All the guest should boot up successful.

Additional info:
Comment 2 David Gibson 2015-09-15 01:50:19 EDT
As the error message implies, the amount of memory in every NUMA node must be a multiple of 256MB.

If you don't explicitly assign memory to each node, that means total memory must be a multiple of (256MB * #-of-nodes).

Note You need to log in before you can comment on or make changes to this bug.