Bug 1096196

Summary: QEMU should abort if NUMA node configuration don't cover all RAM
Product: Red Hat Enterprise Linux 7 Reporter: Shaolong Hu <shu>
Component: qemu-kvm-rhevAssignee: Eduardo Habkost <ehabkost>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0CC: hhuang, juzhang, linchen, michen, mrezanin, rbalakri, virt-maint
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: QEMU 2.1.0 Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 09:46:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Shaolong Hu 2014-05-09 12:11:47 UTC
Description of problem:
----------------------------
NUMA emulation failed with extra memory not specified in any node.


Version-Release number of selected component (if applicable):
----------------------------------------------------------------
qemu-kvm-1.5.3-60.el7_0.1.x86_64


How reproducible:
------------------
100%


Steps to Reproduce:
-----------------------
boot guest with memory lager than specified in all nodes, like:

-m 5G -numa node,mem=1024,cpus=0,nodeid=0 -numa node,mem=1024,cpus=1,nodeid=1 -numa node,mem=1024,cpus=2,nodeid=2 -numa node,mem=1024,cpus=3,nodeid=3


Actual results:
------------------
in guest:

[root@localhost ~]# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3
node 0 size: 5119 MB
node 0 free: 4525 MB
node distances:
node   0 
  0:  10 


(qemu) info numa
4 nodes
node 0 cpus: 0
node 0 size: 1024 MB
node 1 cpus: 1
node 1 size: 1024 MB
node 2 cpus: 2
node 2 size: 1024 MB
node 3 cpus: 3
node 3 size: 1024 MB




With "-m 4G", everything is fine.

I recall long time ago(we do not have a case cover the scenario) i tested this kind of scenario, if there is extra memory not specified in any node, it resides in node 0 by default, and other node is as specified, i am not sure whether it should work like this now, if this is as designed, feel free to close it.

Comment 2 Eduardo Habkost 2014-05-09 13:52:56 UTC
Not a bug, QEMU is doing exactly what was requested by the user. But the guest chooses to ignore the NUMA topology because it doesn't make sense.

There are plans to abort QEMU in case the user provides such invalid configuration, though, so I am keeping the BZ open to track the new feature.

Comment 3 Eduardo Habkost 2014-07-10 20:03:05 UTC
Moving to qemu-kvm-rhev. Patches were included upstream and will be on QEMU 2.1.0.

Comment 5 Lin Chen 2014-10-30 09:47:55 UTC
reproduce this bug with unfix version: 
 qemu-kvm-rhev-1.5.3-60.el7_0.10.x86_64
use command :
/usr/libexec/qemu-kvm -m 5G -numa node,mem=1024,cpus=0,nodeid=0 -numa node,mem=1024,cpus=1,nodeid=1   -numa node,mem=1024,cpus=2,nodeid=2 -numa node,mem=1024,cpus=3,nodeid=3

result:
guest do not abort.
(qemu) info numa
4 nodes
node 0 cpus: 0
node 0 size: 1024 MB
node 1 cpus:
node 1 size: 1024 MB
node 2 cpus:
node 2 size: 1024 MB
node 3 cpus:
node 3 size: 1024 MB

[root@localhost ~]# numactl --hardware
available: 4 nodes (0-3)
node 0 cpus: 0 2 4 6 8 10 12 14
node 0 size: 8157 MB
node 0 free: 2572 MB
node 1 cpus: 16 18 20 22 24 26 28 30
node 1 size: 8192 MB
node 1 free: 2036 MB
node 2 cpus: 1 3 5 7 9 11 13 15
node 2 size: 8192 MB
node 2 free: 2086 MB
node 3 cpus: 17 19 21 23 25 27 29 31
node 3 size: 8175 MB
node 3 free: 3173 MB
node distances:
node   0   1   2   3 
  0:  10  16  16  16 
  1:  16  10  16  16 
  2:  16  16  10  16 
  3:  16  16  16  10 

so, this bug is reproduced.


verify with version : qemu-kvm-rhev-2.1.2-5.el7.x86_64 


steps:
1.execute command:
/usr/libexec/qemu-kvm -m 5G -numa node,mem=1024,cpus=0,nodeid=0 -numa node,mem=1024,cpus=1,nodeid=1   -numa node,mem=1024,cpus=2,nodeid=2 -numa node,mem=1024,cpus=3,nodeid=3

result:
qemu abort and outputs the following info:
qemu-kvm: total memory for NUMA nodes (0x100000000) should equal RAM size (0x140000000)


according to above test result. this bug is fixed.

Comment 8 errata-xmlrpc 2015-03-05 09:46:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0624.html