Bug 896092

Summary:	QEMU doesn't support comma separated list of NUMA node(cell) cpus and Libvirt doesn't care about it.
Product:	[Community] Virtualization Tools	Reporter:	Vladimir Koshelev <vedun-z>
Component:	libvirt	Assignee:	Eric Blake <eblake>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	eblake, perfbz, pkrempa, rbalakri
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-02-24 08:09:19 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1310122

Description Vladimir Koshelev 2013-01-16 15:24:46 UTC

Description of problem:

QEMU don't support comma separated cpu list for -node numa,cpus= argument. Thus, this part of xml file doesn't work properly with QEMU:

  <cpu mode="host-model" match="exact">
	<numa>
		<cell cpus="0,2,4,6,8,10" memory="10000000"/>
		<cell cpus="1,3,5,7,9,11" memory="10000000"/>
	</numa>
  </cpu>

Libvirt generates command like this:

"-numa node,nodeid=0,cpus=0,2,4,6,8,10,mem=9766 -numa node,nodeid=1,cpus=1,3,5,7,9,11,mem=9766"

But it's totally wrong for QEMU, as long as comma separates parameters of -numa key, not cpus in the cpu list. But even worse is that QEMU doesn't care about its parameters and executes VM with incorrect NUMA topology successfully. 


Version-Release number of selected component (if applicable):
0.9.10 .. Git Head


How reproducible:
Just try to create VM with same configuration.

Comment 1 Eric Blake 2013-01-16 16:14:30 UTC

(In reply to comment #0)
> Description of problem:
> 
> QEMU don't support comma separated cpu list for -node numa,cpus= argument.

True for qemu 1.3, but there is an upstream qemu patch that changes that to allow escaped comma (cpus=0,,2,mem=9766) or quoted semicolon ("cpus=0;2,mem=9766") in the works:
https://lists.gnu.org/archive/html/qemu-devel/2013-01/msg01961.html

> Libvirt generates command like this:
> 
> "-numa node,nodeid=0,cpus=0,2,4,6,8,10,mem=9766 -numa
> node,nodeid=1,cpus=1,3,5,7,9,11,mem=9766"

You are correct that libvirt is generating a command line that isn't supported in older qemu, so libvirt should be fixed to fail where qemu can't support it.  Meanwhile, libvirt should use the proper syntax being added in qemu 1.4.

Comment 2 Eric Blake 2013-02-27 03:07:34 UTC

See also:
https://www.redhat.com/archives/libvir-list/2013-February/msg01414.html

Comment 3 Peter Krempa 2016-02-24 08:09:19 UTC

The configuration creating a disjoint numa range was at first rejected by commit:

commit 25dc8ba08b32c7430d81228718c90d277f902f18
Author: Eric Blake <eblake>
Date:   Tue Feb 26 17:43:12 2013 -0700

    qemu: -numa doesn't (yet) support disjoint range
    
    https://bugzilla.redhat.com/show_bug.cgi?id=896092 mentions that
    qemu 1.4 and earlier only accept a simple start-stop range for
    the cpu=... argument of -numa.  Libvirt would attempt to use
    -numa cpu=1,3 for a disjoint range, which did not work as intended.
    
    Upstream qemu will be adding a new syntax for disjoint cpu ranges
    in 1.5; but the design for that syntax is still under discussion
    at the time of this patch.  So for libvirt 1.0.3, it is safest to
    just reject attempts to build an invalid qemu command line; in the
    future, we can add a capability bit and translate to the final
    accepted design for selecting a disjoint cpu range in numa.

and once qemu added support for them it was correctly enabled with commit:

commit 001b9dc1dcd568159cbe253e3736b873534bd254
Author: Martin Kletzander <mkletzan>
Date:   Tue Jun 17 14:16:59 2014 +0200

    qemu: enable disjoint numa cpu ranges