Bug 896092 - QEMU doesn't support comma separated list of NUMA node(cell) cpus and Libvirt doesn't care about it.
Summary: QEMU doesn't support comma separated list of NUMA node(cell) cpus and Libvirt...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Virtualization Tools
Classification: Community
Component: libvirt
Version: unspecified
Hardware: x86_64
OS: All
unspecified
unspecified
Target Milestone: ---
Assignee: Eric Blake
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks: 1310122
TreeView+ depends on / blocked
 
Reported: 2013-01-16 15:24 UTC by Vladimir Koshelev
Modified: 2016-02-24 08:34 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-02-24 08:09:19 UTC
Embargoed:


Attachments (Terms of Use)

Description Vladimir Koshelev 2013-01-16 15:24:46 UTC
Description of problem:

QEMU don't support comma separated cpu list for -node numa,cpus= argument. Thus, this part of xml file doesn't work properly with QEMU:

  <cpu mode="host-model" match="exact">
	<numa>
		<cell cpus="0,2,4,6,8,10" memory="10000000"/>
		<cell cpus="1,3,5,7,9,11" memory="10000000"/>
	</numa>
  </cpu>

Libvirt generates command like this:

"-numa node,nodeid=0,cpus=0,2,4,6,8,10,mem=9766 -numa node,nodeid=1,cpus=1,3,5,7,9,11,mem=9766"

But it's totally wrong for QEMU, as long as comma separates parameters of -numa key, not cpus in the cpu list. But even worse is that QEMU doesn't care about its parameters and executes VM with incorrect NUMA topology successfully. 


Version-Release number of selected component (if applicable):
0.9.10 .. Git Head


How reproducible:
Just try to create VM with same configuration.

Comment 1 Eric Blake 2013-01-16 16:14:30 UTC
(In reply to comment #0)
> Description of problem:
> 
> QEMU don't support comma separated cpu list for -node numa,cpus= argument.

True for qemu 1.3, but there is an upstream qemu patch that changes that to allow escaped comma (cpus=0,,2,mem=9766) or quoted semicolon ("cpus=0;2,mem=9766") in the works:
https://lists.gnu.org/archive/html/qemu-devel/2013-01/msg01961.html

> Libvirt generates command like this:
> 
> "-numa node,nodeid=0,cpus=0,2,4,6,8,10,mem=9766 -numa
> node,nodeid=1,cpus=1,3,5,7,9,11,mem=9766"

You are correct that libvirt is generating a command line that isn't supported in older qemu, so libvirt should be fixed to fail where qemu can't support it.  Meanwhile, libvirt should use the proper syntax being added in qemu 1.4.

Comment 3 Peter Krempa 2016-02-24 08:09:19 UTC
The configuration creating a disjoint numa range was at first rejected by commit:

commit 25dc8ba08b32c7430d81228718c90d277f902f18
Author: Eric Blake <eblake>
Date:   Tue Feb 26 17:43:12 2013 -0700

    qemu: -numa doesn't (yet) support disjoint range
    
    https://bugzilla.redhat.com/show_bug.cgi?id=896092 mentions that
    qemu 1.4 and earlier only accept a simple start-stop range for
    the cpu=... argument of -numa.  Libvirt would attempt to use
    -numa cpu=1,3 for a disjoint range, which did not work as intended.
    
    Upstream qemu will be adding a new syntax for disjoint cpu ranges
    in 1.5; but the design for that syntax is still under discussion
    at the time of this patch.  So for libvirt 1.0.3, it is safest to
    just reject attempts to build an invalid qemu command line; in the
    future, we can add a capability bit and translate to the final
    accepted design for selecting a disjoint cpu range in numa.

and once qemu added support for them it was correctly enabled with commit:

commit 001b9dc1dcd568159cbe253e3736b873534bd254
Author: Martin Kletzander <mkletzan>
Date:   Tue Jun 17 14:16:59 2014 +0200

    qemu: enable disjoint numa cpu ranges


Note You need to log in before you can comment on or make changes to this bug.