RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 893654 - allow non-contiguous CPU ranges on -numa command-line options
Summary: allow non-contiguous CPU ranges on -numa command-line options
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Eduardo Habkost
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 883504
TreeView+ depends on / blocked
 
Reported: 2013-01-09 15:37 UTC by Eduardo Habkost
Modified: 2018-12-05 15:44 UTC (History)
11 users (show)

Fixed In Version: QEMU 2.1.0
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-03-05 09:42:06 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0624 0 normal SHIPPED_LIVE Important: qemu-kvm-rhev security, bug fix, and enhancement update 2015-03-05 14:37:36 UTC

Description Eduardo Habkost 2013-01-09 15:37:34 UTC 
Currently the qemu-kvm handling of -numa options don't allow non-contiguous ranges to be specified, like:

-numa cpus=0,2,4,6,8
or:
-numa cpus=0-1,4-5,8-9


We need to be able to handle these cases. Probably this will require escaping of ",", or the use of a different option separator by libvirt, as "," is already an option separator for the qemu option string parsing.
Comment 4 Eduardo Habkost 2013-11-28 14:36:25 UTC 
Unfortunately the mechanism to implement manual pinning of multi-node VMs is still under review upstream, and libvirt work to support the new mechanism will be possible only after the new mechanism gets in a QEMU release, so the manual pinning mechanism won't be in RHEL-7.0.
Comment 5 Eduardo Habkost 2014-06-04 20:41:37 UTC 
New feature -- moving to qemu-kvm-rhev component.
Comment 6 Eduardo Habkost 2014-07-10 20:03:34 UTC 
Patches were included upstream and will be in QEMU 2.1.0.
Comment 8 FuXiangChun 2014-08-28 15:21:49 UTC 
Tested this bug with qemu-kvm-rhev-2.1.0-2.el7.x86_64. and Tested three scenarios.

S1.Tested non-continues node.
#/usr/libexec/qemu-kvm -M pc-i440fx-rhel7.0.0 -name RHEL-Server-7.0-64 -m 4G -smp 4,cores=2,threads=2,sockets=1,maxcpus=160 -numa node,cpus=0,2,4,12

result:
qemu-kvm: -numa node,cpus=0,2,4,12: Invalid parameter '2

2. Tested continues nodes.
#/usr/libexec/qemu-kvm  -m 4G -smp 12,maxcpus=160 -numa node,cpus=0-5 -numa node,cpus=6-11

result(inside guest):
# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5
node 0 size: 2047 MB
node 0 free: 1349 MB
node 1 cpus: 6 7 8 9 10 11
node 1 size: 2047 MB
node 1 free: 1592 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10 

3. Tested continues nodes.
#/usr/libexec/qemu-kvm -m 6G -smp 18,maxcpus=160 -numa node,cpus=0-5 -numa node,cpus=6-11 -numa node,cpus=18-23

result(inside guest):
# numactl -H
available: 3 nodes (0-2)
node 0 cpus: 0 1 2 3 4 5 12 13 14 15 16 17
node 0 size: 2047 MB
node 0 free: 1220 MB
node 1 cpus: 6 7 8 9 10 11
node 1 size: 2047 MB
node 1 free: 1703 MB
node 2 cpus:
node 2 size: 2048 MB
node 2 free: 1982 MB
node distances:
node   0   1   2 
  0:  10  20  20 
  1:  20  10  20 
  2:  20  20  10 

Test summary
For S1 result.
seem qemu-kvm-rhev-2.1 don't support it. 

For S2 result.
it is expect result.

For S3 result.
node 2 don't include anyone cpu. not sure if it is expect, As host only have two nodes. so it depend on host numa node number. right?

Eduardo,
would you please take a look at this test result?  seem this bug is not fixed. If my understanding is wrong. please correct me.
Comment 9 FuXiangChun 2014-08-28 15:24:57 UTC 
Another, This is host numa architecture.
# numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 12 13 14 15 16 17
node 0 size: 12253 MB
node 0 free: 8453 MB
node 1 cpus: 6 7 8 9 10 11 18 19 20 21 22 23
node 1 size: 16383 MB
node 1 free: 14073 MB
node distances:
node   0   1 
  0:  10  20 
  1:  20  10
Comment 10 Eduardo Habkost 2014-08-28 16:38:29 UTC 
(In reply to FuXiangChun from comment #8)
> Tested this bug with qemu-kvm-rhev-2.1.0-2.el7.x86_64. and Tested three
> scenarios.
> 
> S1.Tested non-continues node.
> #/usr/libexec/qemu-kvm -M pc-i440fx-rhel7.0.0 -name RHEL-Server-7.0-64 -m 4G
> -smp 4,cores=2,threads=2,sockets=1,maxcpus=160 -numa node,cpus=0,2,4,12
> 
> result:
> qemu-kvm: -numa node,cpus=0,2,4,12: Invalid parameter '2

That's not how it works. The right format is: "cpus=0,cpus=2,cpus=4,cpus=12".


> 3. Tested continues nodes.
> #/usr/libexec/qemu-kvm -m 6G -smp 18,maxcpus=160 -numa node,cpus=0-5 -numa
> node,cpus=6-11 -numa node,cpus=18-23

You have only 18 VCPUs (0-17), and you are trying to assign offline (unplugged) VCPUs 18-23 to node 2. numactl will show only the online CPUs. You forgot to assign VCPUs 11-17.



> Test summary
> For S1 result.
> seem qemu-kvm-rhev-2.1 don't support it. 

The syntax is incorrect.

> 
> For S2 result.
> it is expect result.

Correct.

> 
> For S3 result.
> node 2 don't include anyone cpu. not sure if it is expect, As host only have
> two nodes. so it depend on host numa node number. right?

The guest NUMA topology has absolutely nothing to do with the host topology, and depend solely on the QEMU command-line options.
Comment 11 FuXiangChun 2014-08-29 02:01:12 UTC 
According to comment 10. Re-tested S1 and S3.

S1.
Boot guest with -smp 6,...-numa node,cpus=0,cpus=2,cpus=4,cpus=17,cpus=11,cpus=22

got guest numa topology inside guest.
# numactl -H
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5
node 0 size: 27647 MB
node 0 free: 25965 MB
node distances:
node   0 
  0:  10 

S3.
Boot guest with 
-m 27G -smp 24,maxcpus=160 \
-numa node,cpus=0-5 -numa node,cpus=6-11 -numa node,cpus=12-17 -numa node,cpus=18-23 \

Got numa topology inside guest.
# numactl -H
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 5
node 0 size: 6911 MB
node 0 free: 6239 MB
node 1 cpus: 6 7 8 9 10 11
node 1 size: 6912 MB
node 1 free: 6450 MB
node 2 cpus: 12 13 14 15 16 17
node 2 size: 6912 MB
node 2 free: 6625 MB
node 3 cpus: 18 19 20 21 22 23
node 3 size: 6912 MB
node 3 free: 6585 MB
node distances:
node   0   1   2   3 
  0:  10  20  20  20 
  1:  20  10  20  20 
  2:  20  20  10  20 
  3:  20  20  20  10 

According to comment 10, This is expect result.  Base on test result of S1 S2 and S3.  this bug is fixed.
Comment 14 errata-xmlrpc 2015-03-05 09:42:06 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0624.html
Note You need to log in before you can comment on or make changes to this bug.