Bug 1277736 - [RFE][nova]: Virt driver pinning guest vCPUs threads to host pCPUs threads
Summary: [RFE][nova]: Virt driver pinning guest vCPUs threads to host pCPUs threads
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 7.0 (Kilo)
Hardware: Unspecified
OS: Unspecified
high
low
Target Milestone: ga
: 9.0 (Mitaka)
Assignee: Eoghan Glynn
QA Contact: Prasanth Anbalagan
URL: https://blueprints.launchpad.net/nova...
Whiteboard: upstream_milestone_mitaka-3 upstream_...
: 1305568 (view as bug list)
Depends On: 1333195 1333457
Blocks: 1292592 1304381 1305568 1309775 1342856
TreeView+ depends on / blocked
 
Reported: 2015-11-03 23:15 UTC by Stephen Gordon
Modified: 2020-04-15 14:18 UTC (History)
16 users (show)

Fixed In Version: openstack-nova-13.0.0-1.el7ost
Doc Type: Enhancement
Doc Text:
Clone Of:
: 1304381 (view as bug list)
Environment:
Last Closed: 2016-08-24 12:51:49 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1550317 0 None None None 2016-06-08 16:05:54 UTC
OpenStack gerrit 202647 0 None MERGED Add 'CPUThreadAllocationPolicy' enum field 2020-08-17 14:09:10 UTC
OpenStack gerrit 202648 0 None MERGED Add 'cpu_policy' and 'cpu_thread_policy' fields 2020-08-17 14:09:10 UTC
OpenStack gerrit 202649 0 None ABANDONED Add 'hw:cpu_threads_policy=avoid' filtering 2020-08-17 14:09:10 UTC
OpenStack gerrit 202650 0 None MERGED trivial: Add some logs to 'numa_topology_filter' 2020-08-17 14:09:09 UTC
OpenStack gerrit 202651 0 None MERGED Add 'hw:cpu_thread_policy=require' scheduling 2020-08-17 14:09:09 UTC
OpenStack gerrit 202652 0 None ABANDONED Add 'hw:cpu_threads_policy=separate' scheduling 2020-08-17 14:09:10 UTC
OpenStack gerrit 202653 0 None MERGED Make use of 'InstanceNUMACell.cpu_policy' field 2020-08-17 14:09:09 UTC
OpenStack gerrit 202654 0 None ABANDONED trivial: Make use of 'CPU(Thread)Allocation' enum 2020-08-17 14:09:10 UTC
OpenStack gerrit 202655 0 None MERGED Add 'hw:cpu_threads_policy=isolate' scheduling 2020-08-17 14:09:09 UTC
OpenStack gerrit 285232 0 None MERGED virt/hardware: Check for threads when "required" 2020-08-17 14:09:09 UTC
OpenStack gerrit 326944 0 None MERGED virt/hardware: Fix 'isolate' case on non-SMT hosts 2020-08-17 14:09:09 UTC
Red Hat Product Errata RHEA-2016:1761 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 9 General Availability Advisory 2016-08-24 16:49:52 UTC

Description Stephen Gordon 2015-11-03 23:15:55 UTC
Cloned from launchpad blueprint https://blueprints.launchpad.net/nova/+spec/virt-driver-cpu-thread-pinning.

Description:

This feature aims to implement the remaining functionality of the 'virt-driver-cpu-pinning' story. This "remaining functionality" entails adding support for thread policies

Specification URL (additional information):

http://specs.openstack.org/openstack/nova-specs/specs/liberty/approved/virt-driver-cpu-pinning.html

Comment 2 Stephen Gordon 2016-02-11 08:29:37 UTC
*** Bug 1305568 has been marked as a duplicate of this bug. ***

Comment 3 Mike McCune 2016-03-28 22:33:16 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 4 Stephen Gordon 2016-06-08 16:05:35 UTC
Sahid do we also need to pick up:

https://review.openstack.org/326944
https://review.openstack.org/285232

Comment 5 Sahid Ferdjaoui 2016-06-09 11:16:39 UTC
(In reply to Stephen Gordon from comment #4)
> Sahid do we also need to pick up:
>
> https://review.openstack.org/326944
> https://review.openstack.org/285232

We are going to get the first one for free since it will be backported to stable/mitaka. The second one is a nice to have in case users request NUMA things without to have activated the NUMATopologyFilter.

I'll try to follow both patches to make sure we will have them for GA.

Comment 10 Prasanth Anbalagan 2016-08-18 15:00:43 UTC
Verified as follows - Instances are created successfully (pinned to VCPUS) with no overlap between instances even when they are on same host.

***********
VERSION
***********

[root@serverX]# yum list installed | grep openstack-nova
openstack-nova-api.noarch            1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-cert.noarch           1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-common.noarch         1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-compute.noarch        1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-conductor.noarch      1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-console.noarch        1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-novncproxy.noarch     1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
openstack-nova-scheduler.noarch      1:13.1.1-2.el7ost       @rhelosp-9.0-puddle
[root@serverX]# 

******
LOGS
******

[root@serverX]# nova flavor-show tempest-test_flavor_prefer_-795153569
+----------------------------+--------------------------------------------------------------------------------------------------------------------+
| Property                   | Value                                                                                                              |
+----------------------------+--------------------------------------------------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                                                                                              |
| OS-FLV-EXT-DATA:ephemeral  | 0                                                                                                                  |
| disk                       | 5                                                                                                                  |
| extra_specs                | {"aggregate_instance_extra_specs:pinned": "true", "hw:cpu_policy": "dedicated", "hw:cpu_threads_policy": "prefer"} |
| id                         | 1662977424                                                                                                         |
| name                       | tempest-test_flavor_prefer_-795153569                                                                              |
| os-flavor-access:is_public | True                                                                                                               |
| ram                        | 512                                                                                                                |
| rxtx_factor                | 1.0                                                                                                                |
| swap                       |                                                                                                                    |
| vcpus                      | 2                                                                                                                  |
+----------------------------+--------------------------------------------------------------------------------------------------------------------+

[root@serverX]# nova flavor-show 200
+----------------------------+--------------------------------+
| Property                   | Value                          |
+----------------------------+--------------------------------+
| OS-FLV-DISABLED:disabled   | False                          |
| OS-FLV-EXT-DATA:ephemeral  | 0                              |
| disk                       | 5                              |
| extra_specs                | {"hw:cpu_policy": "dedicated"} |
| id                         | 200                            |
| name                       | m1.dedicated                   |
| os-flavor-access:is_public | True                           |
| ram                        | 512                            |
| rxtx_factor                | 1.0                            |
| swap                       |                                |
| vcpus                      | 2                              |
+----------------------------+--------------------------------+
[root@serverX]# 

[root@serverX]# nova boot --flavor 1662977424 --image cirros vm1
+--------------------------------------+----------------------------------------------------+
| Property                             | Value                                              |
+--------------------------------------+----------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                             |
| OS-EXT-AZ:availability_zone          |                                                    |
| OS-EXT-SRV-ATTR:host                 | -                                                  |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                                  |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000041                                  |
| OS-EXT-STS:power_state               | 0                                                  |
| OS-EXT-STS:task_state                | scheduling                                         |
| OS-EXT-STS:vm_state                  | building                                           |
| OS-SRV-USG:launched_at               | -                                                  |
| OS-SRV-USG:terminated_at             | -                                                  |
| accessIPv4                           |                                                    |
| accessIPv6                           |                                                    |
| adminPass                            | HZC3q4NRVdvM                                       |
| config_drive                         |                                                    |
| created                              | 2016-08-18T14:55:48Z                               |
| flavor                               | tempest-test_flavor_prefer_-795153569 (1662977424) |
| hostId                               |                                                    |
| id                                   | 88d944be-e31d-4df7-a33e-b1acaeb156f3               |
| image                                | cirros (9c63f09c-dffc-467d-bb86-c926a822f6a4)      |
| key_name                             | -                                                  |
| metadata                             | {}                                                 |
| name                                 | vm1                                                |
| os-extended-volumes:volumes_attached | []                                                 |
| progress                             | 0                                                  |
| security_groups                      | default                                            |
| status                               | BUILD                                              |
| tenant_id                            | 1e8409b372294934841eeb5e5ef5cde4                   |
| updated                              | 2016-08-18T14:55:48Z                               |
| user_id                              | e62260f2a0b94f519e1e9cd9239c14b7                   |
+--------------------------------------+----------------------------------------------------+
[root@serverX]# nova boot --flavor 200 --image cirros vm2
+--------------------------------------+-----------------------------------------------+
| Property                             | Value                                         |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                        |
| OS-EXT-AZ:availability_zone          |                                               |
| OS-EXT-SRV-ATTR:host                 | -                                             |
| OS-EXT-SRV-ATTR:hypervisor_hostname  | -                                             |
| OS-EXT-SRV-ATTR:instance_name        | instance-00000042                             |
| OS-EXT-STS:power_state               | 0                                             |
| OS-EXT-STS:task_state                | scheduling                                    |
| OS-EXT-STS:vm_state                  | building                                      |
| OS-SRV-USG:launched_at               | -                                             |
| OS-SRV-USG:terminated_at             | -                                             |
| accessIPv4                           |                                               |
| accessIPv6                           |                                               |
| adminPass                            | j5ZfoA7fDJhs                                  |
| config_drive                         |                                               |
| created                              | 2016-08-18T14:56:00Z                          |
| flavor                               | m1.dedicated (200)                            |
| hostId                               |                                               |
| id                                   | a31ccd34-65c3-43d3-9e3a-f4feb338cdda          |
| image                                | cirros (9c63f09c-dffc-467d-bb86-c926a822f6a4) |
| key_name                             | -                                             |
| metadata                             | {}                                            |
| name                                 | vm2                                           |
| os-extended-volumes:volumes_attached | []                                            |
| progress                             | 0                                             |
| security_groups                      | default                                       |
| status                               | BUILD                                         |
| tenant_id                            | 1e8409b372294934841eeb5e5ef5cde4              |
| updated                              | 2016-08-18T14:56:00Z                          |
| user_id                              | e62260f2a0b94f519e1e9cd9239c14b7              |
+--------------------------------------+-----------------------------------------------+
[root@serverX]# 


[root@serverX]# nova list
+--------------------------------------+------+--------+------------+-------------+---------------------+
| ID                                   | Name | Status | Task State | Power State | Networks            |
+--------------------------------------+------+--------+------------+-------------+---------------------+
| 88d944be-e31d-4df7-a33e-b1acaeb156f3 | vm1  | ACTIVE | -          | Running     | public=172.24.4.234 |
| a31ccd34-65c3-43d3-9e3a-f4feb338cdda | vm2  | ACTIVE | -          | Running     | public=172.24.4.235 |
+--------------------------------------+------+--------+------------+-------------+---------------------+

[root@serverX]# virsh vcpupin 88d944be-e31d-4df7-a33e-b1acaeb156f3
VCPU: CPU Affinity
----------------------------------
   0: 1
   1: 7

[root@serverX]# virsh vcpupin a31ccd34-65c3-43d3-9e3a-f4feb338cdda
VCPU: CPU Affinity
----------------------------------
   0: 8
   1: 2

[root@serverX]# lscpu -p
# The following is the parsable format, which can be fed to other
# programs. Each different item in every column has an unique ID
# starting from zero.
# CPU,Core,Socket,Node,,L1d,L1i,L2,L3
0,0,0,0,,0,0,0,0
1,1,0,0,,1,1,1,0
2,2,0,0,,2,2,2,0
3,3,0,0,,3,3,3,0
4,4,0,0,,4,4,4,0
5,5,0,0,,5,5,5,0
6,0,0,0,,0,0,0,0
7,1,0,0,,1,1,1,0
8,2,0,0,,2,2,2,0
9,3,0,0,,3,3,3,0
10,4,0,0,,4,4,4,0
11,5,0,0,,5,5,5,0
[root@serverX]#

Comment 12 errata-xmlrpc 2016-08-24 12:51:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1761.html

Comment 13 Robert Love 2016-09-29 15:16:59 UTC
(In reply to Sahid Ferdjaoui from comment #5)
> (In reply to Stephen Gordon from comment #4)
> > Sahid do we also need to pick up:
> >
> > https://review.openstack.org/326944
> > https://review.openstack.org/285232
> 
> We are going to get the first one for free since it will be backported to
> stable/mitaka. The second one is a nice to have in case users request NUMA
> things without to have activated the NUMATopologyFilter.
> 
> I'll try to follow both patches to make sure we will have them for GA.

Did both of these change(set)s made it into OSP 9?

Comment 14 Sahid Ferdjaoui 2016-09-30 13:11:43 UTC
(In reply to Robert Love from comment #13)
> (In reply to Sahid Ferdjaoui from comment #5)
> > (In reply to Stephen Gordon from comment #4)
> > > Sahid do we also need to pick up:
> > >
> > > https://review.openstack.org/326944
> > > https://review.openstack.org/285232
> > 
> > We are going to get the first one for free since it will be backported to
> > stable/mitaka. The second one is a nice to have in case users request NUMA
> > things without to have activated the NUMATopologyFilter.
> > 
> > I'll try to follow both patches to make sure we will have them for GA.
> 
> Did both of these change(set)s made it into OSP 9?

The first one yes. The second one no, I don't think there is any BZ opened against it, as I have indicated it's a nice to have in case users request NUMA things without to have activated the NUMATopologyFilter which is clearly something we do not support.


Note You need to log in before you can comment on or make changes to this bug.