Bug 1955388 - Auto Pinning Policy only pins some of the vCPUs on a single NUMA host
Summary: Auto Pinning Policy only pins some of the vCPUs on a single NUMA host
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 4.4.6
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: ovirt-4.5.2
: ---
Assignee: Liran Rotenberg
QA Contact: Polina
URL:
Whiteboard:
: 2068929 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-04-30 02:39 UTC by Germano Veit Michel
Modified: 2022-09-08 18:30 UTC (History)
7 users (show)

Fixed In Version: ovirt-engine-4.5.2
Doc Type: Bug Fix
Doc Text:
Previously, the Manager was able to start a virtual machine with a Resize and Pin NUMA policy on a host whose physical sockets did not correspond to the number of NUMA nodes. As a result, the wrong pinning was assigned to the policy. With this release, the Manager does not allow the virtual machine to be scheduled on such a host, making the pinning correct based on the algorithm.
Clone Of:
Environment:
Last Closed: 2022-09-08 11:28:54 UTC
oVirt Team: Virt
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github oVirt ovirt-engine pull 76 0 None open core: validate amount of sockets equal to that of numa nodes 2022-03-15 09:58:10 UTC
Red Hat Knowledge Base (Solution) 6006101 0 None None None 2021-04-30 04:31:01 UTC
Red Hat Product Errata RHSA-2022:6393 0 None None None 2022-09-08 11:29:50 UTC
oVirt gerrit 118148 0 master POST core: validate amount of sockets equal to that of numa nodes 2022-01-03 12:51:11 UTC

Description Germano Veit Michel 2021-04-30 02:39:35 UTC
Description of problem:

The logic is not working well if the host contains a single node and the user just wants an easy way to pin all vCPUs instead of specifying manually.

1. Host with 1 NUMA

# numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31
node 0 size: 3922 MB
node 0 free: 2689 MB
node distances:
node   0 
  0:  10 

2. Create VM with 24 CPUs:

engine=# select vm_name,cpu_pinning,num_of_sockets,cpu_per_socket,threads_per_cpu from vm_static where vm_name = 'VM1';
 vm_name | cpu_pinning | num_of_sockets | cpu_per_socket | threads_per_cpu 
---------+-------------+----------------+----------------+-----------------
 VM1     |             |             12 |              2 |               1

3. Use the 'Existing' policy:
   VM -> Edit -> Host -> Auto Pinning Policy -> Existing  -> OK

4. Result, only half of the vCPUs pinned:

engine=# select vm_name,cpu_pinning,num_of_sockets,cpu_per_socket,threads_per_cpu from vm_static where vm_name = 'VM1';
 vm_name |                     cpu_pinning                      | num_of_sockets | cpu_per_socket | threads_per_cpu 
---------+------------------------------------------------------+----------------+----------------+-----------------
 VM1     | 0#1_1#2_2#3_3#4_4#5_5#6_6#7_7#8_8#9_9#10_10#11_11#12 |             12 |              2 |               1


Version-Release number of selected component (if applicable):
rhvm-4.4.6.5-0.17.el8ev.noarch

How reproducible:
Always

Steps to Reproduce:
As above.

Actual results:
* Some of the vCPUs pinned (i.e.: if the VM and Host have 4 vCPUs it pins just 1)

Expected results:
* Pin all of them

Comment 1 Liran Rotenberg 2021-05-04 08:18:37 UTC
Can we get the output of the host CPU topology (lscpu)?

Comment 2 Germano Veit Michel 2021-05-06 01:16:39 UTC
(In reply to Liran Rotenberg from comment #1)
> Can we get the output of the host CPU topology (lscpu)?

Sure. This is a weird scenario.

This is the "host" :)

    <topology sockets='2' dies='1' cores='16' threads='1'/>

    <numa>
      <cell id='0' cpus='0-31' memory='1048576' unit='KiB'/>
    </numa>

This case 2 sockets are in a single NUMA node. The algorithm seems to use only the first socket of that NUMA node.
I think this only happens on older systems (dual cpu on those with memory controller on chipset) and/or when customers
"disable" numa on the bios setting so everything is reported on a single node. 

The other weird setting is with those Threadripper CPUs, where it may report 2 NUMA with a single socket. But I have
not tested that scenario.

Let's change this to Low severity.

[root@localhost ~]# lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  1
Core(s) per socket:  16
Socket(s):           2
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      QEMU
CPU family:          6
Model:               94
Model name:          Intel Core Processor (Skylake, IBRS)
BIOS Model name:     pc-q35-5.1
Stepping:            3
CPU MHz:             2111.998
BogoMIPS:            4223.99
Virtualization:      VT-x
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            4096K
L3 cache:            16384K
NUMA node0 CPU(s):   0-31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap clflushopt xsaveopt xsavec xgetbv1 xsaves arat umip md_clear arch_capabilities

Comment 3 Arik 2022-03-31 11:52:14 UTC
*** Bug 2068929 has been marked as a duplicate of this bug. ***

Comment 4 meital avital 2022-07-20 08:34:02 UTC
Due to QE capacity we are not going to cover this issue in our automation

Comment 8 Polina 2022-07-26 14:49:26 UTC
Verified on ovirt-engine-4.5.2-0.3.el8ev.noarch

Running resize_and_pin VM on host

on host - 
CPU(s):              48
On-line CPU(s) list: 0-47
Thread(s) per core:  2
Core(s) per socket:  24
Socket(s):           1
NUMA node(s):        4



The host host_mixed_1 did not satisfy internal filter CPUTopology because cannot apply the CPU pinning policy 'Resize and Pin' on a host when the number of CPU sockets is not equal to the number of NUMA nodes.

Comment 10 Casper (RHV QE bot) 2022-09-04 21:33:37 UTC
This bug has low overall severity and is not going to be further verified by QE. If you believe special care is required, feel free to properly align relevant severity, flags and keywords to raise PM_Score or use one of the Bumps ('PrioBumpField', 'PrioBumpGSS', 'PrioBumpPM', 'PrioBumpQA') in Keywords to raise it's PM_Score above verification threashold (1000).

Comment 13 errata-xmlrpc 2022-09-08 11:28:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: RHV Manager (ovirt-engine) [ovirt-4.5.2] bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6393


Note You need to log in before you can comment on or make changes to this bug.