Bug 1295203

Summary: Scheduling CPU filter doesn't take the new "thread-per-core" value in consideration
Product: [oVirt] ovirt-engine Reporter: sefi litmanovich <slitmano>
Component: Backend.CoreAssignee: Jenny Tokar <jtokar>
Status: CLOSED NOTABUG QA Contact: meital avital <mavital>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.6.2CC: bugs, dfediuck, mlibra
Target Milestone: ovirt-3.6.6Flags: dfediuck: ovirt-3.6.z?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-03-16 15:14:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description sefi litmanovich 2016-01-03 14:03:33 UTC
Description of problem:

As a result of bz: https://bugzilla.redhat.com/show_bug.cgi?id=1267165 a new "thread per core" parameter was added to vm's system configurations.
It seems though that when scheduler checks vm's CPU it only multiplies Virtual_sockets * core_per_virtual_scoket but ignores threads_per_core.
Due to this problem I was able e.g. to run on a host with:
8 CPUS = 1_socket * 4_cores_per_socket * 2_threads_per_core..
a vm with a total of 16 Virtual CPU = 2_virtual_sockets * 1_core_per_socket * 8_threads_per_core.
running this vm was not blocked due to scheduling filter (policy is default=None with CPU filter).
When I edited the vm and kept 16 Virtual CPUs but this time with 8 cores_per_socket and 1 threads_per_core (keeping 2 virutal sockets), when trying to run the vm I did get the expected scheduling filter error.

Version-Release number of selected component (if applicable):

rhevm-3.6.2-0.1.el6.noarch

How reproducible:

always

Steps to Reproduce:
1. say you have a cluster with a host of 8 CPU, cluster scheduling policy set to None(Default).
2. create a vm -> in system tab set:
Virtual CPU = 16
virtual_sockets = 2
core_per_socket = 1
threads_per_core = 8
3. run vm.

Actual results:

Vm starts normally.

Expected results:

Vm doesn't start, a message regarding no host satisfying scheduling policy should be invoked.

Comment 1 Marek Libra 2016-01-05 08:28:16 UTC
This was intended behavior since the threads-per-core leverages the simultaneous multithreading.
The vCPUs for scheduling are computed as: num_of_sockets * num_of_cores.
The vCPUs from the guest perspective are computed as num_of_sockets * num_of_cores * num_of_threads.

It is suggested to keep the num_of_threads (guest Threads per Core) set to
- 1 for x86
- 1..8 for ppc
The default is 1, which is the value used before introducing this configuration.
Increase it to explicitly set on or leverage the smt.

The CPU topology is not reported consistently from libvirt.
On PPC, the threads are set on depending on actual load, means they are recently not always reported - i.e. instead of expected 160 CPUs (20 * 8 threads) we get just 20.
In other words, counting guest vthreads during scheduling can lead to underutilization and there would be walk around for it in ovirt.

The guest threads-per-core can lead to overutilization but only when explicitly set.

For completeness, there's an option Count Threads as Cores. It is related to host threads only and makes probably more sense on x86.
When this option is set on, the SMT threads will be reported as cores.

Comment 2 Gil Klein 2016-01-06 13:39:48 UTC
*** Bug 1295204 has been marked as a duplicate of this bug. ***

Comment 3 Martin Sivák 2016-03-16 15:14:03 UTC
The idea here is to check whether the VM can get access to the requested number of fully parallel cores.

Your VM:

Virtual CPU = 16
virtual_sockets = 2
core_per_socket = 1
threads_per_core = 8

= you require only two physical (fully parallel) cores and allow simulating eight threads to coexist on each.

The host has

1_socket * 4_cores_per_socket (* 2_threads_per_core)

= four physical cores that guarantee full parallelism


Wikipedia states:

"Hyper-threading works by duplicating certain sections of the processor—those that store the architectural state—but not duplicating the main execution resources."

Threads are not considered to be a fully parallel unit (only one thread can use the bus of the core at any given moment) and that is why the cpu policy unit only compares whether the requested amount of fully parallel real cores is available. You can simulate as many threads on those as you want (within some reasonable limits).


You really have two options:

1) If you really require access to full cores, update the virtual topology to use more cores and less threads for the same number of vCPUs

2) If you are ok with threads, use threads