Bug 2184860 - NodeSelector for tsc frequency does not tolerate small TSC variations
Summary: NodeSelector for tsc frequency does not tolerate small TSC variations
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Virtualization
Version: 4.12.2
Hardware: x86_64
OS: Linux
urgent
high
Target Milestone: ---
: 4.14.0
Assignee: Jed Lejosne
QA Contact: Kedar Bidarkar
URL:
Whiteboard:
: 2182939 2184859 (view as bug list)
Depends On:
Blocks: 2189960 2189961
TreeView+ depends on / blocked
 
Reported: 2023-04-06 01:55 UTC by Germano Veit Michel
Modified: 2023-11-08 14:05 UTC (History)
7 users (show)

Fixed In Version: v4.14.0.rhel9-546
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2189960 2189961 (view as bug list)
Environment:
Last Closed: 2023-11-08 14:05:27 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kubevirt kubevirt pull 9615 0 None Draft TSC frequency: allow a 250ppm variance when starting/migrating a TSC-enabled VMI 2023-07-31 07:46:15 UTC
Github kubevirt kubevirt pull 9640 0 None Merged TSC frequencies: add 250PPM tolerance 2023-05-02 12:08:51 UTC
Red Hat Issue Tracker CNV-27907 0 None None None 2023-04-06 02:01:41 UTC
Red Hat Knowledge Base (Solution) 7005589 0 None None None 2023-04-06 01:57:56 UTC
Red Hat Product Errata RHSA-2023:6817 0 None None None 2023-11-08 14:05:52 UTC

Description Germano Veit Michel 2023-04-06 01:55:04 UTC
Description of problem:

The node labeller marks the nodes with: 
* their exact TSC frequency
* the lowest TSC frequency in the cluster IF they support tsc-scalable

Hypothetical example, on a cluster where the lowest frequency is X

NAME   TSC-FREQUENCY  TSC-SCALABLE  TSC-FREQUENCY-X  TSC-FREQUENCY-Y
node1  X              true          true              
node2  Y              true          true             true
node3  Y              false                          true

However, TSC scaling may not be an exact number in many CPU models, leaving some possible variation.

See this, same CPU on 2 different systems (or even same system and 2 reboots), there is a 1 Mhz difference that can show up between reboots or systems with same CPU

[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 2297.345 MHz processor
[    4.127014] tsc: Refined TSC clocksource calibration: 2297.339 MHz

[    0.000000] tsc: Fast TSC calibration using PIT
[    0.000000] tsc: Detected 2297.449 MHz processor
[    4.063010] tsc: Refined TSC clocksource calibration: 2297.338 MHz

If we do X = 2297.338 
         Y = 2297.339
Then a Windows VM with re-enlightenment will never run on node3, because its missing TSC-FREQUENCY-X label by 1MHZ off.
The logic will consider this an heterogeneous cluster, but its not.

The system should be able to schedule VMs on any of those 3 nodes, regardless of TSC-SCALABLE or not. Because these are essentially the same frequency.
Lower layers accept this variance, BZ1839095

Version-Release number of selected component (if applicable):
4.12.10

How reproducible:
Always

Steps to Reproduce:
1. Use systems with TSC-SCALABLE = false and same CPUs, reboot until different frequencies.

Actual results:
* VMs fail to schedule on nodes with same CPU

Expected results:
* VMs scheduled

Comment 1 Germano Veit Michel 2023-04-06 01:57:56 UTC
*** Bug 2182939 has been marked as a duplicate of this bug. ***

Comment 3 sgott 2023-04-12 12:03:59 UTC
*** Bug 2184859 has been marked as a duplicate of this bug. ***

Comment 6 Denys Shchedrivyi 2023-09-21 18:53:43 UTC
Verified on v4.14.0.rhel9-2029

Comment 9 errata-xmlrpc 2023-11-08 14:05:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Virtualization 4.14.0 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6817


Note You need to log in before you can comment on or make changes to this bug.