Bug 1957375

Summary: [RFE] x86, tsc: Add kcmdline args for skipping tsc calibration sequences
Product: Red Hat Enterprise Linux 8 Reporter: Akshay Ratnaparkhe <aratnapa>
Component: kernelAssignee: Prarit Bhargava <prarit>
kernel sub component: x86_64 QA Contact: William Gomeringer <wgomerin>
Status: CLOSED ERRATA Docs Contact:
Severity: urgent    
Priority: urgent CC: acaringi, chaithco, darcari, ekovsky, kyoshida, prarit, rvr, skamboj, ttuffin, wgomerin
Version: 8.3Keywords: FutureFeature, Reopened, Triaged
Target Milestone: beta   
Target Release: 8.6   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-4.18.0-361.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-05-10 14:57:41 UTC Type: Feature Request
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
turbostat
none
native_calibrate_cpu none

Description Akshay Ratnaparkhe 2021-05-05 17:19:35 UTC 
Description of problem:

(Below details were shared by the customer).

o Incorrect TSC calibration for overclocked BCLK
o In case of platforms overclocked by bumping BCLK (bus frequency) one ends up with TSC impacted as well.

o On one such hardware that we've evaluated we've seen TSC go up from the nominal 3.4GHz to 3.502GHz. 
This was evidenced by the output of turbostat (see attachement).

o Due to the way the kernel performs TSC calibration, the discovered frequency is still the nominal 3.4GHz as shown by dmesg.
-----------------------------------------------------
> dmesg | grep "tsc: Detected"
[    0.000000] tsc: Detected 3400.000 MHz processor
-----------------------------------------------------

o Modern kernels heavily rely on CPUID to obtain crystal frequency for the TSC (which for our Xeon 6246R SKU is 3.4GHz). With the actual frequency of TSC at 3502MHz the sfptp daemon will never be able to settle at stable freq adjust values and fail to synchronize the system clock against the NIC. Other kernel subsystems that rely on correctly identifying current frequency will also be impacted.

o This can be addressed by forcing the kernel to resort to using MSR rather than CPUID for obtaining TSC frequency in native_calibrate_cpu(void) (see attachement).

o Introducing additional kernel cmdline params to drive calibration method (ie. disabling cpuid as calibration source) would entirely solve this edge case without changing the default behaviour. Such change was successfully tested in customer's environments.

* Important Note:
o Customer has also given link of conversation in between Red Hat and Supermicro and wants to get follow up on the same.

https://lore.kernel.org/lkml/1543865843219.51523@supermicro.com/T/
OR https://marc.info/?t=153151085300010&r=1&w=2


* Request:
Is it possible to merge this patch in RHEL 8?


Version-Release number of selected component (if applicable):

RHEL 8.3.


How reproducible:
Customer has mentioned that it is always reproducible.


Steps to Reproduce:
Not known.


Actual results:
o overclocking impacts PTP clock synchronisation.

Expected results:
o Overclocking should not impact PTP clock synchronisation.

Additional info:
N/A.
Comment 1 Akshay Ratnaparkhe 2021-05-05 17:21:59 UTC 
Created attachment 1779900 [details]
turbostat
Comment 2 Akshay Ratnaparkhe 2021-05-05 17:22:50 UTC 
Created attachment 1779901 [details]
native_calibrate_cpu
Comment 13 Prarit Bhargava 2021-10-14 16:56:03 UTC 
Does the "tsc_early_khz=" parameter help?

        tsc_early_khz=  [X86] Skip early TSC calibration and use the given
                        value instead. Useful when the early TSC frequency discovery
                        procedure is not reliable, such as on overclocked systems
                        with CPUID.16h support and partial CPUID.15h support.
                        Format: <unsigned int>

P.
Comment 27 RHEL Program Management 2021-11-29 16:00:27 UTC 
Development Management has reviewed and declined this request. You may appeal this decision by reopening this request.
Comment 63 errata-xmlrpc 2022-05-10 14:57:41 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: kernel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1988