Bug 1264128 - Convert tuned to use new /sys/devices/system/cpu features for isolated and nohz_full cores
Summary: Convert tuned to use new /sys/devices/system/cpu features for isolated and no...
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tuned
Version: 7.3
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Jaroslav Škarvada
QA Contact: Tereza Cerna
URL:
Whiteboard:
Keywords: Upstream
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-17 14:41 UTC by Jeremy Eder
Modified: 2016-11-04 07:25 UTC (History)
3 users (show)

(edit)
Clone Of:
(edit)
Last Closed: 2016-11-04 07:25:22 UTC


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2479 normal SHIPPED_LIVE tuned bug fix and enhancement update 2016-11-03 14:08:42 UTC

Comment 2 Jaroslav Škarvada 2016-06-16 11:24:53 UTC
The mentioned kernel patches added support to show current isolcpus and nohz_full through sysfs. How could Tuned use this information? This sysfs doesn't seem to help before the machine is rebooted with the new kernel command line.

But the current situation may need improvement. It seems that kernel is quite restrictive about isolcpus and nohz_full settings. It seems it ignores the settings if there are non-valid or non-existent CPUs specified. I think this could be improved by adding Tuned built-in function, e.g. "function_cpulist_unpack_available" or similar to filter user specified CPUs through available CPUs. It is still not optimal solution, because the number of CPUs could change between reboots, but I think it would be better than the current state.

Comment 3 Jaroslav Škarvada 2016-06-16 15:18:49 UTC
I am going to filter the cpulist through:
/sys/devices/system/cpu/present - cpus that have been identified as being present in the system.

I will name the function cpulist_present.

Comment 4 Jeremy Eder 2016-06-16 15:27:33 UTC
The way tuned could use this is to "fail" the profile load if the variables.conf info does not match cpulist_present.  It's for error-checking.

Comment 5 Jaroslav Škarvada 2016-06-16 16:08:22 UTC
(In reply to Jeremy Eder from comment #4)
> The way tuned could use this is to "fail" the profile load if the
> variables.conf info does not match cpulist_present.  It's for error-checking.

Thanks for info.

I added the following functions, which could be useful:
cpulist_present - filters the cpulist through present CPUs
cpulist_pack - to pack the cpulist, e.g. converts 1,2,3 to 1-3

I initially wanted to filter ${isolated_cores} through the cpulist_present in the Tuned profiles, but I will just add the error check.

Comment 6 Jaroslav Škarvada 2016-06-16 16:59:09 UTC
I added 'assertion' function, usage:
dummy = ${f:assertion:error_text:check_val:curr_val}

it compares check_val with curr_val and if they don't match it logs error_text and aborts profile loading. I will add the assertion to realtime profiles.

Comment 7 Jeremy Eder 2016-06-16 17:18:42 UTC
Perfect, thanks Jaroslav!

Comment 9 Jaroslav Škarvada 2016-06-17 13:03:30 UTC
Example of error logged
2016-06-17 14:10:44,267 ERROR    tuned.profiles.functions.function_assertion: assertion 'isolated_cores contains present CPU(s)' failed: '1,2,3,4,5' != '1,2,3'

Comment 12 Tereza Cerna 2016-09-08 05:36:02 UTC
==============================================
Verified in:
    tuned-2.7.1-2.el7.noarch
    tuned-profiles-realtime-2.7.1-2.el7.noarch
    tuned-profiles-nfv-2.7.1-2.el7.noarch
PASS
==============================================

Check /usr/lib/tuned/$PROFILE/tuned.conf files for realtime, realtime-virtual-host, realtime-virtual-guest profiles and patches was applied on these files.

# cat /proc/cpuinfo | grep processor | wc -l
88
# cat /etc/tuned/realtime-variables.conf 
isolated_cores=80-90
# tuned-adm profile realtime
Cannot load profile 'realtime': Assertion 'isolated_cores contains present CPU(s)' failed.
# cat /var/log/tuned/tuned.log | grep ERROR
2016-09-06 07:45:54,884 ERROR    tuned.profiles.functions.function_assertion: assertion 'isolated_cores contains present CPU(s)' failed: '80,81,82,83,84,85,86,87,88,89,90' != '80,81,82,83,84,85,86,87'
# tuned-adm active
Current active profile: balanced

The same behavior also for realtime-virtual-host and realtime-virtual-guest profiles. Content of files isolated and nohz_full in directory /sys/devices/system/cpu.

==================================================
Reproduced in:
    tuned-2.5.1-4.el7_2.3.noarch
    tuned-profiles-realtime-2.5.1-4.el7_2.3.noarch
    tuned-profiles-nfv-2.5.1-4.el7_2.3.noarch
FAIL
==================================================

# cat /proc/cpuinfo | grep processor | wc -l
88
# cat /etc/tuned/realtime-variables.conf 
isolated_cores=80-90
# tuned-adm profile realtime
# cat /var/log/tuned/tuned.log | grep ERROR
# tuned-adm active
Current active profile: realtime

Comment 14 errata-xmlrpc 2016-11-04 07:25:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2479.html


Note You need to log in before you can comment on or make changes to this bug.