+++ This bug was initially created as a clone of Bug #1947858 +++
Description of problem
* Verification and validation of an active tuned(8) profile is a two fold issue:
1) $ tuned-adm(8) verify
- Does not seem to validate all configuration settings done by an active profile.
Ex. while verifying 'realtime' profiles, tuned(8) calls respective 'script.sh verify'
function via 'plugin_script.py' plugin.
===
$ sort -t'.' -k2 /tmp/tuned.log | awk -f tlog.awk |less
[tuned.plugins.plugin_script:]
calling script '/usr/lib/tuned/realtime/script.sh' with arguments '['verify']'
calling script '/usr/lib/tuned/realtime-virtual-host/script.sh' with arguments '['verify']'
verify: passed: '['/usr/lib/tuned/realtime/script.sh', '/usr/lib/tuned/realtime-virtual-host/script.sh']'
===
- But respective script.sh:verify() functions do not validate parameters set by start() functions.
2) $ tuned-adm(8) verify
- Does not offer convenient user-interface/interaction for users to be able to see and confirm
that tuned-adm(8) verify is working well.
- It does not present verification results to a user properly.
Verification results get logged to the /var/log/tnued/tuned.log file.
===
# tuned-adm verify
Verification failed, current system settings differ from the preset profile.
You can mostly fix this by restarting the Tuned daemon, e.g.:
systemctl restart tuned
or
service tuned restart
Sometimes (if some plugins like bootloader are used) a reboot may be required.
See tuned log file ('/var/log/tuned/tuned.log') for details.
===
Version-Release number of selected component (if applicable):
$ cat /etc/redhat-release
Red Hat Enterprise Linux release 8.4 Beta (Ootpa)
$ rpm -q tuned
tuned-2.15.0-2.el8.noarch
$
$ tuned --versio
tuned 2.15.0
How reproducible: 100%
Steps to Reproduce:
1. install tuned
2. activate realtime profile
3. run tuned-adm verify
Expected results:
- Make it easy for users to validate active profile settings with certainty.
--- Additional comment from Prasad J Pandit on 2021-04-09 12:53:47 UTC ---
===
1) $ tuned-adm(8) verify
- Does not seem to validate all configuration settings done by an active profile.
Ex. while verifying 'realtime' profiles, tuned(8) calls respective 'script.sh verify'
function via 'plugin_script.py' plugin.
===
* This is a proposed patch to address this issue.
* One limitation is a Boolean return value from verify() function fails to convey if all tests passed or all tests failed. There could be a case wherein some tests pass and some fail.
* Another limitation is, output from these verify() functions does not get logged to the /var/log/tuned/tuned.log file.
--- Additional comment from Prasad J Pandit on 2021-04-09 13:18:20 UTC ---
===
2) $ tuned-adm(8) verify
- Does not offer convenient user-interface/interaction for users to be able to see and confirm
that tuned-adm(8) verify is working well.
- It does not present verification results to a user properly.
Verification results get logged to the /var/log/tnued/tuned.log file.
===
* This is a proposed patch to address above issue.
* One limitation is it only reflects verification status for a section (ex. [sysctl])
It does not show results for each parameter verification as logged in the /var/log/tuned/tuned.log file.
--- Additional comment from Prasad J Pandit on 2021-04-09 13:28:46 UTC ---
===
* One limitation is it only reflects verification status for a section (ex. [sysctl])
It does not show results for each parameter verification as logged in the /var/log/tuned/tuned.log file.
===
* This is a quick awk(1) script to parse '/var/log/tuned/tuned.log' file and better present the verifications result for each parameter.
* $ sort -t'.' -k2 /tmp/tuned.log | awk -f tlog.awk
[tuned.plugins.base:]
verify: failed: 'kernel.numa_balancing' = 'None', expected ''
verify: failed: 'net.core.busy_poll' = 'None', expected '50'
verify: failed: 'net.core.busy_read' = 'None', expected '50'
verify: passed: device cpu0: 'energy_perf_bias' = 'performance'
verify: passed: device cpu10: 'energy_perf_bias' = 'performance'
verify: passed: device cpu11: 'energy_perf_bias' = 'performance'
...
[tuned.plugins.plugin_net:]
unknown channels parameter(s): {'check_net_queue_count'}
unknown channels parameter(s): {'check_net_queue_count'}
unknown channels parameter(s): {'check_net_queue_count'}
unknown channels parameter(s): {'check_net_queue_count'}
unknown channels parameter(s): {'check_net_queue_count'}
unknown channels parameter(s): {'check_net_queue_count'}
...
[tuned.plugins.plugin_scheduler:]
verify: failed: 'SMP affinity of IRQ 0' = '[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]', expected '[0, 1, 2, 3, 4, 6, 8, 10, 12, 13, 14, 15]'
verify: failed: 'SMP affinity of IRQ 40' = '[5]', expected '[0, 1, 2, 3, 4, 6, 8, 10, 12, 13, 14, 15]'
verify: failed: 'SMP affinity of IRQ 41' = '[7]', expected '[0, 1, 2, 3, 4, 6, 8, 10, 12, 13, 14, 15]'
verify: failed: 'SMP affinity of IRQ 42' = '[9]', expected '[0, 1, 2, 3, 4, 6, 8, 10, 12, 13, 14, 15]'
verify: failed: 'SMP affinity of IRQ 43' = '[11]', expected '[0, 1, 2, 3, 4, 6, 8, 10, 12, 13, 14, 15]'
verify: passed: 'default IRQ SMP affinity' = '[0, 1, 2, 3, 4, 6, 8, 10, 12, 13, 14, 15]'
verify: passed: 'SMP affinity of IRQ 100' = '[0, 2, 4, 6, 8, 10, 12, 14]'
verify: passed: 'SMP affinity of IRQ 10' = '[0, 2, 4, 6, 8, 10, 12, 14]'
verify: passed: 'SMP affinity of IRQ 101' = '[0, 2, 4, 6, 8, 10, 12, 14]'
verify: passed: 'SMP affinity of IRQ 102' = '[0, 2, 4, 6, 8, 10, 12, 14]'
...
[tuned.plugins.plugin_script:]
calling script '/usr/lib/tuned/realtime/script.sh' with arguments '['verify']'
calling script '/usr/lib/tuned/realtime-virtual-host/script.sh' with arguments '['verify']'
verify: passed: '['/usr/lib/tuned/realtime/script.sh', '/usr/lib/tuned/realtime-virtual-host/script.sh']'
[tuned.plugins.plugin_sysctl:]
Failed to read sysctl parameter 'kernel.numa_balancing', the parameter does not exist
Failed to read sysctl parameter 'net.core.busy_poll', the parameter does not exist
Failed to read sysctl parameter 'net.core.busy_read', the parameter does not exist
===
* It is a quick PoC attempt, shall need more work.
--- Additional comment from Prasad J Pandit on 2021-04-30 12:35:36 UTC ---
(In reply to Prasad J Pandit from comment #1)
> 1) $ tuned-adm(8) verify
> - Does not seem to validate all configuration settings done by an active profile.
> Ex. while verifying 'realtime' profiles, tuned(8) calls respective
> 'script.sh verify' function via 'plugin_script.py' plugin.
> ===
Raised a PR for this change
-> https://github.com/redhat-performance/tuned/pull/341
@Jaroslav: will use this bug for this change alone. And open another Bug+PR for issue 2) above.
Hope that's okay.
Thank you.
--- Additional comment from Jaroslav Škarvada on 2021-04-30 18:13:55 UTC ---
(In reply to Prasad J Pandit from comment #4)
> (In reply to Prasad J Pandit from comment #1)
> > 1) $ tuned-adm(8) verify
> > - Does not seem to validate all configuration settings done by an active profile.
> > Ex. while verifying 'realtime' profiles, tuned(8) calls respective
> > 'script.sh verify' function via 'plugin_script.py' plugin.
> > ===
>
> Raised a PR for this change
> -> https://github.com/redhat-performance/tuned/pull/341
>
>
> @Jaroslav: will use this bug for this change alone. And open another Bug+PR
> for issue 2) above.
> Hope that's okay.
>
> Thank you.
Thanks for PR, comments in the PR.
--- Additional comment from Prasad J Pandit on 2021-05-03 10:07:10 UTC ---
(In reply to Jaroslav Škarvada from comment #5)
> Thanks for PR, comments in the PR.
Updated the PR with the suggested change to check for old parameter value.
Thank you.
--- Additional comment from Prasad J Pandit on 2021-05-13 08:24:47 UTC ---
Revised PR#345: RHBZ#1947858 profiles: improve realtime verify functions
-> https://github.com/redhat-performance/tuned/pull/345
--- Additional comment from Prasad J Pandit on 2021-05-17 05:55:26 UTC ---
(In reply to Prasad J Pandit from comment #7)
>> https://github.com/redhat-performance/tuned/pull/345
Merged upstream:
-> https://github.com/redhat-performance/tuned/commit/6407ca44c6c83e66e5ce3b9ba921437d1787b59d
--- Additional comment from Luiz Capitulino on 2021-05-19 00:57:40 UTC ---
Jaroslav,
We need this for 8.5 for sure. But since the intent of this work is to improve debugability for real-time KVM, I'd also like to request a backport for 8.4.z and even to 8.2.z if possible because those two releases are TUS (extended lifecycle support for telco). Can we do that?
--- Additional comment from Jaroslav Škarvada on 2021-05-19 11:45:49 UTC ---
(In reply to Luiz Capitulino from comment #9)
> Jaroslav,
>
> We need this for 8.5 for sure. But since the intent of this work is to
> improve debugability for real-time KVM, I'd also like to request a backport
> for 8.4.z and even to 8.2.z if possible because those two releases are TUS
> (extended lifecycle support for telco). Can we do that?
I added RPL request for 8.5, despite of it the #345 (comment 7) should get in through the planned 8.5 TuneD rebase (it is already merged upstream). It's bugfix, so I think backport to z-stream shouldn't be problem.