Bug 1714595

Summary: BAD user experience, ERROR when profile is loaded
Product: Red Hat Enterprise Linux 7 Reporter: Robin Hack <rhack>
Component: tunedAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED ERRATA QA Contact: Robin Hack <rhack>
Severity: unspecified Docs Contact: Marie Hornickova <mdolezel>
Priority: unspecified    
Version: 7.7CC: jeder, jskarvad, lkuprova, olysonek, ovasik, pasik, psklenar
Target Milestone: rcKeywords: Patch, Regression, Upstream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tuned-2.11.0-5.el7 Doc Type: Bug Fix
Doc Text:
.Error messages in `/var/log/tuned/tuned.log` referring to non-existent sysctl settings no longer occur when a `Tuned` profile is loaded Previously, the `Tuned` daemon treated non-existent sysctl settings as an error. For example `net.bridge.bridge-nf-call-ip6tables`, `net.bridge.bridge-nf-call-iptables`, or `net.bridge.bridge-nf-call-arptables`, which are unavailable on some systems, could trigger error in the `/var/log/tuned/tuned.log` file: ---- Failed to set sysctl parameter 'net.bridge.bridge-nf-call-ip6tables' to '0', the parameter does not exist ---- With this update, `Tuned` has been fixed, and the error messages no longer occur within `/var/log/tuned/tuned.log` under the described circumstances.
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-06 13:04:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1709724    

Description Robin Hack 2019-05-28 12:03:15 UTC
Description of problem:

When profile is applied, then ERROR appears in logs:
2019-05-17 10:44:17,410 ERROR    tuned.plugins.plugin_sysctl: Failed to set sysctl parameter 'net.bridge.bridge-nf-call-ip6tables' to '0', the parameter does not exist
2019-05-17 10:44:17,410 ERROR    tuned.plugins.plugin_sysctl: Failed to set sysctl parameter 'net.bridge.bridge-nf-call-iptables' to '0', the parameter does not exist
2019-05-17 10:44:17,410 ERROR    tuned.plugins.plugin_sysctl: Failed to set sysctl parameter 'net.bridge.bridge-nf-call-arptables' to '0', the parameter does not exist

Version-Release number of selected component (if applicable):
tuned-2.11.0-3.el7.noarch

How reproducible:
always

Steps to Reproduce:
(for example - basically each profile which sets sysctl can be used)
1. tuned-adm profile virtual-guest
2.
3.

Actual results:
ERROR message in log

Expected results:
maybe something much user friendly... like no ERROR message?

Additional info:
Tuned insides:
content of tuned/plugins/plugin_sysctl.py

SYSCTL_CONFIG_DIRS = [ "/run/sysctl.d",
                "/etc/sysctl.d",
                "/usr/local/lib/sysctl.d",
                "/usr/lib/sysctl.d",
                "/lib/sysctl.d" ]


# cat /usr/lib/sysctl.d/00-system.conf

# Disable netfilter on bridges.
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

then it reads each file and use:
_apply_sysctl_config_line to set sysctl tunable.

Sometimes, this is just false error, because bridge kernel module is missing.
More info:
https://wiki.libvirt.org/page/Net.bridge.bridge-nf-call_and_sysctl.conf

Comment 2 Ondřej Lysoněk 2019-05-28 13:05:41 UTC
Thanks for the report. I think we should preserve the old behaviour in this case and ignore nonexistent sysctl settings from the system locations (/run/sysctl.d, /etc/sysctl.d, etc.).

Comment 3 Ondřej Lysoněk 2019-05-28 13:08:17 UTC
For the record, the errors are completely harmless.

Comment 4 Robin Hack 2019-05-29 11:00:22 UTC
Hello. I agree. This bug is harmless however it breaks my tests which looks for ERROR messages.
And I would like to have my tests sane without workarounds ;P.

Also, it's just very bad user experience. I remember Logging Polish initiative which should solve issues like this.
Customers aren't happy with false ERRORs.

I agree that ignoring missing keys or broken lines are solution. As we talked with olysonek, previous, not so nice, solution uses same approach (just ignore missing keys).

I will not provide ack for now (rhel7.7) but we can release this fix later :).

Comment 5 Robin Hack 2019-05-29 12:08:27 UTC
BTW: it's not even issue with initscripts :).

root@ci-vm-10-0-136-236 ~]# modprobe br_netfilter
[root@ci-vm-10-0-136-236 ~]# sysctl -a  | grep bridge
net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

Just not kernel module loaded when tuned starts which is normal.

Comment 6 Ondřej Lysoněk 2019-05-29 14:22:36 UTC
In the new implementation, there are a couple of other newly added error messages that could get printed:
1. Syntax error in one of the sysctl config files [1] [2], attempting to set a deprecated sysctl option [4]
   * 'sysctl --system' reports errors about these problems, however the old Tuned doesn't pass the errors to the log
2. Attempting to set a multi-line sysctl option, which Tuned doesn't support [3]
   * The old Tuned doesn't say anything about these in the log. The old Tuned naively sets and restores these options, but it doesn't work.

[1] https://github.com/redhat-performance/tuned/blob/196131584488a2cdaaecb74e14d14f6a89004f54/tuned/plugins/plugin_sysctl.py#L126
[2] https://github.com/redhat-performance/tuned/blob/196131584488a2cdaaecb74e14d14f6a89004f54/tuned/plugins/plugin_sysctl.py#L132
[3] https://github.com/redhat-performance/tuned/blob/196131584488a2cdaaecb74e14d14f6a89004f54/tuned/plugins/plugin_sysctl.py#L148
[4] https://github.com/redhat-performance/tuned/blob/196131584488a2cdaaecb74e14d14f6a89004f54/tuned/plugins/plugin_sysctl.py#L166

Jardo, Robine, what do you think we should do about these error messages? Personally, I would keep printing them (maybe with the exception of the multi-line options - Tuned could just ignore these if they are in the sysctl config files) - the error messages could reveal real problems.

I hate having to make silly decisions like these, but here we are...

Comment 7 Ondřej Lysoněk 2019-05-29 14:23:04 UTC
See comment#6.

Comment 8 Ondřej Lysoněk 2019-05-29 15:35:01 UTC
Upstream pull request:
https://github.com/redhat-performance/tuned/pull/189

Comment 22 errata-xmlrpc 2019-08-06 13:04:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2201