Bug 1249618

Summary: Add kernel module tuning support to tuned
Product: Red Hat Enterprise Linux 7 Reporter: Jeremy Eder <jeder>
Component: tunedAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED ERRATA QA Contact: Tereza Cerna <tcerna>
Severity: medium Docs Contact: Jiri Herrmann <jherrman>
Priority: medium    
Version: 7.3CC: jeder, jherrman, jscotka, jskarvad, lcapitulino, mtosatti, tcerna
Target Milestone: rcKeywords: Upstream
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: tuned-2.7.0-1.el7 Doc Type: Release Note
Doc Text:
The new *modules* Tuned plug-in The *modules* plug-in allows Tuned to load and reload kernel modules with parameters specified in the the settings of the Tuned profiles.
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-11-04 07:23:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1240765    

Description Jeremy Eder 2015-08-03 12:14:50 UTC
Description of problem:

Tuned should support adjusting kernel module options.

The NFV effort is currently adjusting the kvm kernel module in a script:

realtime-virtual-host/script.sh:        echo "options kvm kvmclock_periodic_sync=0" > /etc/modprobe.d/kvm.rt.tuned.conf
realtime-virtual-host/script.sh:        echo "options kvm_intel ple_gap=0" >> /etc/modprobe.d/kvm.rt.tuned.conf

The script also manages:
- creation/deletion of the modprobe configuration file
- testing for kernel module loaded/not loaded
- testing for kernel module support for certain options
- start/stop/verify functions

In addition, the NFV effort may support SR-IOV configurations, which requires adjusting kernel modules parameters as well.  The ideal integration might look like OpenStack/NFV product adding 2 variables to /etc/tuned/realtime-virtual-host-variables.conf:

- PCI ID#
- Number of Virtual Functions to create (this is mostly a sysfs, but could be a module param as well)

Comment 1 Jaroslav Škarvada 2016-07-04 17:22:44 UTC
Upstream commit adding the requested functionality:
https://git.fedorahosted.org/cgit/tuned.git/commit/?id=6793c2abff1b96f8e1d066be34d6fcff4eef77c5

Added new plugin called 'modules'. This plugin can set parameters of kernel modules. It generates /etc/modprobe.d/tuned.conf file.
    
Usage in tuned.conf:
    
[modules]
MODULE=OPTION1=VALUE1 OPTION2=VALUE2...
   
E.g:
    
[modules]
netrom=nr_ndevs=2
    
This sets module 'netrom' to have 'nr_ndevs' option set to '2'.

Unfortunately modules may not expose all their parameters through sysfs interface (/sys/module/MODULENAME/parameters), thus the verification is forced to run with ignore_missing = True. Module parameters which are not exposed through the sysfs interface are skipped during verification. Currently I don't know better way how to check parameters of loaded modules.

The module can be also forced to load/reload by using special '+r' prefix, e.g.:
    
[modules]
netrom=+r nr_ndevs=2
    
Will also cause that Tuned will try to remove 'netrom' module (if loaded) and
will try to (re)insert it with the specified parameters. The '+r' can be followed by an optional comma (','), i.e.: '+r,' for better readability.
    
By using '+r' the module will be loaded even if not needed.

Available for testing in build tuned-2.6.0-1.20160704git6793c2ab.

Comment 2 Jeremy Eder 2016-07-05 12:38:43 UTC
Hi,

modinfo could be used to find if there are any parameters that are not exposed in sysfs.

# modinfo -p kvm | sort
halt_poll_ns: (uint)
ignore_msrs: (bool)
kvmclock_periodic_sync: (bool)
lapic_timer_advance_ns: (uint)
min_timer_period_us: (uint)
tsc_tolerance_ppm: (uint)

# find /sys/module/kvm/parameters -type f | sort                                                                                                       
/sys/module/kvm/parameters/halt_poll_ns
/sys/module/kvm/parameters/ignore_msrs
/sys/module/kvm/parameters/kvmclock_periodic_sync
/sys/module/kvm/parameters/lapic_timer_advance_ns
/sys/module/kvm/parameters/min_timer_period_us
/sys/module/kvm/parameters/mmu_audit
/sys/module/kvm/parameters/tsc_tolerance_ppm


Actually, mmu_audit doesn't show up in modinfo...weird.

Comment 3 Jaroslav Škarvada 2016-07-07 07:57:55 UTC
(In reply to Jeremy Eder from comment #2)
It's for 'tuned-adm verify', it's checking whether the modules are loaded with the requested parameters. The 'modinfo -p' shows which parameters the module accept, not the values with which the module is currently loaded in. I am unaware of the way how to reliable find out this (tips are welcome). So currently the plugin consults /sys/module/*/parameters to check whether the requested parameter value matches the value the module is currently loaded in. If the parameter is not found in the /sys/module/*/parameters, its verification is skipped.

Comment 4 Jeremy Eder 2016-07-07 11:41:32 UTC
Hmm, I think /sys/module/*/parameters is right.  I just tried a quick test anyway, and that does reflect a change I passed in via modprobe.



# modprobe -r kvm_intel
# modprobe -r kvm
# modprobe kvm
# cat /sys/module/kvm/parameters/kvmclock_periodic_sync 
Y
# systool -m kvm -av                                                                                                                   
Module = "kvm"

  Attributes:
    coresize            = "525409"
    initsize            = "0"
    initstate           = "live"
    refcnt              = "0"
    rhelversion         = "7.2"
    srcversion          = "EBE7F50365DED1DC12FEE5C"
    taint               = ""
    uevent              = <store method only>

  Parameters:
    halt_poll_ns        = "0"
    ignore_msrs         = "N"
    kvmclock_periodic_sync= "Y" <<<------------ set to Y by default
    lapic_timer_advance_ns= "0"
    min_timer_period_us = "500"
    mmu_audit           = "N"
    tsc_tolerance_ppm   = "250"
<snip>

# modprobe -r kvm_intel kvm
# modprobe kvm kvmclock_periodic_sync=0
# cat /sys/module/kvm/parameters/kvmclock_periodic_sync 
N
# systool -m kvm -av                                                                                                                   
Module = "kvm"

  Attributes:
    coresize            = "525409"
    initsize            = "0"
    initstate           = "live"
    refcnt              = "0"
    rhelversion         = "7.2"
    srcversion          = "EBE7F50365DED1DC12FEE5C"
    taint               = ""
    uevent              = <store method only>

  Parameters:
    halt_poll_ns        = "0"
    ignore_msrs         = "N"
    kvmclock_periodic_sync= "N"  <<<------------ now you can see the effect of the module option I passed in via modprobe
    lapic_timer_advance_ns= "0"
    min_timer_period_us = "500"
    mmu_audit           = "N"
    tsc_tolerance_ppm   = "250"
<snip>

Comment 5 Jaroslav Škarvada 2016-07-07 11:57:50 UTC
It depends on the module code, you can specify attribs in the kernel parameters macro and it's possible not to expose the module parameter through the sysfs by it, e.g. module netrom, parameter nr_ndevs, but there are other examples as well.

Comment 10 Tereza Cerna 2016-09-20 12:05:14 UTC
==============================================
Verified in:
    tuned-2.7.1-2.el7.noarch             
    tuned-profiles-realtime-2.7.1-2.el7.noarch
    tuned-profiles-nfv-2.7.1-2.el7.noarch
PASS
==============================================

=== Module reload manually ===

# cat /sys/module/kvm_intel/parameters/nested
N
# cat /etc/tuned/testprofile/tuned.conf 
[modules]
kvm_intel=nested=Y
# tuned-adm profile testprofile
# modprobe -r kvm_intel
# modprobe kvm_intel
# cat /sys/module/kvm_intel/parameters/nested
Y
#

=== Module reload automatically ===

# cat /sys/module/kvm_intel/parameters/nested
N
# cat /etc/tuned/testprofile/tuned.conf 
[modules]
kvm_intel=+r nested=Y
# tuned-adm profile testprofile
# cat /sys/module/kvm_intel/parameters/nested
Y

==============================================
Reproduced in:
    tuned-profiles-nfv-2.5.1-1.el7.noarch
    tuned-profiles-realtime-2.5.1-1.el7.noarch
    tuned-2.5.1-1.el7.noarch
FAIL
==============================================

# echo > /var/log/tuned/tuned.log
# tuned-adm profile testprofile
# cat /var/log/tuned/tuned.log 
2016-09-19 09:30:55,229 ERROR    tuned.units.manager: failed to initialize plugin modules
2016-09-19 09:30:55,229 ERROR    tuned.units.manager: No module named plugin_modules
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tuned/units/manager.py", line 45, in create
    plugin = self._plugins_repository.create(plugin_name)
  File "/usr/lib/python2.7/site-packages/tuned/plugins/repository.py", line 33, in create
    plugin_cls = self.load_plugin(plugin_name)
  File "/usr/lib/python2.7/site-packages/tuned/utils/plugin_loader.py", line 28, in load_plugin
    return self._get_class(module_name)
  File "/usr/lib/python2.7/site-packages/tuned/utils/plugin_loader.py", line 32, in _get_class
    module = __import__(module_name)
ImportError: No module named plugin_modules

Comment 12 errata-xmlrpc 2016-11-04 07:23:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2479.html