RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1249618 - Add kernel module tuning support to tuned
Summary: Add kernel module tuning support to tuned
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tuned
Version: 7.3
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Jaroslav Škarvada
QA Contact: Tereza Cerna
Jiri Herrmann
URL:
Whiteboard:
Depends On:
Blocks: kvm-rt-tuned
TreeView+ depends on / blocked
 
Reported: 2015-08-03 12:14 UTC by Jeremy Eder
Modified: 2016-11-04 07:23 UTC (History)
7 users (show)

Fixed In Version: tuned-2.7.0-1.el7
Doc Type: Release Note
Doc Text:
The new *modules* Tuned plug-in The *modules* plug-in allows Tuned to load and reload kernel modules with parameters specified in the the settings of the Tuned profiles.
Clone Of:
Environment:
Last Closed: 2016-11-04 07:23:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:2479 0 normal SHIPPED_LIVE tuned bug fix and enhancement update 2016-11-03 14:08:42 UTC

Description Jeremy Eder 2015-08-03 12:14:50 UTC
Description of problem:

Tuned should support adjusting kernel module options.

The NFV effort is currently adjusting the kvm kernel module in a script:

realtime-virtual-host/script.sh:        echo "options kvm kvmclock_periodic_sync=0" > /etc/modprobe.d/kvm.rt.tuned.conf
realtime-virtual-host/script.sh:        echo "options kvm_intel ple_gap=0" >> /etc/modprobe.d/kvm.rt.tuned.conf

The script also manages:
- creation/deletion of the modprobe configuration file
- testing for kernel module loaded/not loaded
- testing for kernel module support for certain options
- start/stop/verify functions

In addition, the NFV effort may support SR-IOV configurations, which requires adjusting kernel modules parameters as well.  The ideal integration might look like OpenStack/NFV product adding 2 variables to /etc/tuned/realtime-virtual-host-variables.conf:

- PCI ID#
- Number of Virtual Functions to create (this is mostly a sysfs, but could be a module param as well)

Comment 1 Jaroslav Škarvada 2016-07-04 17:22:44 UTC
Upstream commit adding the requested functionality:
https://git.fedorahosted.org/cgit/tuned.git/commit/?id=6793c2abff1b96f8e1d066be34d6fcff4eef77c5

Added new plugin called 'modules'. This plugin can set parameters of kernel modules. It generates /etc/modprobe.d/tuned.conf file.
    
Usage in tuned.conf:
    
[modules]
MODULE=OPTION1=VALUE1 OPTION2=VALUE2...
   
E.g:
    
[modules]
netrom=nr_ndevs=2
    
This sets module 'netrom' to have 'nr_ndevs' option set to '2'.

Unfortunately modules may not expose all their parameters through sysfs interface (/sys/module/MODULENAME/parameters), thus the verification is forced to run with ignore_missing = True. Module parameters which are not exposed through the sysfs interface are skipped during verification. Currently I don't know better way how to check parameters of loaded modules.

The module can be also forced to load/reload by using special '+r' prefix, e.g.:
    
[modules]
netrom=+r nr_ndevs=2
    
Will also cause that Tuned will try to remove 'netrom' module (if loaded) and
will try to (re)insert it with the specified parameters. The '+r' can be followed by an optional comma (','), i.e.: '+r,' for better readability.
    
By using '+r' the module will be loaded even if not needed.

Available for testing in build tuned-2.6.0-1.20160704git6793c2ab.

Comment 2 Jeremy Eder 2016-07-05 12:38:43 UTC
Hi,

modinfo could be used to find if there are any parameters that are not exposed in sysfs.

# modinfo -p kvm | sort
halt_poll_ns: (uint)
ignore_msrs: (bool)
kvmclock_periodic_sync: (bool)
lapic_timer_advance_ns: (uint)
min_timer_period_us: (uint)
tsc_tolerance_ppm: (uint)

# find /sys/module/kvm/parameters -type f | sort                                                                                                       
/sys/module/kvm/parameters/halt_poll_ns
/sys/module/kvm/parameters/ignore_msrs
/sys/module/kvm/parameters/kvmclock_periodic_sync
/sys/module/kvm/parameters/lapic_timer_advance_ns
/sys/module/kvm/parameters/min_timer_period_us
/sys/module/kvm/parameters/mmu_audit
/sys/module/kvm/parameters/tsc_tolerance_ppm


Actually, mmu_audit doesn't show up in modinfo...weird.

Comment 3 Jaroslav Škarvada 2016-07-07 07:57:55 UTC
(In reply to Jeremy Eder from comment #2)
It's for 'tuned-adm verify', it's checking whether the modules are loaded with the requested parameters. The 'modinfo -p' shows which parameters the module accept, not the values with which the module is currently loaded in. I am unaware of the way how to reliable find out this (tips are welcome). So currently the plugin consults /sys/module/*/parameters to check whether the requested parameter value matches the value the module is currently loaded in. If the parameter is not found in the /sys/module/*/parameters, its verification is skipped.

Comment 4 Jeremy Eder 2016-07-07 11:41:32 UTC
Hmm, I think /sys/module/*/parameters is right.  I just tried a quick test anyway, and that does reflect a change I passed in via modprobe.



# modprobe -r kvm_intel
# modprobe -r kvm
# modprobe kvm
# cat /sys/module/kvm/parameters/kvmclock_periodic_sync 
Y
# systool -m kvm -av                                                                                                                   
Module = "kvm"

  Attributes:
    coresize            = "525409"
    initsize            = "0"
    initstate           = "live"
    refcnt              = "0"
    rhelversion         = "7.2"
    srcversion          = "EBE7F50365DED1DC12FEE5C"
    taint               = ""
    uevent              = <store method only>

  Parameters:
    halt_poll_ns        = "0"
    ignore_msrs         = "N"
    kvmclock_periodic_sync= "Y" <<<------------ set to Y by default
    lapic_timer_advance_ns= "0"
    min_timer_period_us = "500"
    mmu_audit           = "N"
    tsc_tolerance_ppm   = "250"
<snip>

# modprobe -r kvm_intel kvm
# modprobe kvm kvmclock_periodic_sync=0
# cat /sys/module/kvm/parameters/kvmclock_periodic_sync 
N
# systool -m kvm -av                                                                                                                   
Module = "kvm"

  Attributes:
    coresize            = "525409"
    initsize            = "0"
    initstate           = "live"
    refcnt              = "0"
    rhelversion         = "7.2"
    srcversion          = "EBE7F50365DED1DC12FEE5C"
    taint               = ""
    uevent              = <store method only>

  Parameters:
    halt_poll_ns        = "0"
    ignore_msrs         = "N"
    kvmclock_periodic_sync= "N"  <<<------------ now you can see the effect of the module option I passed in via modprobe
    lapic_timer_advance_ns= "0"
    min_timer_period_us = "500"
    mmu_audit           = "N"
    tsc_tolerance_ppm   = "250"
<snip>

Comment 5 Jaroslav Škarvada 2016-07-07 11:57:50 UTC
It depends on the module code, you can specify attribs in the kernel parameters macro and it's possible not to expose the module parameter through the sysfs by it, e.g. module netrom, parameter nr_ndevs, but there are other examples as well.

Comment 10 Tereza Cerna 2016-09-20 12:05:14 UTC
==============================================
Verified in:
    tuned-2.7.1-2.el7.noarch             
    tuned-profiles-realtime-2.7.1-2.el7.noarch
    tuned-profiles-nfv-2.7.1-2.el7.noarch
PASS
==============================================

=== Module reload manually ===

# cat /sys/module/kvm_intel/parameters/nested
N
# cat /etc/tuned/testprofile/tuned.conf 
[modules]
kvm_intel=nested=Y
# tuned-adm profile testprofile
# modprobe -r kvm_intel
# modprobe kvm_intel
# cat /sys/module/kvm_intel/parameters/nested
Y
#

=== Module reload automatically ===

# cat /sys/module/kvm_intel/parameters/nested
N
# cat /etc/tuned/testprofile/tuned.conf 
[modules]
kvm_intel=+r nested=Y
# tuned-adm profile testprofile
# cat /sys/module/kvm_intel/parameters/nested
Y

==============================================
Reproduced in:
    tuned-profiles-nfv-2.5.1-1.el7.noarch
    tuned-profiles-realtime-2.5.1-1.el7.noarch
    tuned-2.5.1-1.el7.noarch
FAIL
==============================================

# echo > /var/log/tuned/tuned.log
# tuned-adm profile testprofile
# cat /var/log/tuned/tuned.log 
2016-09-19 09:30:55,229 ERROR    tuned.units.manager: failed to initialize plugin modules
2016-09-19 09:30:55,229 ERROR    tuned.units.manager: No module named plugin_modules
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/tuned/units/manager.py", line 45, in create
    plugin = self._plugins_repository.create(plugin_name)
  File "/usr/lib/python2.7/site-packages/tuned/plugins/repository.py", line 33, in create
    plugin_cls = self.load_plugin(plugin_name)
  File "/usr/lib/python2.7/site-packages/tuned/utils/plugin_loader.py", line 28, in load_plugin
    return self._get_class(module_name)
  File "/usr/lib/python2.7/site-packages/tuned/utils/plugin_loader.py", line 32, in _get_class
    module = __import__(module_name)
ImportError: No module named plugin_modules

Comment 12 errata-xmlrpc 2016-11-04 07:23:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2479.html


Note You need to log in before you can comment on or make changes to this bug.