RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1100826 - Create new plugin for process scheduling/priority tuning
Summary: Create new plugin for process scheduling/priority tuning
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tuned
Version: 7.1
Hardware: All
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Jaroslav Škarvada
QA Contact: qe-baseos-daemons
URL:
Whiteboard:
Depends On:
Blocks: 1148546
TreeView+ depends on / blocked
 
Reported: 2014-05-23 14:16 UTC by Jeremy Eder
Modified: 2016-01-14 14:12 UTC (History)
7 users (show)

Fixed In Version: tuned-2.4.0-0.el7
Doc Type: Enhancement
Doc Text:
Feature: Process scheduling/priority tuning. Reason: See comment 0. Result: For description of the implemented feature and how to use it see comment 3.
Clone Of:
: 1148546 (view as bug list)
Environment:
Last Closed: 2015-03-05 13:29:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0550 0 normal SHIPPED_LIVE tuned bug fix and enhancement update 2015-03-05 16:45:23 UTC

Comment 3 Jaroslav Škarvada 2014-09-30 23:06:47 UTC
Upstream commit adding this feature:
https://git.fedorahosted.org/cgit/tuned.git/commit/?id=bf42cb9ba3c39e1e5aec680f2fdc98c4a51666e5

Currently it is substitution for the rtctl tool, but there are small differences in the syntax (due to tuned specific internals).

Format for the rules is:

group.GROUPNAME=RULE_PRIO:SCHED:PRIO:AFFINITY:REGEX

where RULE_PRIO defines tuned priority of the rule. Rules are sorted according to priority. This is needed for inheritence to be able to reorder previously defined rule. For same RULE_PRIO, rules should be processed in the order there were defined (but this is Python interpreter dependant). To disable inherited rule for GROUPNAME use:

group.GROUPNAME=

SCHED must be one of:
  'f' for FIFO,
  'b' for batch,
  'r' for round robin,
  'o' for other,
  '*' means not to change.

AFFINITY is a hex number, see taskset(1) for details about number of CPUs. The '*' means not to change.

REGEX is Python regex. It must match against output of

ps -eo cmd

Any given process name may match more than one group. Default priority and scheduling policy are taken from the last matching regex.

You need to write 'group.' prefix before your group name. This is because there may be added other parameters to the plugin later.

Example of usage:

[scheduler]
group.kthreads=0:*:1:*:\[.*\]$
group.watchdog=0:f:99:*:\[watchdog.*\]

Comment 4 Jaroslav Škarvada 2014-09-30 23:08:45 UTC
CCing mgrepl, tuned now calls chrt and taskset.

Comment 5 Jaroslav Škarvada 2014-10-01 00:12:34 UTC
Improvements over rtctl tool:
- inheritance - group names can be inherited from ancestor profiles
- automatic revert of settings - on profile unload / change everything what was changed is reverted back to its initial state (before profile load). During the revert both PID and full process name are checked and both need to match for the revert to take action.

Comment 6 Jaroslav Škarvada 2014-10-01 12:55:18 UTC
Regarding tuning of newly created processes I need to think a bit about it. Systemtap / LD_PRELOAD may bring performance penalty (for loads creating many processes - probably not typical loads). Other possibility is to poll e.g. each 10 seconds. There is already infrastructure for this that was used for dynamic tuning (already disabled in RHEL). This may be improved just to only run monitors for processes polling. Anyway this will require some more engineering work, 7.1 is not realistic for it.

Comment 7 Jeremy Eder 2014-10-01 13:21:23 UTC
Hi Jaroslav,

Thanks!  I'm going to play around with this.

About comment #6, newly created processes.  I can say that the polling method will not be accepted (by customers); especially in the context of MRG, but also RHEL as well.

If I remember correctly, Clark had some ideas in this area, but I don't exactly recall what the were.

If a user adds their own tuned profile that uses this plugin, and puts their application thread regexs in there...the problem is that tuned starts before their app, and thus tuned will not see any matching threads to apply any of the tunings, is that correct ?

Comment 8 Jaroslav Škarvada 2014-10-01 13:33:25 UTC
(In reply to Jeremy Eder from comment #7)
> If a user adds their own tuned profile that uses this plugin, and puts their
> application thread regexs in there...the problem is that tuned starts before
> their app, and thus tuned will not see any matching threads to apply any of
> the tunings, is that correct ?

Yes, this is correct. Currently it works similar to rtctl. For tuned to apply tunings for customers user space apps, they would need to restart tuned or switch profile.

Feel free to forward me any ideas/proposals regarding tracing of new processes. I would like to select most suitable / universal approach for the implementation.

Comment 9 Clark Williams 2014-10-01 13:36:14 UTC
(In reply to Jaroslav Škarvada from comment #6)
> Regarding tuning of newly created processes I need to think a bit about it.
> Systemtap / LD_PRELOAD may bring performance penalty (for loads creating
> many processes - probably not typical loads). Other possibility is to poll
> e.g. each 10 seconds. There is already infrastructure for this that was used
> for dynamic tuning (already disabled in RHEL). This may be improved just to
> only run monitors for processes polling. Anyway this will require some more
> engineering work, 7.1 is not realistic for it.


You may want to look at 'tuna', where we get the perf subsystem to send us a thread creation event. Tuna used to poll /proc for new threads but we moved to perf where it exists, since it's much lower overhead. 

I'll see if I can find that code and point you at it.

Comment 10 Jaroslav Škarvada 2014-10-01 13:41:50 UTC
(In reply to Clark Williams from comment #9)
> You may want to look at 'tuna', where we get the perf subsystem to send us a
> thread creation event. Tuna used to poll /proc for new threads but we moved
> to perf where it exists, since it's much lower overhead. 
> 
> I'll see if I can find that code and point you at it.

OK, thanks, NP, I know perf and I saw this code in tuna. So for now this is candidate no. 1.

Comment 14 Jaroslav Škarvada 2014-10-01 17:02:40 UTC
Closing this for 7.1 and cloning as RFE to 7.2 for runtime detection of processes.

Comment 16 Miroslav Grepl 2014-10-02 08:24:54 UTC
(In reply to Jaroslav Škarvada from comment #4)
> CCing mgrepl, tuned now calls chrt and taskset.

Are there SELinux issues?

Comment 19 errata-xmlrpc 2015-03-05 13:29:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0550.html


Note You need to log in before you can comment on or make changes to this bug.