Bug 1225135

tuned: added support for built-in functions
   
Built-in functions has plugin architecture. They are defined in tuned/profiles/functions. Each function (plugin) is python code named function_NAME.py, where NAME is the name of the built-in function defined. It is easy to implement own functions, for example see provided functions.
    
Functions are expanded in tuned profile configuration file during profile   load. Functions are expanded after variables. The syntax for function call is  ${f:NAME:ARG1:ARG2...}. The function NAME will be called and it's result    substituted. It is possible to escape $ by \$ and : by \:. Functions are     loaded and executed on demand. Function which is once loaded during profile    application is not reloaded on multiple executions. Variables in profile arguments are supported, but function calls aren't.
    
The following functions have been implemented so far:
    
exec:ARG1:...:ARGN        - execute external command with arguments and subtitute its output.

kb2s:ARG                  - converts kbytes specified as ARG to sectors.

s2kb:ARG                  - converts sectors specified as ARG to kbytes.

cpulist2hex:ARG1:...:ARGN - converts CPU list to hexadecimal mask, takes arbitrary number of arguments, each argument can also contain "compact values", e.g.: "${f:cpulist2hex:0-3,4:5-6}"
    
hex2cpulist:ARG1 - converts hexadecimal mask to CPU list, takes one argument, the hexadecimal mask.
    
cpus_online:ARG1:...:ARGN - checks whether CPUs from the list (which is formatted the same way as with cpulist2hex) are online, returns only those which are online.
     
Example:
    
[variables]
cmd = date
curr_date = ${f:exec:${cmd}}

[variables]
cpus = ${f:hex2cpulist:0x0000001f}
online = ${f:cpus_online:${cpus}}

You cannot call function from function, but it is possible to workaround it by "helper" variable. This is for simplification of the code, otherwise it would require stack, and stack depth checking, etc.

Available in snapshot 20150605.

For the 20150608 nightly also made available the cpulist_unpack built-in function. The implementation was there since 20150605 in internal API.

Usage:

[variables]
cpus = ${f:cpulist_unpack:1,2,3-5}

The 'cpus' will be set to '1,2,3,4,5'. Also the built-in function can take
arbitrary number of arguments which will be concatenated, e.g.:

[variables]
cpus = ${f:cpulist_unpack:1-3,5:6-7,9}

This means two arguments to 'cpulist_unpack' function. The first argument: '1-3,5' and the second '6-7,9'. The result: '1,2,3,5,6,7,9'.

functions: added 'cpulist_invert' built-in function

Inverts list of CPUs (makes its complement). For the complement it
gets number of present CPUs from the /sys/devices/system/cpu/present,
e.g. system with 4 CPUs (0-3):

[variables]
cpus = ${f:cpulist_invert:1-3,5:6-7,9}

The 'cpus' will be set to '0'

Renamed 'cpus_online' built in function to 'cpulist_online' to be consistent.

commit dfdbd88e679b4735d1a62ac75114ec1c92e5c38a

functions: added autodetection of format to cpulist_unpack

Now CPUs starting with 0x are taken as hexmask. As cpulist_unpack is used as preprocessor of CPU list for other built-in functions this also works with other functions (e.g. cpulist_invert).

Example:
[variables]
cpus = ${f:cpulist_invert:1-2,0x8}

On four CPU machines (CPUs 0-3) it expands to '0'.

(In reply to Jaroslav Škarvada from comment #7)
> commit dfdbd88e679b4735d1a62ac75114ec1c92e5c38a
> 
> functions: added autodetection of format to cpulist_unpack
> 
> Now CPUs starting with 0x are taken as hexmask. As cpulist_unpack is used as
> preprocessor of CPU list for other built-in functions this also works with
> other functions (e.g. cpulist_invert).
> 
> Example:
> [variables]
> cpus = ${f:cpulist_invert:1-2,0x8}
> 
> On four CPU machines (CPUs 0-3) it expands to '0'.

One thing to be aware of when dealing with proc/sys bitmasks is that on systems with more than 32-cores you get comma-separated masks. E.g. on a 64-core system, the all-cpu mask is:  ffffffff,ffffffff. The cores are broken up into 32-core sections. 

If you haven't already dealt with that, I have some code you can start with. If you have dealt with it, then Never Mind(tm). :)

(In reply to Clark Williams from comment #8)
> (In reply to Jaroslav Škarvada from comment #7)
> > commit dfdbd88e679b4735d1a62ac75114ec1c92e5c38a
> > 
> > functions: added autodetection of format to cpulist_unpack
> > 
> > Now CPUs starting with 0x are taken as hexmask. As cpulist_unpack is used as
> > preprocessor of CPU list for other built-in functions this also works with
> > other functions (e.g. cpulist_invert).
> > 
> > Example:
> > [variables]
> > cpus = ${f:cpulist_invert:1-2,0x8}
> > 
> > On four CPU machines (CPUs 0-3) it expands to '0'.
> 
> One thing to be aware of when dealing with proc/sys bitmasks is that on
> systems with more than 32-cores you get comma-separated masks. E.g. on a
> 64-core system, the all-cpu mask is:  ffffffff,ffffffff. The cores are
> broken up into 32-core sections. 
> 
> If you haven't already dealt with that, I have some code you can start with.
> If you have dealt with it, then Never Mind(tm). :)

Will be fixed in 20150609.

(In reply to Jaroslav Škarvada from comment #9)
> (In reply to Clark Williams from comment #8)
> > (In reply to Jaroslav Škarvada from comment #7)
> > > commit dfdbd88e679b4735d1a62ac75114ec1c92e5c38a
> > > 
> > > functions: added autodetection of format to cpulist_unpack
> > > 
> > > Now CPUs starting with 0x are taken as hexmask. As cpulist_unpack is used as
> > > preprocessor of CPU list for other built-in functions this also works with
> > > other functions (e.g. cpulist_invert).
> > > 
> > > Example:
> > > [variables]
> > > cpus = ${f:cpulist_invert:1-2,0x8}
> > > 
> > > On four CPU machines (CPUs 0-3) it expands to '0'.
> > 
> > One thing to be aware of when dealing with proc/sys bitmasks is that on
> > systems with more than 32-cores you get comma-separated masks. E.g. on a
> > 64-core system, the all-cpu mask is:  ffffffff,ffffffff. The cores are
> > broken up into 32-core sections. 
> > 
> > If you haven't already dealt with that, I have some code you can start with.
> > If you have dealt with it, then Never Mind(tm). :)
> 
> Will be fixed in 20150609.

The code supported more than 32 cores, but didn't support comma separator. Currently in 20150610 it works the following way:

It's possible to combine CPU lists with hexmasks, but hexmask needs to be prefixed by "0x" (hex2cpulist function doesn't need the prefix). Hexmasks can include commas, but they are not mandatory. To separate hexmask from CPU list use double comma ',,' (there can be whitespaces between commas).

Example:
[variables]
cpus = ${f:cpulist_unpack:1-2,0x8,ffffffff,,37-39}

It's also possible to separate hexmasks and CPU list by specifying CPU list as another argument (arguments are separated by colons ':'), e.g. the following also works:

[variables]
cpus = ${f:cpulist_unpack:1-2,0x8,ffffffff:37-39}

Or separate it all by commas:

[variables]
cpus = ${f:cpulist_unpack:1-2:0x8,ffffffff:37-39}

Or without commas in hexmask:

[variables]
cpus = ${f:cpulist_unpack:1-2:0x8ffffffff:37-39}

Or any combination of the above.

(In reply to Jaroslav Škarvada from comment #11)
> Or separate it all by commas:
> 
> [variables]
> cpus = ${f:cpulist_unpack:1-2:0x8,ffffffff:37-39}
> 

Typo, should be: separate it all by colons

functions: added 'cpulist2hex_invert' built-in function

It converts CPU list to hexadecimal CPU mask and inverts it.

Example:

[variables]
cpus = ${f:cpulist2hex_invert:1-2}

On machine with four CPUs it returns 00000009.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2375.html