Bug 2117327

After doing some investigation I found 4 issues with rteval and isolcpus:

- cyclictest excludes isolated CPUs by default, since it excludes cores that are not included in the CPU affinity mask of its process, unless -a is used with a CPU list argument (passed by rteval's --measurement-cpulist). This does not reflect the typical use case of isolated CPUs being used for critical tasks here represented by cyclictest.
- loads use the number of online CPUs, including isolated CPUs, to compute the load size (e.g. number of make jobs for kcompile), while they do not get scheduled on isolated CPUs by default. This is the wanted behavior, since they represent non-critical tasks, but the load might be too high.
- loads do not execute properly on isolated CPUs when specifying multiple CPUs in --loads-cpulist. This is caused by the behavior of the scheduler, which won't migrate threads either from, to, or among isolated CPUs (see https://access.redhat.com/solutions/480473); therefore for the loads to execute properly, their tasks would have to be bound to a single CPU per thread in this case (cyclictest already does this).
- reports mention loads and measurements being run on all CPUs including isolated ones even in the default case where the process CPU affinity mask (excluding isolated CPUs) is used. This can cause misleading results, since it reports the same number of CPUs as when --loads-cpulist and --measurement-cpulist is used to specify all CPUs (in which case the isolated CPUs *are* used).

None of these are RHEL-specific and all can be resolved upstream. Possible fixes:

- default to all CPUs including isolated ones for cyclictest measurement, i.e. pass `-a 0-<ncpus>` to it by default instead of just `-a`.
- exclude isolated CPUs from load size calculations.
- when including isolated CPUs in --loads-cpulist, pick jobs to bind specifically to individual isolated CPUs from the set.
- report isolated CPUs properly, including information on if they are used or not by the current run on rteval.

(In reply to Tomáš Glozar from comment #1)
> After doing some investigation I found 4 issues with rteval and isolcpus:
> 
> - cyclictest excludes isolated CPUs by default, since it excludes cores that
> are not included in the CPU affinity mask of its process, unless -a is used
> with a CPU list argument (passed by rteval's --measurement-cpulist). This
> does not reflect the typical use case of isolated CPUs being used for
> critical tasks here represented by cyclictest.

I think this is okay, the default should be to not run on an isolated cpu unless the
user explicitly asks to do so.

> - loads use the number of online CPUs, including isolated CPUs, to compute
> the load size (e.g. number of make jobs for kcompile), while they do not get
> scheduled on isolated CPUs by default. This is the wanted behavior, since
> they represent non-critical tasks, but the load might be too high.

That's a good point.

> - loads do not execute properly on isolated CPUs when specifying multiple
> CPUs in --loads-cpulist. This is caused by the behavior of the scheduler,
> which won't migrate threads either from, to, or among isolated CPUs (see
> https://access.redhat.com/solutions/480473); therefore for the loads to
> execute properly, their tasks would have to be bound to a single CPU per
> thread in this case (cyclictest already does this).
> - reports mention loads and measurements being run on all CPUs including
> isolated ones even in the default case where the process CPU affinity mask
> (excluding isolated CPUs) is used. This can cause misleading results, since
> it reports the same number of CPUs as when --loads-cpulist and
> --measurement-cpulist is used to specify all CPUs (in which case the
> isolated CPUs *are* used).
> 
> None of these are RHEL-specific and all can be resolved upstream. Possible
> fixes:

Right, this is the standard way in which we work anyway, you have to push
a fix upstream before you are allowed to put it in RHEL
> 
> - default to all CPUs including isolated ones for cyclictest measurement,
> i.e. pass `-a 0-<ncpus>` to it by default instead of just `-a`.
> - exclude isolated CPUs from load size calculations.
> - when including isolated CPUs in --loads-cpulist, pick jobs to bind
> specifically to individual isolated CPUs from the set.
> - report isolated CPUs properly, including information on if they are used
> or not by the current run on rteval.

Okay, please discuss approaches with me offline before implementing anything, thanks.

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.