Bug 1432240 - tuned failed to activate profile with certain core list
tuned failed to activate profile with certain core list
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: tuned (Show other bugs)
7.4
Unspecified Unspecified
unspecified Severity urgent
: rc
: ---
Assigned To: Jaroslav Škarvada
Tereza Cerna
: Patch, Upstream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-03-14 17:06 EDT by jianzzha
Modified: 2017-08-01 08:35 EDT (History)
4 users (show)

See Also:
Fixed In Version: tuned-2.8.0-1.el7
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1434360 (view as bug list)
Environment:
Last Closed: 2017-08-01 08:35:21 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description jianzzha 2017-03-14 17:06:08 EDT
Description of problem:
on a system with 2 processors (24 cores), tuned failed to activate  cpu-partitioning profile if the configured core-list are even cores (2,4,6,8). On the same system, if the configured core-list are odd cores (1,3,5,7), tuned was able to activate the cpu-partitioning profile.

Version-Release number of selected component 
kernel version: 3.10.0-514.2.2

How reproducible:
with odd-core list:
[root@controller-0 tuned]# cat cpu-partitioning-variables.conf
# Examples:
# isolated_cores=1,3,5,7
# isolated_cores=1,3,5,7
#
isolated_cores=1,3,5,7

[root@controller-0 tuned]# tuned-adm  profile cpu-partitioning
[root@controller-0 tuned]#

with even-core list:
[root@controller-0 tuned]# cat cpu-partitioning-variables.conf
# Examples:
# isolated_cores=2,4,6,8
# isolated_cores=2,4,6,8
#
isolated_cores=2,4,6,8

[root@controller-0 tuned]# tuned-adm  profile cpu-partitioning
Cannot load profile 'cpu-partitioning': Assertion 'isolated_cores contains present CPU(s)' failed.
[root@controller-0 tuned]#


Actual results:


Expected results:


Additional info:
[root@controller-0 tuned]# cat /proc/cpuinfo | grep processor
processor	: 0
processor	: 1
processor	: 2
processor	: 3
processor	: 4
processor	: 5
processor	: 6
processor	: 7
processor	: 8
processor	: 9
processor	: 10
processor	: 11
processor	: 12
processor	: 13
processor	: 14
processor	: 15
processor	: 16
processor	: 17
processor	: 18
processor	: 19
processor	: 20
processor	: 21
processor	: 22
processor	: 23
Comment 2 jianzzha 2017-03-14 17:41:45 EDT
further tests indicates this seems to happen when the core list across 8, 16.

for example "7,8" will have problem, while "8,9" is fine
Comment 3 Chen 2017-03-20 10:20:26 EDT
Hi team,

Sorry for raising the severity. This bug is preventing serveral huawei NFV PoC projects.

# cat /etc/tuned/cpu-partitioning-variables.conf 
# Examples:
# isolated_cores=2,4-7
# isolated_cores=2-23
#
isolated_cores=2,3,11

# tuned-adm profile cpu-partitioning
Cannot load profile 'cpu-partitioning': Assertion 'isolated_cores contains present CPU(s)' failed.

Snip of /var/log/tuned/tuned.log:

2017-03-20 10:15:18,321 INFO     tuned.profiles.loader: loading profile: cpu-partitioning
2017-03-20 10:15:18,323 ERROR    tuned.profiles.functions.function_assertion: assertion 'isolated_cores contains present CPU(s)' failed: '2,3,11' != '3,2,11'

<Snip>

2017-03-20 10:15:18,897 INFO     tuned.daemon.daemon: static tuning from profile 'cpu-partitioning' applied

By looking at the logs seems the profile has been applied. So is this a harmless warning or a real problem ?

Best Regards,
Chen
Comment 4 Jaroslav Škarvada 2017-03-21 04:43:40 EDT
I think the following upstream commit should resolve the problem:
https://github.com/redhat-performance/tuned/commit/e052fd14a17e7872d0880720b15db2097b58f5a4
Comment 5 Chen 2017-03-21 04:48:05 EDT
Hi Jaroslav,

Thank you for your reply.

I've confirmed that the upstream patch works.

Do you have any idea of the following question ?

>By looking at the logs seems the profile has been applied. So is this a harmless warning or a real problem ?

Best Regards,
Chen
Comment 6 Jaroslav Škarvada 2017-03-21 05:30:29 EDT
(In reply to Chen from comment #5)
> Hi Jaroslav,
> 
> Thank you for your reply.
> 
> I've confirmed that the upstream patch works.
> 
> Do you have any idea of the following question ?
> 
> >By looking at the logs seems the profile has been applied. So is this a harmless warning or a real problem ?

After the assertion kicks in it should stop loading the profile, but from your log it doesn't seem so. I was unable to reproduce the problem, but it could be different (probably minor) bug or maybe it's already fixed upstream.

What's the Tuned version used?
tuned-2.7.1-5.el7 from FDP?

In such case there is high probability that the problem is already fixed.

The latest scratch-build (unsupported) version for testing is:
tuned*-2.7.1-1.20170321gita50d5e02.el7

available from:
https://jskarvad.fedorapeople.org/tuned/devel/repo/

Or could you lend me the machine for closer observation what's going there?
Comment 9 Jaroslav Škarvada 2017-03-21 07:04:00 EDT
Clonned as bug 1434360.
Comment 12 Tereza Cerna 2017-05-04 10:03:04 EDT
Works as expected. Loading of profile doesn't stop with assertion.
 -> VERIFIED

======================================================
Verified in:
    tuned-2.8.0-2.el7.noarch
    tuned-profiles-cpu-partitioning-2.8.0-2.el7.noarch
PASS
======================================================

# tuned-adm profile balanced
# cat /etc/tuned/cpu-partitioning-variables.conf
isolated_cores=2,3,11
# tuned-adm profile cpu-partitioning
# tuned-adm active
Current active profile: cpu-partitioning
# reboot

# tuned-adm verify
Verfication succeeded, current system settings match the preset profile.
See tuned log file ('/var/log/tuned/tuned.log') for details.

======================================================
Reproduced in:
    tuned-2.7.1-5.20170314git92d558b8.el7.noarch
    tuned-profiles-cpu-partitioning-2.7.1-5.20170314git92d558b8.el7.noarch
FAIL
======================================================

# tuned-adm profile balanced
# cat /etc/tuned/cpu-partitioning-variables.conf 
isolated_cores=2,3,11
# tuned-adm profile cpu-partitioning
Cannot load profile 'cpu-partitioning': Assertion 'isolated_cores contains present CPU(s)' failed.
# tuned-adm active
Current active profile: balanced
# cat /var/log/tuned/tuned.log | grep ERROR
2017-05-04 09:56:12,644 ERROR    tuned.profiles.functions.function_assertion: assertion 'isolated_cores contains present CPU(s)' failed: '2,3,11' != '3,2,11'
Comment 13 errata-xmlrpc 2017-08-01 08:35:21 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2102

Note You need to log in before you can comment on or make changes to this bug.