Bug 1452357

Summary: realtime-virtual-guest: not safe to run PMD thread with fifo:1
Product: Red Hat Enterprise Linux 7 Reporter: Luiz Capitulino <lcapitulino>
Component: tunedAssignee: Jaroslav Škarvada <jskarvad>
Status: CLOSED ERRATA QA Contact: Tereza Cerna <tcerna>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.4CC: atragler, hhuang, jeder, jskarvad, juzhang, mtessun, mtosatti, ovasik, pezhang, sgordon, tcerna, thozza
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tuned-2.8.0-5.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 12:35:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1240765, 1438120    

Description Luiz Capitulino 2017-05-18 18:28:05 UTC
Description of problem:

The realtime-virtual-guest profile doesn't allow for running a PMD thread with fifo:1 priority in the guest. This is so because the profile doesn't organizes CPU bound kernel threads into FIFO slots as done by the realtime-virtual-host profile.

This is how the realtime-virtual-host profile organizes kernel threads into FIFO slots:

   151  FF   4  [rcuc/15] *
   153  FF   3  [ktimersoftd/15] *
   154  FF   2  [ksoftirqd/15] *

As we can see, rcuc > ktimersoftd > ksoftirqd. This allows for running a PMD thread with fifo:1 without running into the risk of starving those important threads.

However, the realtime-virtual-guest doesn't do any of this:

    17  FF   2  [rcuc/1] *
    19  FF   1  [ktimersoftd/1] *
    20  TS   -  [ksoftirqd/1] *

So, a PMD thread running in the guest with fifo:1 can starve ktimersoftd and ksoftirqd should those threads ever become runnable.

The solution should be the realtime-virtual-guest profile to assign FIFO priority to kernel threads the same way the realtime-virtual-host is doing.

Version-Release number of selected component (if applicable): tuned-2.8.0-3.el7.noarch

Comment 13 Luiz Capitulino 2017-06-13 14:47:58 UTC
I've verified the kernel threads priorities in the guest are now correct. I've also run a quick cyclictest measurement test and it worked fine.

Comment 14 Tereza Cerna 2017-06-14 13:45:52 UTC
Thank you, Luiz, for your test.

I did some other test. It works as expected -> VERIFIED.

Existing test was repaired for this issue. See /CoreOS/tuned/Regression/create-new-nfv-profiles.

===============================================
Verified in:
    tuned-2.8.0-5.el7.noarch
    tuned-profiles-nfv-guest-2.8.0-5.el7.noarch
PASS
===============================================

# tuned-adm profile realtime-virtual-guest

# ps axo pid,class,rtprio,comm | grep 'ktimersoft\|softirq\|rcuc\|rcub'
    3 FF       2 ksoftirqd/0
   12 FF       2 ksoftirqd/1
   18 FF       2 ksoftirqd/2
   24 FF       2 ksoftirqd/3
   30 FF       2 ksoftirqd/4
   34 FF       2 ksoftirqd/5
   38 FF       2 ksoftirqd/6
   42 FF       2 ksoftirqd/7
 9144 FF       3 ktimersoftd
 9150 FF       4 rcub
 9156 FF       4 rcuc

===============================================
Reproduced in:
    tuned-2.8.0-3.el7.noarch
    tuned-profiles-nfv-guest-2.8.0-3.el7.noarch
FAIL
===============================================

# tuned-adm profile realtime-virtual-guest

# ps axo pid,class,rtprio,comm | grep 'ktimersoft\|softirq\|rcuc\|rcub'
    3 TS       - ksoftirqd/0
   12 TS       - ksoftirqd/1
   18 TS       - ksoftirqd/2
   24 TS       - ksoftirqd/3
   30 TS       - ksoftirqd/4
   34 TS       - ksoftirqd/5
   38 TS       - ksoftirqd/6
   42 TS       - ksoftirqd/7
 9943 TS       - ktimersoftd
 9947 TS       - rcub
 9949 TS       - rcuc

Comment 15 errata-xmlrpc 2017-08-01 12:35:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2102

Comment 16 Fedora Update System 2017-10-13 14:20:34 UTC
tuned-2.9.0-0.1.rc1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-d9c6b990df

Comment 17 Fedora Update System 2017-10-29 21:06:48 UTC
tuned-2.9.0-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-0e45ce4685

Comment 18 Fedora Update System 2017-10-29 21:12:55 UTC
tuned-2.9.0-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-c30e9bd1ea