Bug 1178986

Summary:	[RFE]: add support for setting RT priority of vCPU threads
Product:	Red Hat Enterprise Linux 7	Reporter:	Luiz Capitulino <lcapitulino>
Component:	libvirt	Assignee:	Martin Kletzander <mkletzan>
Status:	CLOSED ERRATA	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	7.2	CC:	dyuan, honzhang, juzhang, lhuang, lmiksik, mzhan, rbalakri, xfu
Target Milestone:	rc	Keywords:	FutureFeature, Upstream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	libvirt-1.2.13-1.el7	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-11-19 06:06:41 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	1175463

Description Luiz Capitulino 2015-01-05 20:30:13 UTC

Description of problem:

For real-time support, the system administrator needs to be able to set real-time priority of QEMU vCPUs. This is somewhat related to CPU pinning, which libvirt already supports via the cputune schema element.


Version-Release number of selected component (if applicable): libvirt-1.2.8-11.el7.x86_64

Comment 1 Martin Kletzander 2015-01-13 08:00:41 UTC

Patches proposed upstream (with per-thread options):

https://www.redhat.com/archives/libvir-list/2015-January/msg00363.html

Comment 2 Martin Kletzander 2015-02-11 16:34:53 UTC

Fixed upstream with v1.2.12-116-gb6a2828 -- v1.2.12-118-g104ba59:

commit b6a2828e53717efdb41beb6a39b8e3a0166b9ef0
Author: Martin Kletzander <mkletzan>
Date:   Thu Jan 8 15:36:54 2015 +0100

    util: Add virProcessSetScheduler() function for scheduler settings

commit 8680ea9749a22230714a982df2531ac8847551bf
Author: Martin Kletzander <mkletzan>
Date:   Thu Jan 8 13:54:21 2015 +0100

    docs, schema, conf: Add support for setting scheduler parameters of guest threads

commit 104ba5966a440f40e0287f49fa46f04add52b1c5
Author: Martin Kletzander <mkletzan>
Date:   Thu Jan 8 15:37:50 2015 +0100

    qemu: Add support for setting vCPU and I/O thread scheduler setting

Comment 4 Luyao Huang 2015-06-23 06:30:18 UTC

Verify this issue with libvirt-1.2.16-1.el7.x86_64:

Test with kernel: 3.10.0-237.rt56.148.el7.x86_64.virt

1. prepare a xml like this in guest xml:

...

  <iothreads>4</iothreads>
  <iothreadids>
    <iothread id='1'/>
    <iothread id='2'/>
    <iothread id='3'/>
    <iothread id='4'/>
  </iothreadids>
  <cputune>
    <emulatorpin cpuset='0-7'/>
    <iothreadpin iothread='1' cpuset='6'/>
    <iothreadpin iothread='2' cpuset='8'/>
    <vcpusched vcpus='0-2,4' scheduler='fifo' priority='1'/>
    <iothreadsched iothreads='1-2' scheduler='fifo' priority='1'/>
  </cputune>
...

2. start the guest

# virsh start test3

3. check qemu pid

# ps aux|grep qemu

qemu     21353 13.8  0.0 8286200 41092 ?       Sl   Jun01 596:53 /usr/libexec/qemu-kvm -name test3

4. check the children process in qemu: 

# cat /proc/21353/task/*/status |grep Cpus_allowed_list
Cpus_allowed_list:    0-7
Cpus_allowed_list:    0-7
Cpus_allowed_list:    6     <------iothread 1 pid
Cpus_allowed_list:    8      <----- iothread 2 pid
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-6
Cpus_allowed_list:    0-7

5. use chrt show the RT info:

# chrt -p 21363
pid 21363's current scheduling policy: SCHED_FIFO
pid 21363's current scheduling priority: 1

# chrt -p 21364
pid 21364's current scheduling policy: SCHED_FIFO
pid 21364's current scheduling priority: 1

6. check the vcpu thread id:

virsh # qemu-monitor-command test3 --hmp info cpus
* CPU #0: pc=0xffffffff81055e06 (halted) thread_id=21413
  CPU #1: pc=0xffffffff81055e06 (halted) thread_id=21414
  CPU #2: pc=0xffffffff81055e06 (halted) thread_id=21415
  CPU #3: pc=0xffffffff81055e06 (halted) thread_id=21416
  CPU #4: pc=0xffffffff81055e06 (halted) thread_id=21417
  CPU #5: pc=0xffffffff81055e06 (halted) thread_id=21418

7. use chrt show the RT info: 

# chrt -p 21413
pid 21413's current scheduling policy: SCHED_FIFO
pid 21413's current scheduling priority: 1

# chrt -p 21414
pid 21414's current scheduling policy: SCHED_FIFO
pid 21414's current scheduling priority: 1

# chrt -p 21415
pid 21415's current scheduling policy: SCHED_FIFO
pid 21415's current scheduling priority: 1

# chrt -p 21416
pid 21416's current scheduling policy: SCHED_OTHER
pid 21416's current scheduling priority: 0

# chrt -p 21417
pid 21417's current scheduling policy: SCHED_FIFO
pid 21417's current scheduling priority: 1

# chrt -p 21418
pid 21418's current scheduling policy: SCHED_OTHER
pid 21418's current scheduling priority: 0

8. if we set a invalid value and start guest will get error:

# virsh start test3
error: Failed to start domain test3
error: unsupported configuration: Scheduler priority 100 out of range [1, 99]

9. also test with other policy

# chrt -p 1873
pid 1873's current scheduling policy: SCHED_RR
pid 1873's current scheduling priority: 11

# chrt -p 1877
pid 1877's current scheduling policy: SCHED_BATCH
pid 1877's current scheduling priority: 0

# chrt -p 1883
pid 1883's current scheduling policy: SCHED_IDLE
pid 1883's current scheduling priority: 0

# chrt -p 1893
pid 1893's current scheduling policy: SCHED_OTHER
pid 1893's current scheduling priority: 0

Comment 6 errata-xmlrpc 2015-11-19 06:06:41 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html