Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionLuiz Capitulino
2016-06-14 19:31:23 UTC
Description of problem:
A domain using this vCPU placement & configuration:
<vcpu placement='static'>2</vcpu>
<cputune>
<vcpupin vcpu='0' cpuset='14'/>
<vcpupin vcpu='1' cpuset='15'/>
<emulatorpin cpuset='0,2,4,6'/>
<vcpusched vcpus='0-1' scheduler='fifo' priority='1'/>
</cputune>
Won't start on a system using the NFV profile but running the non-RT kernel:
[root@virtlab508 ~]# virsh start RHEL73-RT
error: Failed to start domain RHEL73-RT
error: Cannot set scheduler parameters for pid 161919: Operation not permitted
[root@virtlab508 ~]#
However, it starts fine if we do one of the following:
- Run the RT kernel in the host
- Drop the vcpusched line
Version-Release number of selected component (if applicable): libvirt-1.3.4-1.el7.x86_64
How reproducible:
Steps to Reproduce:
1. Install the tuned profile-nfv in the host and isolate some cores (this step might not be needed)
2. Use the cputune and vcpusched elements to pin and assign RT priority to vCPUs
3. Try to start the guest with "virsh start"
4. Guest won't start
Karl Rister debugged this issue some time ago and found the following:
"""
I believe I have traced it back to a cgroup partitioning issue. Libvirt
is placing the guest process in a specific partition which does not have
any RT runtime allocated to it, by doing so and then propagating the
values through the cgroup hierarchy I am finally able to manually chrt
the vcpu pids:
root@host ~ $ virsh qemu-monitor-command master --hmp info cpus
* CPU #0: pc=0xffffffff8115ac90 thread_id=3784
CPU #1: pc=0xffffffff8104a086 (halted) thread_id=3785
CPU #2: pc=0xffffffff8104a086 (halted) thread_id=3786
root@host ~ $ chrt -f -p 1 3785
chrt: failed to set pid 3785's policy: Operation not permitted
root@host ~ $ chrt -f -p 1 3786
chrt: failed to set pid 3786's policy: Operation not permitted
root@host ~ $ cat /sys/fs/cgroup/cpu,cpuacct/cpu.rt_runtime_us
950000
root@host ~ $ cat
/sys/fs/cgroup/cpu,cpuacct/machine.slice/cpu.rt_runtime_us
0
root@host ~ $ echo 950000 >
/sys/fs/cgroup/cpu,cpuacct/machine.slice/cpu.rt_runtime_us
root@host ~ $ cat
/sys/fs/cgroup/cpu\,cpuacct/machine.slice/machine-qemu\\x2dmaster.scope/cpu.rt_runtime_us
0
root@host ~ $ echo 950000 >
/sys/fs/cgroup/cpu\,cpuacct/machine.slice/machine-qemu\\x2dmaster.scope/cpu.rt_runtime_us
root@host ~ $ cat
/sys/fs/cgroup/cpu\,cpuacct/machine.slice/machine-qemu\\x2dmaster.scope/vcpu1/cpu.rt_runtime_us
0
root@host ~ $ echo 950000 >
/sys/fs/cgroup/cpu\,cpuacct/machine.slice/machine-qemu\\x2dmaster.scope/vcpu1/cpu.rt_runtime_us
root@host ~ $ chrt -f -p 1 3785
root@host ~ $ cat
/sys/fs/cgroup/cpu\,cpuacct/machine.slice/machine-qemu\\x2dmaster.scope/vcpu2/cpu.rt_runtime_us
0
root@host ~ $ echo 950000 >
/sys/fs/cgroup/cpu\,cpuacct/machine.slice/machine-qemu\\x2dmaster.scope/vcpu2/cpu.rt_runtime_us
bash: echo: write error: Invalid argument
I have to split the available runtime...
"""
This makes sense because it seems that the RT kernel has less cgroups options enabled, so maybe it's missing a kernel config option that enforces this behavior which makes the guest work on the RT kernel.
Harald Jensås also got this issue when debugged it down to a tuned profile setting. Closing as a dupe.
*** This bug has been marked as a duplicate of bug 1346715 ***
Description of problem: A domain using this vCPU placement & configuration: <vcpu placement='static'>2</vcpu> <cputune> <vcpupin vcpu='0' cpuset='14'/> <vcpupin vcpu='1' cpuset='15'/> <emulatorpin cpuset='0,2,4,6'/> <vcpusched vcpus='0-1' scheduler='fifo' priority='1'/> </cputune> Won't start on a system using the NFV profile but running the non-RT kernel: [root@virtlab508 ~]# virsh start RHEL73-RT error: Failed to start domain RHEL73-RT error: Cannot set scheduler parameters for pid 161919: Operation not permitted [root@virtlab508 ~]# However, it starts fine if we do one of the following: - Run the RT kernel in the host - Drop the vcpusched line Version-Release number of selected component (if applicable): libvirt-1.3.4-1.el7.x86_64 How reproducible: Steps to Reproduce: 1. Install the tuned profile-nfv in the host and isolate some cores (this step might not be needed) 2. Use the cputune and vcpusched elements to pin and assign RT priority to vCPUs 3. Try to start the guest with "virsh start" 4. Guest won't start