RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2037734 - Reflect different limits in CGroupV1/V2 for cpu.weight
Summary: Reflect different limits in CGroupV1/V2 for cpu.weight
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libvirt
Version: unspecified
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: rc
: 9.3
Assignee: Pavel Hrdina
QA Contact: Meina Li
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-01-06 13:11 UTC by Michal Privoznik
Modified: 2023-11-07 09:38 UTC (History)
9 users (show)

Fixed In Version: libvirt-9.1.0-1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 2035518
Environment:
Last Closed: 2023-11-07 08:30:47 UTC
Type: Bug
Target Upstream Version: 9.1.0
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-106958 0 None None None 2022-01-06 13:15:12 UTC
Red Hat Product Errata RHSA-2023:6409 0 None None None 2023-11-07 08:31:21 UTC

Description Michal Privoznik 2022-01-06 13:11:15 UTC 
+++ This bug was initially created as a clone of Bug #2035518 +++

Description of problem:
As now in rhel9, cgroup V2 is enabled by default. All weights are in the range [1, 10000]. VM with the value of cpu.weight larger than 10000 can not be started, while this kind of VMs can be started on RHEL8 host which uses cgroup V1. Due to the same reason, migrating this kind of VMs from RHEL8 to RHEL9 will fail. 

Version-Release:
On Source host:
libvirt-7.6.0-6.module+el8.5.0+13051+7ddbe958.x86_64
qemu-kvm-6.0.0-33.module+el8.5.0+13041+05be2dc6.x86_64
On target host:
libvirt-7.10.0-1.el9.x86_64
qemu-kvm-6.2.0-1.el9.x86_64

1. check the guest xml
<domain type='kvm'>
  <name>VM</name>
...
 <cputune>
    <shares>10001</shares>
  </cputune>
...

2. try to migrate the guest from source host to target guest.
# virsh migrate VM qemu+ssh://$target/system --verbose --live
error: error from service: GDBus.Error:org.freedesktop.DBus.Error.InvalidArgs: Value specified in CPUWeight is out of range
(migration failed)

On RHEL, we can migrate this kind of VM successfully by specifying xml, e.g.,
3. dump the VM xml
# virsh dumpxml VM > VM.xml

4. change the shares value from 10001 to the value smaller than 10000,
# cat VM.xml
...
 <cputune>
    <shares>10000</shares>
  </cputune>
...

5. migrate the VM by specify the above xml
# virsh migrate VM qemu+ssh://$target/system --verbose --live --xml VM.xml
Migration: [100 %]

My question is:
1. Is there any problem with migrating this kind of VM on openstack?
2. If there is problem now, can the above workaround be accepted?
2. If the above workaround can be accepted, can the customers specify the xml, then migrate the guest on openstack now?

If there is no problem with migrating this kind of VM on openstack, or it can be solved by other method, feel free to close this bug.

--- Additional comment from Michal Privoznik on 2022-01-04 13:43:30 CET ---

Right, libvirt currently uses the same set of limits for both CGroupV1 and V2. That should be fixed on libvirt level. And for mgmt apps (like OpenStack) they need to provide an XML during migration with the values recalculated to fit into CGroupV2 limits.
Comment 1 John Ferlan 2022-01-11 11:59:09 UTC 
Jarda - the ITR is set causing the Triaged flag to be set (silly RHEL bot action). 

Since ITR is set this should be assigned to someone or ITR cleared if there's no upstream patches yet.
Comment 2 Pavel Hrdina 2022-01-26 10:28:12 UTC 
No upstream patches yet so removing ITR.
Comment 4 Meina Li 2023-07-05 03:19:59 UTC 
Hi Pavel,

This bug can still be reproduced now. Do we have plan to fix it? If yes, we need to delay the "stale date". Please help to check it again. Thanks.

Reproduced Version:
source host:
libvirt-8.0.0-21.module+el8.9.0+19166+e262ca96.x86_64
qemu-kvm-6.2.0-36.module+el8.9.0+19222+f46ac890.x86_64


target host:
libvirt-9.5.0-0rc1.1.el9.x86_64
qemu-kvm-8.0.0-5.el9.x86_64

Reproduced Steps:
1. Prepare a guest xml with the following cputune.
  <cputune>
    <shares>10001</shares>
  </cputune>
2. Start the guest.
# virsh start rhel
Domain 'rhel' started
------We hope the guest can not be started as RHEL 9
3. Migrate the guest to RHEL 9.
# virsh migrate rhel qemu+ssh://dell-per750-39.lab.eng.pek2.redhat.com/system --live --verbose
root.eng.pek2.redhat.com's password: 
error: invalid argument: shares '10001' must be in range [1, 10000]
Comment 5 Pavel Hrdina 2023-07-13 10:15:12 UTC 
Hi Meina,

Getting error when migrating from cgroup v1 host to cgroup v2 host where the cputune
shares value is out of cgroup v2 range is expected behavior. In libvirt we cannot simply
modify the value to fit within the new range as we would not be able to figure out what
new value to use while not breaking the expected behavior for users.

For the error message from libvirt that was fixed by the following commits:

commit cf3414a85b8383d71d6ae2a53daf63c331cc2230
Author: Pavel Hrdina <phrdina>
Date:   Tue Jan 17 10:02:07 2023 +0100

    vircgroupv2: fix cpu.weight limits check

commit 38af6497610075e5fe386734b87186731d4c17ac
Author: Pavel Hrdina <phrdina>
Date:   Tue Jan 17 10:08:08 2023 +0100

    domain_validate: drop cpu.shares cgroup check

commit ead6e1b00285cbd98e0f0727efb8adcb29ebc1ba
Author: Pavel Hrdina <phrdina>
Date:   Tue Jan 17 10:33:22 2023 +0100

    docs: document correct cpu shares limits with both cgroups v1 and v2

and is part of libvirt 9.1.0 so already part of the rebase to 9.1.0.

Since this BZ covers only libvirt reporting correct error messages changing priority to low
and severity to low.
Comment 6 Meina Li 2023-07-26 03:39:45 UTC 
Comment 4 is expected result. Also do some basic test here:

Test Version:
libvirt-9.5.0-3.el9.x86_64
qemu-kvm-8.0.0-9.el9.x86_64

Test Steps:
S1: Start guest with 10000 weights.
1. Prepare a guest with 10000 shares.
 <cputune>
    <shares>10000</shares>
  </cputune>
2. Start the guest.
# virsh start rhel
Domain 'rhel' started
# virsh dumpxml rhel --xpath cputune
<cputune>
  <shares>10000</shares>
</cputune>

S2: Start guest with 10001/0 weights.
1. Prepare a guest with 10000 shares.
 <cputune>
    <shares>10001</shares>
  </cputune>
2. Start the guest.
# virsh start rhel
error: Failed to start domain 'rhel'
error: invalid argument: shares '10001' must be in range [1, 10000]
3. Prepare a guest with 10000 shares.
 <cputune>
    <shares>0</shares>
  </cputune>
4. Start the guest.
# virsh start rhel
error: Failed to start domain 'rhel'
error: invalid argument: shares '0' must be in range [1, 10000]
Comment 9 Meina Li 2023-08-15 03:49:23 UTC 
Verified Version:
libvirt-9.5.0-5.el9.x86_64
qemu-kvm-8.0.0-11.el9.x86_64

Verified Steps:
As comment 6
Comment 11 errata-xmlrpc 2023-11-07 08:30:47 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: libvirt security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:6409
Note You need to log in before you can comment on or make changes to this bug.