Bug 1688736 - libvirt: all VMs fail to start with Operation not supported: operation 'setCpusetMemoryMigrate' not supported
Summary: libvirt: all VMs fail to start with Operation not supported: operation 'setCp...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: libvirt
Version: 30
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Libvirt Maintainers
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-03-14 10:38 UTC by Zbigniew Jędrzejewski-Szmek
Modified: 2019-04-14 00:01 UTC (History)
11 users (show)

Fixed In Version: libvirt-5.1.0-4.fc30
Clone Of:
Environment:
Last Closed: 2019-04-14 00:01:30 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Zbigniew Jędrzejewski-Szmek 2019-03-14 10:38:50 UTC
Description of problem:

Error starting domain: Operation not supported: operation 'setCpusetMemoryMigrate' not supported

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 75, in cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 111, in tmpcb
    callback(*args, **kwargs)
  File "/usr/share/virt-manager/virtManager/libvirtobject.py", line 66, in newfn
    ret = fn(self, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/domain.py", line 1420, in startup
    self._backend.create()
  File "/usr/lib64/python3.7/site-packages/libvirt.py", line 1080, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirt.libvirtError: Operation not supported: operation 'setCpusetMemoryMigrate' not supported

I use cgroupv2 mounted on /sys/fs/cgroup (systemd.unified-cgroup-hierarchy).
Is the error somehow related to the cpuset controller reappearing in kernel 5.0.

Version-Release number of selected component (if applicable):
libvirt-daemon-5.1.0-1.fc30.x86_64
kernel-core-5.0.0-0.rc8.git0.1.fc30.x86_64
(generally up-to-date F30)

How reproducible:
seems entirely reproducible, I cannot start any VM.

Comment 1 Cole Robinson 2019-03-15 00:51:42 UTC
Pavel sounds like cgroup v2, have you heard of this issue?

Comment 2 Pavel Hrdina 2019-03-15 08:35:12 UTC
Hi, yes, that is cgroup v2 and it's related to cpuset controller.

In upstream it's already implemented and should be fixed in libvirt 5.2.0
so I guess that we need to backport these patches to Fedora 30 or wait
for upstream libvirt 5.2.0 release and rebase in Fedora.


commit 74e7da060543a87610b42fc6ba26a45b0a6e3974
Author: Pavel Hrdina <phrdina>
Date:   Tue Feb 19 15:42:51 2019 +0100

    util: implement virCgroupV2(Set|Get)CpusetMems

commit 77c1cf4da2f761a91756c09fa4fd37ae1802e650
Author: Pavel Hrdina <phrdina>
Date:   Tue Feb 19 15:53:34 2019 +0100

    util: implement virCgroupV2(Set|Get)CpusetMemoryMigrate

commit 3b72c84ff1c1b8b393ba9c2ccb004f8eb1ebda95
Author: Pavel Hrdina <phrdina>
Date:   Tue Feb 19 15:55:38 2019 +0100

    util: implement virCgroupV2(Set|Get)CpusetCpus

commit a6aedcf39bd3212a3cd624b765bb724fd36d6a8a
Author: Pavel Hrdina <phrdina>
Date:   Wed Feb 20 13:50:23 2019 +0100

    util: enable cgroups v2 cpuset controller for threads

Comment 3 Daniel Berrangé 2019-03-15 09:36:31 UTC
I think we should fail more gracefully by just not using cpuset if it doesn't exist, just rely on traditional CPU affinity. We already have a config option to let us disable various controllers including cpuset from v1, so our code ought to cope with this already, if we just handle the error better.

Comment 4 Pavel Hrdina 2019-03-15 09:47:09 UTC
The issue here is that the cpuset controller exists on that system but it's functions are not implemented yet in libvirt.  If some controller doesn't exist libvirt will correctly ignore it.
Every call to setCpusetMemoryMigrate is covered by virCgroupHasController(priv->cgroup, VIR_CGROUP_CONTROLLER_CPUSET), the issue here is that the cpuset controller is available.

To fix this we would have to modify virCgroupV1HasController and virCgroupV2HasController to return true only if the controller is available in kernel and also implemented in libvirt.

Comment 5 Fedora Update System 2019-04-02 16:43:41 UTC
libvirt-5.1.0-4.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-ecdfd445d4

Comment 6 Fedora Update System 2019-04-03 02:27:33 UTC
libvirt-5.1.0-4.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-ecdfd445d4

Comment 7 Zbigniew Jędrzejewski-Szmek 2019-04-04 14:16:59 UTC
Seems to work fine with libvirt-daemon-5.1.0-4.fc30.x86_64 on kernel-5.0.5-300.fc30.x86_64.

Comment 8 Fedora Update System 2019-04-14 00:01:30 UTC
libvirt-5.1.0-4.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.