Bug 703095

Created attachment 497745 [details]
error displayed in rhevm

Created attachment 497746 [details]
vdsm.log

Created attachment 497747 [details]
rhevm.log

Can this be reproduced with vdsm on RHEL?  Or is it only on rhevh?

it should be vdsm component, if not pls correct it for me, thanks!
VDSM version: vdsm-4.9-62.el6.x86_64

Also doubt it related to bug 688616, if so, it should be fixed in vdsm-4.9-63.
（will retest in vdsm-4.9-63.)

retested on rhev-hypervisor-6.1-20110506 which vdsm version is vdsm-4.9-63,
the vm can run normally.

Hence the bug is fixed in vdsm-4.9-63.

*** This bug has been marked as a duplicate of bug 688616 ***

the bug is re-happened on ic144 + rhevh-6.2-20111006, read last comment of bug 688616, is it due to cgroup back to beta3? 
run below command on rhevh-6.2-20111006:
# lscgroup 
cgroups can't be listed: Cgroup is not mounted

# cat /proc/mounts | grep cgroup
none /cgroup tmpfs rw,rootcontext=system_u:object_r:var_lib_t:s0,seclabel,relatime 0 0

vdsm-4.9-106.el6.x86_64
libcgroup-0.37-3.el6.x86_64
kernel-2.6.32-206.el6.x86_64

Right, we still do not enabled cgropus in vdsm-4.9-106. But Vdsm should start VMs nontheless. Would you provide modern vdsm.log so we can tell why the VM did not start?

Created attachment 527060 [details]
vdsm.log-20111009

The log has

libvirtError: Unable to create cgroup for www: No such device or address

This is not what I am getting when cgroup is off. Would you provide libvirtd.log and libvirt version, too? Would it help to kill libvirtd after vdsmd has started?

In any case, libvirt should be able to start VMs even with cgroups off.

Please provide libvirt related log informations and
a reproducer for libvirt. I can reproduce this but only
if I remove cgroup support while libvirtd is running. If I
restart libvirtd while cgoups are off things are just fine:

[root@test ~]# cat /proc/mounts | grep cgroup
cgroup /cgroup/cpuset cgroup rw,relatime,cpuset 0 0
cgroup /cgroup/cpu cgroup rw,relatime,cpu 0 0
cgroup /cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
cgroup /cgroup/memory cgroup rw,relatime,memory 0 0
cgroup /cgroup/devices cgroup rw,relatime,devices 0 0
cgroup /cgroup/freezer cgroup rw,relatime,freezer 0 0
cgroup /cgroup/net_cls cgroup rw,relatime,net_cls 0 0
cgroup /cgroup/blkio cgroup rw,relatime,blkio 0 0
[root@test ~]# umount /cgroup/blkio /cgroup/net_cls /cgroup/freezer /cgroup/devices /cgroup/memory /cgroup/cpuacct /cgroup/cpu /cgroup/cpuset
[root@test ~]# lscgroup 
cgroups can't be listed: Cgroup is not mounted
[root@test ~]# virsh list
 Id Name                 State
----------------------------------

[root@test ~]# virsh start test6.1
error: Failed to start domain test6.1
error: Unable to create cgroup for test6.1: No such device or address

[root@test ~]# /etc/init.d/libvirtd restart
Stopping libvirtd daemon:                                  [  OK  ]
Starting libvirtd daemon:                                  [  OK  ]
[root@test ~]# lscgroup 
cgroups can't be listed: Cgroup is not mounted
[root@test ~]# virsh start test6.1
Domain test6.1 started

[root@test ~]# virsh list
 Id Name                 State
----------------------------------
  1 test6.1              running

[root@test ~]# 

  Libvirt uses cgroups if available but it won't dynamically
chase if they are available or not.

  What is happening in a rhevm environment, is vdsm turning
cgroup off without restarting the libvirt daemon ?

Daniel

If you make any change to the cgroups mount point configuration, while libvirtd is running, it is *required* that you restart libvirtd, so it can re-initialize its cgroups structure.

Thanks for your help, folks. vdsm -107 (for beta 4) no longer turns cgroups off, so the issue is moot.

(In reply to comment #21)
> vdsm -107 (for beta 4) no longer turns cgroups off
> so the issue is moot.

we still need workaround for beta3, please add relnote

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
due to a kernel bug 742414, vdsm has to turn cgroups off when it first starts (service cgroup stop && chconfig cgroup off).
If a users turns cgroups on, and restart libvirtd, vdsm would stop cgroups on its startup, and render libvirt unable to start VMs.
Users should avoid enabling cgroups in rhev-3.0 beta 3.

if libvirtd starts when cgroups service is on, and then cgroups service is turned off, libvirtd won't run new VMs.

RHEV-H should keep cgroups service off from boot time, until vdsm stops stopping cgroups on its startup.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2011-1782.html