When using RHEL6.3/KVM (libvirt-0.9.10-21.el6_3.5.x86_64) as a hypervisor, the following block should be added to /etc/libvirt/qemu.conf as a part of Nova compute setups. ======== clear_emulator_capabilities = 0 user = "root" group = "root" cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun", ] ======== Without that, launching new VM fails with the following error in /var/log/nova/compute.log. 2012-11-19 08:43:20 ERROR nova.compute.manager [req-94207b9f-cc0c-4da5-a7c1-491acd400348 609869efdf344b58aaa490cf54a19a79 5e308a4f4a73488d9facbc3fb23c7d38] [instance: b65f1705-4a3e-43e7-bc71-891417dcb0c3] Build error: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 496, in _run_instance\n injected_files, admin_password)\n', ' File "/usr/lib/python2.6/site-packages/nova/compute/manager.py", line 743, in _spawn\n block_device_info)\n', ' File "/usr/lib/python2.6/site-packages/nova/exception.py", line 117, in wrapped\n temp_level, payload)\n', ' File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__\n self.gen.next()\n', ' File "/usr/lib/python2.6/site-packages/nova/exception.py", line 92, in wrapped\n return f(*args, **kw)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1062, in spawn\n block_device_info)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1888, in _create_domain_and_network\n domain = self._create_domain(xml)\n', ' File "/usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py", line 1867, in _create_domain\n domain.createWithFlags(launch_flags)\n', ' File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 187, in doit\n result = proxy_call(self._autowrap, f, *args, **kwargs)\n', ' File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 147, in proxy_call\n rv = execute(f,*args,**kwargs)\n', ' File "/usr/lib/python2.6/site-packages/eventlet/tpool.py", line 76, in tworker\n rv = meth(*args,**kwargs)\n', ' File "/usr/lib64/python2.6/site-packages/libvirt.py", line 650, in createWithFlags\n if ret == -1: raise libvirtError (\'virDomainCreateWithFlags() failed\', dom=self)\n', "libvirtError: internal error Process exited while reading console log output: char device redirected to /dev/pts/2\nwarning: could not open /dev/net/tun: no virtual network emulation\nqemu-kvm: -netdev tap,ifname=tap892a554b-db,script=,id=hostnet0: Device 'tap' could not be initialized\n\n"]
So user = "root" and group = "root" are already the default are they not? Setting clear_emulator_capabilities = 0 is a bit worrying TBH. For my information, "/dev/net/tun" is added to the list above, and that was removed upstream with: http://libvirt.org/git/?p=libvirt.git;a=commit;h=c52cbe4
I'll prepare the reproduction setup with RHEL6.4 and Fedora18 next week. Pls wait for a while.
Hi, I checked the exact requirement with RHEL6.4Beta and Fedora18. Note that this is tested with Quantum LinuxBridge Agent. Under OVS agent, requirement may be different. If I have time, I'll test with OVS agent later. 1) RHEL6.4Beta with the following packages. # rpm -q qemu-kvm libvirt openstack-quantum-linuxbridge openstack-nova-compute qemu-kvm-0.12.1.2-2.335.el6.x86_64 libvirt-0.10.2-9.el6.x86_64 openstack-quantum-linuxbridge-2012.2.1-1.el6ost.noarch openstack-nova-compute-2012.2.1-2.el6ost.noarch The following was necessary to avoid the error. ======== user = "root" group = "root" cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun", ] ======== qemu-kvm runs under root. # ps -ef | grep qemu root 2810 1 20 21:31 ? 00:00:14 /usr/libexec/qemu-kvm -name instance-0000002d ... 2) Fedora18 with the following packages. # rpm -q qemu-kvm libvirt openstack-quantum-linuxbridge openstack-nova-compute qemu-kvm-1.2.0-24.fc18.x86_64 libvirt-0.10.2.1-3.fc18.x86_64 openstack-quantum-linuxbridge-2012.2.1-1.fc18.noarch openstack-nova-compute-2012.2-1.fc18.noarch Only cgroup_device_acl was necessary. ======== cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun", ] ======== qemu-kvm runs under qemu user. # ps -ef | grep qemu qemu 3021 1 11 11:31 ? 00:00:13 /usr/bin/qemu-kvm -name instance-0000002c ... --- So the bottom line is: - cgroup_device_acl is necessary both for RHEL6/Fedora18 - The default user/group is qemu both for RHEL6/Fedora18 (the comment in qemu.conf seems wrong), and it should be changed to root for RHEL6. Thanks.
Hi, I did an additional test with OVS plugin. I found that under OVS plugin, the additional options in qemu.conf are all unnecessary. The reason probably comes from the difference of how VIF driver configures VM nics. 1) LinuxBridge Agent In nova.conf: libvirt_vif_driver = nova.virt.libvirt.vif.QuantumLinuxBridgeVIFDriver with this driver, qemu-kvm has the following options. "-netdev tap,ifname=tape8a17a92-d8,script=,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=fa:16:3e:11:71:f6,bus=pci.0,addr=0x3" As this probably directly accesses /dev/net/tun, additional options are required. 2) OVS Agent In nova.conf libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver with this driver, qemu-kvm has the following options. "-netdev tap,fd=22,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=fa:16:3e:ff:11:9a,bus=pci.0,addr=0x3" This follows the convention as in http://libvirt.org/git/?p=libvirt.git;a=commit;h=c52cbe4 , so additional options are not necessary. I'm not 100% sure but I guess this comes from the difference of the interface type. QuantumLinuxBridgeVIFDriver uses ethernet type while QuantumLinuxBridgeVIFDriver uses bridge type.
Bruce, I checked the conditions again with the latest RPMs. The result is the same as before. 1) LinuxBridge Agent For Fedora18: cgroup_device_acl is necessary. cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun", ] For RHEL6.4Beta: cgroup_device_acl and user/group is necessary. user = "root" group = "root" cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun", ] 2) OVS Agent No additional options are necessary both for Fedora18 and RHEL6.4Beta. The following is the set-ups. [Fedora18] # uname -a Linux opst02 3.7.4-204.fc18.x86_64 #1 SMP Wed Jan 23 16:44:29 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux # rpm -q qemu-kvm libvirt openstack-nova-compute qemu-kvm-1.2.2-6.fc18.x86_64 libvirt-0.10.2.3-1.fc18.x86_64 openstack-nova-compute-2012.2.2-1.fc18.noarch # rpm -qa | grep openstack-quantum openstack-quantum-openvswitch-2012.2.1-1.fc18.noarch openstack-quantum-linuxbridge-2012.2.1-1.fc18.noarch openstack-quantum-2012.2.1-1.fc18.noarch [RHEL6.4Beta] # uname -a Linux opst03 2.6.32-343.el6.x86_64 #1 SMP Mon Nov 19 16:46:10 EST 2012 x86_64 x86_64 x86_64 GNU/Linux # rpm -q qemu-kvm-rhev libvirt openstack-nova-compute qemu-kvm-rhev-0.12.1.2-2.351.el6.x86_64 libvirt-0.10.2-9.el6.x86_64 openstack-nova-compute-2012.2.2-8.el6ost.noarch # rpm -qa | grep openstack-quantum openstack-quantum-openvswitch-2012.2.1-3.el6ost.noarch openstack-quantum-2012.2.1-3.el6ost.noarch openstack-quantum-linuxbridge-2012.2.1-3.el6ost.noarch
Added instructions in section "Configuring Vif-plugging in Nova" at linux bridge plugin to add: user = "root" group = "root" cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun", ] to qemu.conf
Setting this bug to Inst& Config Guide because Getting Started Guide no longer has manual install steps and hence Nova install, and I don't see this in the ICG on deathstar yet.
Fixed the spacing issue in "Configuring Virtual Interface Plugging" - ID: 15829 [rev: 466986].