Hide Forgot
Description of problem: When an interface with type ethernet is created, but the tap device it is intended to connect to is not precreated, qemu exits when its TUNSETIFF call fails with an EPERM. When libvirt is starting qemu without sufficient privileges to create a tap device (which requires CAP_NET_ADMIN and often root), it would be the Right Thing to precreate a dynamically named tap device (just as qemu would do itself if it had sufficient permissions) and pass its name in on the command line. Version-Release number of selected component (if applicable): libvirt-0.8.1-1 How reproducible: 100% Steps to Reproduce: 1. Configure /etc/libvirt/qemu.conf with user != root 2. Define a guest with at least one network interface having @type='ethernet' and a valid script/@path but no target element defined 3. Attempt to start this guest. Actual results: Guest startup fails with an error from qemu: warning: could not configure /dev/net/tun: no virtual network emulation on using strace to investigate, this error is caused by EPERM being returned by the TUNSETIFF ioctl. Expected results: The guest is started and the script given is called with a dynamically allocated device name as an argument.
The type=ethernet mode of networking is not supported when libvirt is either compiled with capng, or qemu is running non-root. This is because TUNSETIFF requires CAP_NET_ADMIN. No non-root process has this, and when using capng, we further block this when qemu is running as root, or the setup script is setuid, because having CAP_NET_ADMIN would be a significant security threat. We need to invenstigate whether there's a better way to support type=ethernet, which does not involve QEMU itself running the scripts, thus avoiding need for it to have CAP_NET_ADMIN
TUNSETIFF is only called with type=ethernet when no precreated device is passed in (as via <target dev='mytap0'/>). Thus, CAP_NET_ADMIN is not quite so unconditionally needed for type=ethernet. My proposed fix for this is to have libvirt create a vnet%d device by default for type=ethernet just as it does for type=network. Doing this without any other changes _does_ mean that the network script provided by the user would be invoked without much by way of privileges. However, this is not an unworkable environment for the author of a qemu-ifup script; various privilege escalation mechanisms are workable. That said, precreating a vnet%d device, executing the ifup scripts within libvirt before calling qemu at all, and then invoking qemu with script=off and that precreated ethernet device passed as it would be for a device specified in target/@dev seems like a reasonable approach as well.
Passing a pre-created TAP device only avoids that first CAP_NET_ADMIN check though. Pretty much any networking related command (iptables, route, ip, brctl) is going to need CAP_NET_ADMIN, which means even with a pre-created TAP device there's not much a ifup script will be able todo.
There are plenty of means for an ifup script to escalate its own privileges -- means which require enough work that the system administrator must clearly want them there (arguably a feature, not a bug, from the security perspective). Examples: - Signal an external process with escalated privileges to perform the desired actions. - Use sudo with a tailored, targeted sudoers entry (I'm actually surprised that this works with the capabilities limited, but it's what I use in practice, and the script invoked by sudo works just fine for things I would expect to require CAP_NET_ADMIN, including adding iptables rules). I'm not arguing that requiring the ifup script to do its own privilege escalation is necessarily the right thing to do -- that's your call. I'm arguing only that it's a better state of affairs than what we have now.
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux major release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Major release. This request is not yet committed for inclusion.
We need this backported as the minimum effort "fix" http://libvirt.org/git/?p=libvirt.git;a=commit;h=8b5bc6c4790e7b269ec6360e2000bc5a9503a8da
Created attachment 422968 [details] Add qemu.conf option for clearing capabilities If the user disables this option (as in, qemu capabilities are not altered), then interface scripts should be able to work.
libvirt-0_8_1-9_el6 has been built in RHEL-6-candidate with the fix. Dave
Still have the error. Error detail: Traceback (most recent call last): File "/usr/share/virt-manager/virtManager/engine.py", line 878, in run_domain vm.startup() File "/usr/share/virt-manager/virtManager/domain.py", line 1313, in startup self._backend.create() File "/usr/lib64/python2.6/site-packages/libvirt.py", line 333, in create if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self) libvirtError: internal error Process exited while reading console log output: char device redirected to /dev/pts/3 warning: could not configure /dev/net/tun: no virtual network emulation qemu-kvm: -netdev tap,script=/etc/qemu-ifup,id=hostnet1: Device 'tap' could not be initialized Related pkg version: kernel-2.6.32-54.el6.x86_64 libvirt-0.8.1-20.el6.x86_64 qemu-kvm-0.12.1.2-2.104.el6.x86_64 virt-manager-0.8.4-7.el6.noarch
The partial fix is to clarify the comment in qemu.conf that disabling clear_emulator_capabilities will allow this operation to succeed. Did you disable clear_emulator_capabilities in qemu.conf?
Fixing this issue requires changing an option in /etc/libvirt/qemu.conf. QE needs to set clear_emulator_capabilities = 1 and restart libvirtd. Moving back to ON_QA Also, requesting a release note, since this use case will not work out of the box.
Technical note added. If any revisions are required, please edit the "Technical Notes" field accordingly. All revisions will be proofread by the Engineering Content Services team. New Contents: Libvirt's default behavior of dropping QEMU/KVM process capabilities prevents <interface type='ethernet'/> from working correctly, and will error with: could not configure /dev/net/tun: no virtual network emulation. To libvirt's default behavior: Edit /etc/libvirt/qemu.conf, change clear_emulator_capabilities = 1 service libvirtd restart This has security implications.
I have some problem regarding this as currently my qemu.conf has hash (#) above entry as i start vm's by manually running qemu-kvm command instead of virsh start and passing network scripts same above error come "could not configure /dev/net/tun: no virtual network emulation" but one tap1 interface is created/working f9, but if i add another interface tap2 for eth1 on vm, its not working , no traffic out from eth1 and problem is arp table has no valid entry for this on host, so can you please tell, why it fails for secondary tap interface for eth1 on guest vms as its work f9 on one tap1 for eth0, as same thing is working in past. Thanks Muzi
Can you know how it would be resolve ??? And what do to know for activating secondary interface eth1 on guest vms with out effecting security implications ??? Thanks Muzi
Re-test this bug with libvirt-0.8.1-25.el6.x86_64. The following is my steps: 1. Edit /etc/libvirt/qemu.conf, change the following lines from # clear_emulator_capabilities = 1 to clear_emulator_capabilities = 0 from #user = "root" to user = "jialiu" NOTE: jialiu is belonged to root group Allow jialiu to run any command via sudo, add a line with visudo command: jialiu ALL=(ALL) ALL 2. Restart libvirtd service. 3. Set selinux to permissive. 4. Edit a domain xml as following: .... <interface type='ethernet'> <mac address='52:54:00:d8:05:14'/> <script path='/etc/my-qemu-ifup'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> .... 4. Prepare the script - /etc/my-qemu-ifup # ll /etc/my-qemu-ifup -rwxrwxrwx. 1 root root 72 Aug 11 02:41 /etc/my-qemu-ifup # cat /etc/my-qemu-ifup #!/bin/sh sudo /sbin/ifconfig $1 0.0.0.0 up sudo /usr/sbin/brctl addif virbr0 $1 5. Try to start the domain # virsh start rhel6_x86_64 error: Failed to start domain rhel6_x86_64 error: internal error Process exited while reading console log output: char device redirected to /dev/pts/3 warning: could not configure /dev/net/tun: no virtual network emulation qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initialized If I set user = "root" in /etc/libvirt/qemu.conf in step 1, the guest is started successfuly. I am not sure this behaviour is expected, or I miss some steps???
Your ifup script only runs the ifconfig + brctl commands privileged, QEMU itself is still unprivileged & thus can't create the initial tap device. You need to have 'user=root' for this to work.
Verify this bug with libvirt-0.8.1-27.el6.x86_64, and PASSED. According comment 20, the following is my steps: 1. Edit /etc/libvirt/qemu.conf, change the following lines from # clear_emulator_capabilities = 1 to clear_emulator_capabilities = 0 from #user = "root" to user = "root" 2. Restart libvirtd service. 3. Set selinux to permissive. 4. Edit a domain xml as following: .... <interface type='ethernet'> <mac address='52:54:00:d8:05:14'/> <script path='/etc/my-qemu-ifup'/> <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface> .... 4. Prepare the script - /etc/my-qemu-ifup # ll /etc/my-qemu-ifup -rwxrwxrwx. 1 root root 72 Aug 11 02:41 /etc/my-qemu-ifup # cat /etc/my-qemu-ifup #!/bin/sh /sbin/ifconfig $1 0.0.0.0 up /usr/sbin/brctl addif virbr0 $1 5. Start the domain # virsh start rhel6_x86_64 Domain rhel6_x86_64 started
Johnny Liu kindly confirm are you sure if we use your above steps in qemu.conf, then its not effect on any security consideration ?? like as mention in Comment #2 by Daniel Berrange Kindly suggest if it has no security consideration as per this thread and you recommended to use above configuration to resolve this issue. ?? Thank Muzi
> Kindly suggest if it has no security consideration as per this thread and you > recommended to use above configuration to resolve this issue. ?? Setting clear_emulator_capabilities=0 and user=root *WILL LOWER* security protection against a compromised QEMU process. We strongly recommend *against* using this <interface type='ethernet'> guest setup for this reason. Go for either a standard bridging or NAT config instead as per http://wiki.libvirt.org/page/Networking
Thanks for the suggestion, but i am using routed networking , instead of NAT/Bridging, as my guests are on public ip, and i need routing to route ips some time, so i cant afford bridging also. I think its lack in qemu-kvm who not support the routed networking now from FC12, its a problem for big and routing base networks. Thanks Mui
Kindly verify, my vms has has secondary ethernet eth1, is stop working, as in past its working f9, secondary interface type = ethernet , as some details i mentioned in comment #17, i just want to confirm either this emulator restrictions effect this, i guess its not effect with this, as eth0 is working f9, but eth1 is unable to communicate, after troubleshoot, its kvm host, not define the arp entry against this interface and then cant communicate, I also open a separate bug for this, but sill no positive response from there, so asking here to confirm this, either its effected with emulator or not. Details are mention below https://bugzilla.redhat.com/show_bug.cgi?id=620246 Please advice, we are badly suffer from this problem. Thanks Muzi
Red Hat Enterprise Linux 6.0 is now available and should resolve the problem described in this bug report. This report is therefore being closed with a resolution of CURRENTRELEASE. You may reopen this bug report if the solution does not work for you.
(In reply to comment #21) > Verify this bug with libvirt-0.8.1-27.el6.x86_64, and PASSED. > > According comment 20, the following is my steps: > 1. Edit /etc/libvirt/qemu.conf, change the following lines > from > # clear_emulator_capabilities = 1 > to > clear_emulator_capabilities = 0 > > from > #user = "root" > to > user = "root" No need to disable emulator_capabilities, i verified it using clear_emulator_capabilities = 1 and its working.