The type=ethernet  mode of networking is not supported when libvirt is either compiled with capng, or qemu is running non-root. 

This is because TUNSETIFF requires CAP_NET_ADMIN. No non-root process has this, and when using capng, we further block this when qemu is running as root, or the setup script is setuid, because having CAP_NET_ADMIN would be a significant security threat.

We need to invenstigate whether there's a better way to support type=ethernet, which does not involve QEMU itself running the scripts, thus avoiding need for it to have CAP_NET_ADMIN

TUNSETIFF is only called with type=ethernet when no precreated device is passed in (as via <target dev='mytap0'/>). Thus, CAP_NET_ADMIN is not quite so unconditionally needed for type=ethernet. My proposed fix for this is to have libvirt create a vnet%d device by default for type=ethernet just as it does for type=network.

Doing this without any other changes _does_ mean that the network script provided by the user would be invoked without much by way of privileges. However, this is not an unworkable environment for the author of a qemu-ifup script; various privilege escalation mechanisms are workable.

That said, precreating a vnet%d device, executing the ifup scripts within libvirt before calling qemu at all, and then invoking qemu with script=off and that precreated ethernet device passed as it would be for a device specified in target/@dev seems like a reasonable approach as well.

Passing a pre-created TAP device only avoids that first CAP_NET_ADMIN check though. Pretty much any networking related command (iptables, route, ip, brctl) is going to need CAP_NET_ADMIN, which means even with a pre-created TAP device there's not much a ifup script will be able todo.

There are plenty of means for an ifup script to escalate its own privileges -- means which require enough work that the system administrator must clearly want them there (arguably a feature, not a bug, from the security perspective).

Examples:

 - Signal an external process with escalated privileges to perform the desired actions.

 - Use sudo with a tailored, targeted sudoers entry (I'm actually surprised that this works with the capabilities limited, but it's what I use in practice, and the script invoked by sudo works just fine for things I would expect to require CAP_NET_ADMIN, including adding iptables rules).

I'm not arguing that requiring the ifup script to do its own privilege escalation is necessarily the right thing to do -- that's your call. I'm arguing only that it's a better state of affairs than what we have now.

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

We need this backported as the minimum effort "fix"

http://libvirt.org/git/?p=libvirt.git;a=commit;h=8b5bc6c4790e7b269ec6360e2000bc5a9503a8da

Created attachment 422968 [details]
Add qemu.conf option for clearing capabilities

If the user disables this option (as in, qemu capabilities are not altered), then interface scripts should be able to work.

libvirt-0_8_1-9_el6 has been built in RHEL-6-candidate with the fix.

Dave

Still have the error. Error detail:
Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/engine.py", line 878, in run_domain
    vm.startup()
  File "/usr/share/virt-manager/virtManager/domain.py", line 1313, in startup
    self._backend.create()
  File "/usr/lib64/python2.6/site-packages/libvirt.py", line 333, in create
    if ret == -1: raise libvirtError ('virDomainCreate() failed', dom=self)
libvirtError: internal error Process exited while reading console log output: char device redirected to /dev/pts/3
warning: could not configure /dev/net/tun: no virtual network emulation
qemu-kvm: -netdev tap,script=/etc/qemu-ifup,id=hostnet1: Device 'tap' could not be initialized

Related pkg version:
kernel-2.6.32-54.el6.x86_64
libvirt-0.8.1-20.el6.x86_64
qemu-kvm-0.12.1.2-2.104.el6.x86_64
virt-manager-0.8.4-7.el6.noarch

The partial fix is to clarify the comment in qemu.conf that disabling clear_emulator_capabilities will allow this operation to succeed.  Did you disable clear_emulator_capabilities in qemu.conf?

Fixing this issue requires changing an option in /etc/libvirt/qemu.conf. QE needs to set

clear_emulator_capabilities = 1

and restart libvirtd. Moving back to ON_QA

Also, requesting a release note, since this use case will not work out of the box.

Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.

New Contents:
Libvirt's default behavior of dropping QEMU/KVM process capabilities prevents <interface type='ethernet'/> from working correctly, and will error with:

could not configure /dev/net/tun: no virtual network emulation.

To libvirt's default behavior:

Edit /etc/libvirt/qemu.conf, change clear_emulator_capabilities = 1

service libvirtd restart

This has security implications.

I have some problem regarding this as currently my qemu.conf has hash (#) above entry as i start vm's by manually running qemu-kvm command instead of virsh start and passing network scripts same above error come "could not configure /dev/net/tun: no virtual network emulation" but one tap1 interface is created/working f9, but if i add another interface tap2 for eth1 on vm, its not working , no traffic out from eth1 and problem is arp table has no valid entry for this on host, so can you please tell, why it fails for secondary tap interface for eth1 on guest vms as its work f9 on one tap1 for eth0, as same thing is working in past.

Thanks
Muzi

Can you know how it would be resolve ??? And what do to know for activating secondary interface eth1 on guest vms with out effecting security implications ???

Thanks
Muzi

Re-test this bug with libvirt-0.8.1-25.el6.x86_64. The following is my steps:
1. Edit /etc/libvirt/qemu.conf, change the following lines
   from
   # clear_emulator_capabilities = 1
   to
   clear_emulator_capabilities = 0

   from
   #user = "root"
   to
   user = "jialiu"

   NOTE:
   jialiu is belonged to root group
   Allow jialiu to run any command via sudo, add a line with visudo command:
   jialiu  ALL=(ALL)       ALL

2. Restart libvirtd service.
3. Set selinux to permissive.
4. Edit a domain xml as following:
   ....
    <interface type='ethernet'>
      <mac address='52:54:00:d8:05:14'/>
      <script path='/etc/my-qemu-ifup'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    ....
4. Prepare the script - /etc/my-qemu-ifup
# ll /etc/my-qemu-ifup
-rwxrwxrwx. 1 root root 72 Aug 11 02:41 /etc/my-qemu-ifup
# cat /etc/my-qemu-ifup
#!/bin/sh
sudo /sbin/ifconfig $1 0.0.0.0 up
sudo /usr/sbin/brctl addif virbr0 $1
      
5. Try to start the domain
# virsh start rhel6_x86_64
error: Failed to start domain rhel6_x86_64
error: internal error Process exited while reading console log output: char device redirected to /dev/pts/3
warning: could not configure /dev/net/tun: no virtual network emulation
qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initialized


If I set user = "root" in /etc/libvirt/qemu.conf in step 1, the guest is started successfuly.
I am not sure this behaviour is expected, or I miss some steps???

Your ifup script only runs the ifconfig + brctl commands privileged, QEMU itself is still unprivileged & thus can't create the initial tap device. You need to have 'user=root' for this to work.

Verify this bug with libvirt-0.8.1-27.el6.x86_64, and PASSED.

According comment 20, the following is my steps:
1. Edit /etc/libvirt/qemu.conf, change the following lines
   from
   # clear_emulator_capabilities = 1
   to
   clear_emulator_capabilities = 0

   from
   #user = "root"
   to
   user = "root"

2. Restart libvirtd service.
3. Set selinux to permissive.
4. Edit a domain xml as following:
   ....
    <interface type='ethernet'>
      <mac address='52:54:00:d8:05:14'/>
      <script path='/etc/my-qemu-ifup'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
    </interface>
    ....
4. Prepare the script - /etc/my-qemu-ifup
# ll /etc/my-qemu-ifup
-rwxrwxrwx. 1 root root 72 Aug 11 02:41 /etc/my-qemu-ifup
# cat /etc/my-qemu-ifup
#!/bin/sh
/sbin/ifconfig $1 0.0.0.0 up
/usr/sbin/brctl addif virbr0 $1

5. Start the domain
# virsh start rhel6_x86_64
Domain rhel6_x86_64 started

Johnny Liu kindly confirm are you sure if we use your above steps in qemu.conf, then its not effect on any security consideration ?? like as mention in Comment #2 by Daniel Berrange 

Kindly suggest if it has no security consideration as per this thread and you recommended to use above configuration to resolve this issue. ??

Thank
Muzi

> Kindly suggest if it has no security consideration as per this thread and you
> recommended to use above configuration to resolve this issue. ??

Setting clear_emulator_capabilities=0 and user=root  *WILL LOWER* security protection against a compromised QEMU process. We strongly recommend *against* using this <interface type='ethernet'> guest setup for this reason. Go for either a standard bridging or NAT config instead as per  http://wiki.libvirt.org/page/Networking

Thanks for the suggestion, but i am using routed networking , instead of NAT/Bridging, as my guests are on public ip, and i need routing to route ips some time, so i cant afford bridging also. I think its lack in qemu-kvm who not support the routed networking now from FC12, its a problem for big and routing base networks.

Thanks
Mui

Kindly verify, my vms has has secondary ethernet eth1, is stop working, as in past its working f9, secondary interface type = ethernet , as some details i mentioned in comment #17, i just want to confirm either this emulator restrictions effect this, i guess its not effect with this, as eth0 is working f9, but eth1 is unable to communicate, after troubleshoot, its kvm host, not define the arp entry against this interface and then cant communicate, I also open a separate bug for this, but sill no positive response from there, so asking here to confirm this, either its effected with emulator or not.

Details are mention below 

https://bugzilla.redhat.com/show_bug.cgi?id=620246

Please advice, we are badly suffer from this problem.

Thanks
Muzi

Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.

(In reply to comment #21)
> Verify this bug with libvirt-0.8.1-27.el6.x86_64, and PASSED.
> 
> According comment 20, the following is my steps:
> 1. Edit /etc/libvirt/qemu.conf, change the following lines
>    from
>    # clear_emulator_capabilities = 1
>    to
>    clear_emulator_capabilities = 0
> 
>    from
>    #user = "root"
>    to
>    user = "root"


No need to disable emulator_capabilities, i verified it using clear_emulator_capabilities = 1 and its working.