Bug 770020

Summary:	virtual network of ethernet type guest can not start
Product:	Red Hat Enterprise Linux 6	Reporter:	Huang Wenlong <whuang>
Component:	libvirt	Assignee:	Laine Stump <laine>
Status:	CLOSED NOTABUG	QA Contact:	Virtualization Bugs <virt-bugs>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	6.3	CC:	acathrow, cwei, dallan, matthew.philpott, mzhan, rwu, xhu
Target Milestone:	rc
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-02-29 19:53:38 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Huang Wenlong 2011-12-23 03:10:13 UTC

Description of problem:
virtual network of ethernet type  guest can not start
Regression bug : https://bugzilla.redhat.com/show_bug.cgi?id=593903 
In the same environment  just downgrade to libvirt-0.8.1-27.el6.x86_64.rpm ,the guest can start . 

Version-Release number of selected component (if applicable):
libvirt-0.9.8-1.el6.x86_64 
qemu-kvm-0.12.1.2-2.211.el6.x86_64

How reproducible:
100%

Steps to Reproduce:

1. Edit /etc/libvirt/qemu.conf, change the following lines
   from
   # clear_emulator_capabilities = 1
   to
   clear_emulator_capabilities = 0

   from
   #user = "root"
   to
   user = "root"

2. Restart libvirtd service.
#service libvirtd restart

3. Set selinux to permissive.
#setenforce 0

4. Edit a domain xml as following:

   ....
    <interface type='ethernet'>
      <mac address='52:54:00:d8:05:14'/>
      <script path='/etc/my-qemu-ifup'/>
      <model type='virtio'/>
    </interface>
    ....

5. Prepare the script - /etc/my-qemu-ifup
# ll /etc/my-qemu-ifup
-rwxrwxrwx. 1 root root 72 Aug 11 02:41 /etc/my-qemu-ifup
# cat /etc/my-qemu-ifup
#!/bin/sh
/sbin/ifconfig $1 0.0.0.0 up
/usr/sbin/brctl addif virbr0 $1

6. Start the domain
# virsh start rhel6_x86_64
error: Failed to start domain t
error: internal error Process exited while reading console log output: char device redirected to /dev/pts/1
warning: could not open /dev/net/tun: no virtual network emulation
qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initialized



libvirtd.log

2011-12-15 09:34:25.010+0000: 2756: warning : qemuDomainObjTaint:1134 : Domain id=2 name='t' uuid=fc8ff82e-4bef-25f2-ce30-22fb9c560c1b is tainted: high-privileges
2011-12-15 09:34:25.010+0000: 2756: warning : qemuDomainObjTaint:1134 : Domain id=2 name='t' uuid=fc8ff82e-4bef-25f2-ce30-22fb9c560c1b is tainted: shell-scripts
2011-12-15 09:34:25.176+0000: 2756: error : qemuProcessReadLogOutput:1006 : internal error Process exited while reading console log output: char device redirected to /dev/pts/1
warning: could not open /dev/net/tun: no virtual network emulation
qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initialized



  
Actual results:
guest can not start 

Expected results:
guest can start

Additional info:

Comment 3 Laine Stump 2012-01-04 18:37:25 UTC

Can you post the qemu commandline generated by both the working and non-working libvirts? (from /var/log/libvirt/qemu/$guest.log)?

Also, is it necessary to downgrade all the way to 0.8.1? What about 0.8.7?

Comment 4 Huang Wenlong 2012-01-05 08:31:02 UTC

(In reply to comment #3)
> Can you post the qemu commandline generated by both the working and non-working
> libvirts? (from /var/log/libvirt/qemu/$guest.log)?
> 
> Also, is it necessary to downgrade all the way to 0.8.1? What about 0.8.7?

Hi,Laine

Sorry , there is a mistake of my environment  , if you want to reproduce this bug , need setenforce = 1  always, then  old libvirt (libvirt-0.8.7-18.el6.x86_64) and new libvirt (libvirt-0.9.8-1.el6.x86_64) will all fail  :
error: Failed to start domain t
error: internal error Process exited while reading console log output: char
device redirected to /dev/pts/1
warning: could not open /dev/net/tun: no virtual network emulation
qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could
not be initialized


the guest should be started with SElinux ,right ?  



Wenlong

Comment 5 Dave Allan 2012-01-05 20:19:28 UTC

(In reply to comment #4)
> (In reply to comment #3)
> > Can you post the qemu commandline generated by both the working and non-working
> > libvirts? (from /var/log/libvirt/qemu/$guest.log)?
> > 
> > Also, is it necessary to downgrade all the way to 0.8.1? What about 0.8.7?
> 
> Hi,Laine
> 
> Sorry , there is a mistake of my environment  , if you want to reproduce this
> bug , need setenforce = 1  always, then  old libvirt
> (libvirt-0.8.7-18.el6.x86_64) and new libvirt (libvirt-0.9.8-1.el6.x86_64) will
> all fail  :
> error: Failed to start domain t
> error: internal error Process exited while reading console log output: char
> device redirected to /dev/pts/1
> warning: could not open /dev/net/tun: no virtual network emulation
> qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could
> not be initialized
> 
> 
> the guest should be started with SElinux ,right ?  

No, it was explicitly stated in 593903 and the description of this BZ that selinux has to be disabled for this to work.  This is NOTABUG.

Comment 6 Huang Wenlong 2012-01-06 03:24:32 UTC

Hi, Dave

I can reproduce this bug after remove the Host's bridge device  , now guest can not start as bug description even setenforce = 0  , so  it is still a bug ,please assign it thanks . 


libvirt-0.9.8-1.el6.x86_64  qemu.log

2012-01-06 03:21:02.755+0000: starting up
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name q2 -uuid 07e8d4f7-7c14-29f4-8c33-83cd04dd5ad1 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/q2.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/var/lib/libvirt/images/q2.img,if=none,id=drive-ide0-0-0,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d8:05:14,bus=pci.0,addr=0x6 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
Domain id=3 is tainted: high-privileges
Domain id=3 is tainted: shell-scripts
char device redirected to /dev/pts/2
warning: could not open /dev/net/tun: no virtual network emulation
qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initialized
2012-01-06 03:21:03.061+0000: shutting down



libvirt-0.8.1-27.el6_0.6.x86_64    qemu.log 
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name q2 -uuid 07e8d4f7-7c14-29f4-8c33-83cd04dd5ad1 -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/q2.monitor,server,nowait -mon chardev=monitor,mode=control -rtc base=utc -boot c -drive file=/var/lib/libvirt/images/q2.img,if=none,id=drive-ide0-0-0,format=qcow2,cache=none -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d8:05:14,bus=pci.0,addr=0x6 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 
char device redirected to /dev/pts/2

and libvirt-0.8.7-18.el6.x86_64  will fail with same error 



Wenlong

Comment 7 Dave Allan 2012-01-06 17:13:51 UTC

Laine, can you reproduce this?

Comment 8 Laine Stump 2012-01-06 20:37:03 UTC

Okay, I've tried this with 3 different libvirt versions (all other packages on the host are identical:

  libvirt-0.8.1-27.el6_0.5 - WORKS
  libvirt-0.8.7-18         - FAILS
  libvirt-0.9.4-11         - FAILS

In all cases, selinux was se to to permissive, user:group in qemu.conf was set to root:root, and clear_emulator_capabilities was set to 0. The script used was identical to the script used by Wenlong, and was set to be executable (qemu fails to execute it otherwise.

When I compare the qemu commandlines, there doesn't seem to be any relevant changes between the working and non-working versions. This leads me to be believe there may be some problem with clearing capabilities when we shouldn't be, or something like that.

The simplest way to find the problem may be to bisect between 0.8.1 and 0.8.7+patches.

Comment 9 Dave Allan 2012-01-06 20:54:35 UTC

Yeah, sounds like it.

Comment 12 Matt 2012-02-10 09:30:50 UTC

I saw same thing on libvirt-0.9.10.

adding /dev/net/tun to cgroup in  /etc/libvirt/qemu.conf seems to fix the issue.

cgroup_device_acl = [
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc", "/dev/hpet", "/dev/net/tun",
]

Comment 13 Laine Stump 2012-02-29 15:58:57 UTC

Thanks for the suggestion, Matt!

I just tried it on a RHEL system and it does indeed solve the problem.

So, in summary, in order to use <interface type='ethernet'>, you must make the following changes to your system:

1) disable SELinux

2) in /etc/libvirt/qemu.conf add/edit the following lines:

  a) clear_emulator_capabilities = 0
  b) user = root
  c) group = root
  d)
     cgroup_device_acl = [
         "/dev/null", "/dev/full", "/dev/zero",
         "/dev/random", "/dev/urandom",
         "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
         "/dev/rtc", "/dev/hpet", "/dev/net/tun",
     ]

Since each of these steps is decreasing security on the system, we obviously can't configure things this way by default. The fix for this "bug" then, is to put this somewhere useful in documentation. What is the best place for that? Should this BZ be closed as NOTABUG, or should it be made dependent on some in-tree change to documentation?

Comment 14 Dave Allan 2012-02-29 17:20:00 UTC

I'd add it to the upstream Troubleshooting wiki.

Comment 15 Laine Stump 2012-02-29 19:53:38 UTC

Okay, I've added an entry to the troubleshooting wiki, and am including a link in case anyone in the future finds this BZ before the wiki:

http://wiki.libvirt.org/page/Guest_won%27t_start_-_warning:_could_not_open_/dev/net/tun_%28%27generic_ethernet%27_interface%29

(This is linked off of http://wiki.libvirt.org/page/Troubleshooting which is in general very useful reading).

Since it is already known that reconfiguration to decrease security protections is required to use interface type='ethernet', and we don't recommend its use in RHEL, I am closing this as NOTABUG.

Comment 16 Dave Allan 2012-07-25 13:38:25 UTC

Removing the Regression keyword since this is not a bug and therefore is not a regression either.