Bug 698071

Summary: Failed to start lxc guest with 'Unknown failure'
Product: Red Hat Enterprise Linux 6 Reporter: zhanghaiyan <yoyzhang>
Component: libvirtAssignee: Jiri Denemark <jdenemar>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.1CC: ajia, berrange, dallan, dyuan, eblake, jyang, llim, mshao, mzhan
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.9.1-1.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-12-06 11:05:36 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
libvirtd.log none

Description zhanghaiyan 2011-04-20 05:20:22 UTC
Description of problem:
Failed to start lxc guest with 'Unknown failure'

Version-Release number of selected component (if applicable):
- kernel-2.6.32-131.0.1.el6.x86_64
- libvirt-0.8.7-18.el6.x86_64

How reproducible:
always

Steps to Reproduce:
1. # cat vm1.xml 
<domain type='lxc'>
 <name>vm1</name>
 <memory>500000</memory>
 <os>
  <type>exe</type>
  <init>/bin/sh</init>
 </os>
 <vcpu>1</vcpu>
 <clock offset='utc'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>destroy</on_crash>
 <devices>
  <emulator>/usr/libexec/libvirt_lxc</emulator>
  <interface type='network'>
   <source network='default'/>
  </interface>
  <console type='pty' />
 </devices>
</domain>
2. # virsh -c lxc:///
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # define vm1.xml 
Domain vm1 defined from vm1.xml

virsh # start vm1
error: Failed to start domain vm1
error: Unknown failure

virsh # 

  
Actual results:
Failed to start lxc guest

Expected results:
Succeed to start lxc guest 

Additional info:

Comment 4 Osier Yang 2011-04-20 08:07:35 UTC
It looks failed on creating peer network, could you please check
what's the output of "ip link show" on your testing box? and also debug log?

Comment 5 zhanghaiyan 2011-04-20 08:29:34 UTC
# ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:25:64:a7:1f:4d brd ff:ff:ff:ff:ff:ff
3: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:54:00:b7:7f:5e brd ff:ff:ff:ff:ff:ff
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 52:54:00:b7:7f:5e brd ff:ff:ff:ff:ff:ff

# tail -f /var/log/message
Apr 20 04:29:10 localhost lldpad[1478]: evb_ifdown:port veth1 remove failed
Apr 20 04:29:10 localhost lldpad[1478]: vdp_ifdown:veth1 vdp data remove failed
Apr 20 04:29:10 localhost lldpad[1478]: evb_ifdown:port veth0 remove failed
Apr 20 04:29:10 localhost lldpad[1478]: vdp_ifdown:veth0 vdp data remove failed
Apr 20 04:29:10 localhost NetworkManager[1532]: <warn> /sys/devices/virtual/net/veth0: couldn't determine device driver; ignoring...
Apr 20 04:29:10 localhost NetworkManager[1532]: <warn> /sys/devices/virtual/net/veth1: couldn't determine device driver; ignoring...

Comment 6 zhanghaiyan 2011-04-20 08:30:05 UTC
Created attachment 493375 [details]
libvirtd.log

Comment 7 Osier Yang 2011-04-20 08:40:31 UTC
<snip>
04:31:31.245: 3491: debug : lxcSetupInterfaces:1089 : bridge: virbr0
04:31:31.245: 3491: debug : lxcSetupInterfaces:1096 : calling vethCreate()
04:31:31.245: 3491: debug : vethCreate:101 : veth1: (null) veth2: (null)
04:31:31.245: 3491: debug : vethCreate:107 : Assigned veth1: veth12
04:31:31.245: 3491: debug : vethCreate:120 : Assigned veth2: veth13
04:31:31.245: 3491: debug : vethCreate:124 : veth1: veth12 veth2: veth13
04:31:31.245: 3491: debug : virRunWithHook:833 : ip link add veth12 type veth peer name veth13
04:31:31.250: 3491: debug : virDomainFree:2294 : dom=0x7ff8b0000940, (VM: name=vm1, uuid=5e93c129-c15e-eaba-9769-15034e9a9f1d),
</snip>

Above is the useful log in the attachment, it exposes two problems, 1) problem on creating the peer network, 2) no useful error message when "vethCreate()" fails.

Comment 8 Daniel Berrangé 2011-04-20 10:04:56 UTC
That will be fixed in this commit upstream.

http://libvirt.org/git/?p=libvirt.git;a=commitdiff;h=c59176c109457161861fd883111373410d3b3a66

It isn't actually a regression from 6.0, it is a non-deterministic bug, so has always existed, just not necessarily reproducable

Comment 9 Jiri Denemark 2011-04-20 14:33:44 UTC
Fixed upstream by v0.8.8-236-gc59176c:

commit c59176c109457161861fd883111373410d3b3a66
Author: Daniel P. Berrange <berrange>
Date:   Thu Mar 17 15:54:24 2011 +0000

    Fix uninitialized variable & error reporting in LXC veth setup
    
    THe veth setup in LXC had a couple of flaws, first brInit did
    not report any error when it failed. Second vethCreate() did
    not correctly initialize the variable containing the return
    code, so could report failure even when it succeeded.

Comment 12 zhanghaiyan 2011-05-25 09:09:08 UTC
Verified this bug pass with libvirt-0.9.1-1.el6.x86_64
# virsh -c lxc:///
Welcome to virsh, the virtualization interactive terminal.

Type:  'help' for help with commands
       'quit' to quit

virsh # define vm1.xml 
Domain vm1 defined from vm1.xml

virsh # start vm1
Domain vm1 started

virsh # list 
 Id Name                 State
----------------------------------
679 vm1                  running

virsh # dumpxml vm1
<domain type='lxc' id='679'>
  <name>vm1</name>
  <uuid>627d217b-18a1-7ab4-6e8a-2aea39484a31</uuid>
  <memory>500000</memory>
  <currentMemory>500000</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/bin/sh</init>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <interface type='network'>
      <mac address='52:54:00:1a:cf:bd'/>
      <source network='default'/>
      <target dev='veth0'/>
    </interface>
    <console type='pty' tty='/dev/pts/1'>
      <source path='/dev/pts/1'/>
      <target type='serial' port='0'/>
    </console>
  </devices>
</domain>

virsh # exit

Comment 14 Min Zhan 2011-07-18 07:16:54 UTC
Move to Verified according to Comment #12

Comment 15 errata-xmlrpc 2011-12-06 11:05:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1513.html