Bug 364581

Summary: libvirtd bridge virbr0 collides with Xen bridge xenbr0
Product: [Fedora] Fedora Reporter: Aleksander Adamowski <bugs-redhat>
Component: libvirtAssignee: Daniel Veillard <veillard>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 6CC: berrange, triage
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: bzcl34nup
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-05-06 19:47:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Aleksander Adamowski 2007-11-02 20:12:30 UTC
Description of problem:
After updating a FC5 Xen domain 0 installation to FC6, I've discovered that all
the guest domains have lost network connectivity.

After investigating the problem, it turned out that "brctl show" indicates two
bridges present in the system with the same bridge id.

The "virbr0" bridge had been connected to the physical eth0 device (peth0), and
got some default address based on /etc/libvirt/qemu/networks/default.xml settings.

The xenbr0 bridge got no physical connectivity, and all the virtual interfaces
of guest domains (vifX.Y) were gathered under it. So the domains obviously lost
network connectivity.

After disabling libvirtd service ("chkconfig libvirtd off") and restarting the
server I restored network access to guest domains.

Version-Release number of selected component (if applicable):
libvirt-0.3.1-1.fc6

Expected results:
libvirtd shouldn't create a bridge that "steals" physical network device from Xen.

Comment 1 Daniel Berrangé 2007-11-02 20:23:50 UTC
Libvirt doesn't put any physical device into virbr0. The whole point of virbr0
is that it is not attach to any physical device - its connectivity is IP
forwarding + NAT. I suspect a bug in Xen's script for setting up xenbr0 instead.

Please provide version number of 'xend', 'libvirt' RPMs

Then, use chkconfig to turn off the xend & libvirtd init scripts & reboot.

From this plain state run  'ifconfig -a', 'brctl show' and 'ip route -n' and
attach the output of these 4 commands.

Next, run  'service libvirtd start' and when it is done, capture 'ifconfig -a',
'brctl show' and 'ip route -n ' again.

Finally run  'service xend start' and when it is done , capture 'ifconfig -a',
'brctl show' and 'ip route -n ' again.

Attach all the data collected to this ticket.


Comment 2 Aleksander Adamowski 2007-11-28 05:21:43 UTC
I'll try but not very soon, since those are production servers and scheduled
downtime is very rare.

Comment 3 Bug Zapper 2008-04-04 07:39:53 UTC
Fedora apologizes that these issues have not been resolved yet. We're
sorry it's taken so long for your bug to be properly triaged and acted
on. We appreciate the time you took to report this issue and want to
make sure no important bugs slip through the cracks.

If you're currently running a version of Fedora Core between 1 and 6,
please note that Fedora no longer maintains these releases. We strongly
encourage you to upgrade to a current Fedora release. In order to
refocus our efforts as a project we are flagging all of the open bugs
for releases which are no longer maintained and closing them.
http://fedoraproject.org/wiki/LifeCycle/EOL

If this bug is still open against Fedora Core 1 through 6, thirty days
from now, it will be closed 'WONTFIX'. If you can reporduce this bug in
the latest Fedora version, please change to the respective version. If
you are unable to do this, please add a comment to this bug requesting
the change.

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we are following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

And if you'd like to join the bug triage team to help make things
better, check out http://fedoraproject.org/wiki/BugZappers

Comment 4 Bug Zapper 2008-05-06 19:47:54 UTC
This bug is open for a Fedora version that is no longer maintained and
will not be fixed by Fedora. Therefore we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen thus bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 5 Aleksander Adamowski 2008-06-13 13:51:21 UTC
For the record, one can workaround this problem by specifying the bridge to
which a guest domain's virtual interface will be assigned manually.

So this is a default generated virtual interface specification that would be put
into virbr0 in newer releases:

vif = [ 'mac=00:16:3e:67:64:11' ]

One can force it to continue using xenbr0 this way:
vif = [ 'mac=00:16:3e:67:64:11,bridge=xenbr0' ]

Comment 6 Aleksander Adamowski 2008-06-13 13:55:29 UTC
Now I have a state where I have all guest domains forced into xenbr0 brigde.
If I don't, some of them go to virbr0 and some into xenbr0. Some of those guests
run Fedora 8.


Here are requested values, from an RHEL5 this time:

# brctl show
bridge name     bridge id               STP enabled     interfaces
virbr0          8000.000000000000       no
xenbr0          8000.feffffffffff       no              vif12.0
                                                        vif11.0
                                                        vif2.0
                                                        vif1.0
                                                        peth0
                                                        vif0.0



# ip route 
192.168.254.0/24 dev eth0  proto kernel  scope link  src 192.168.254.209 
192.168.122.0/24 dev virbr0  proto kernel  scope link  src 192.168.122.1 
169.254.0.0/16 dev eth0  scope link 


# ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:1E:0B:78:26:C2  
          inet addr:192.168.254.209  Bcast:192.168.254.255  Mask:255.255.255.0
          inet6 addr: fe80::21e:bff:fe78:26c2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:210236462 errors:0 dropped:0 overruns:0 frame:0
          TX packets:102055769 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:283694505921 (264.2 GiB)  TX bytes:6963041304 (6.4 GiB)

eth1      Link encap:Ethernet  HWaddr 00:1E:0B:78:26:C0  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Interrupt:24 Memory:fc000000-fc012100 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:2253596 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2253596 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:392919987 (374.7 MiB)  TX bytes:392919987 (374.7 MiB)

peth0     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:228194442 errors:0 dropped:0 overruns:0 frame:0
          TX packets:126545820 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:289364816297 (269.4 GiB)  TX bytes:20547668210 (19.1 GiB)
          Interrupt:20 Memory:f8000000-f8012100 

sit0      Link encap:IPv6-in-IPv4  
          NOARP  MTU:1480  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

veth1     Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

veth2     Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

veth3     Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif0.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:102055770 errors:0 dropped:0 overruns:0 frame:0
          TX packets:210236462 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:6963041337 (6.4 GiB)  TX bytes:283694505921 (264.2 GiB)

vif0.1    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif0.2    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif0.3    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

vif1.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:1638005 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16183737 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:3066668627 (2.8 GiB)  TX bytes:3360154245 (3.1 GiB)

vif2.0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:17088990 errors:0 dropped:0 overruns:0 frame:0
          TX packets:27351785 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 

vif11.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:486 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1544 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:84411 (82.4 KiB)  TX bytes:338332 (330.4 KiB)

vif12.0   Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:25 errors:0 dropped:0 overruns:0 frame:0
          TX packets:882 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:1376 (1.3 KiB)  TX bytes:151606 (148.0 KiB)

virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2703 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:75684 (73.9 KiB)  TX bytes:468 (468.0 b)

xenbr0    Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF  
          inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:10288388 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1609766442 (1.4 GiB)  TX bytes:0 (0.0 b)



Comment 7 Daniel Berrangé 2008-06-13 14:03:22 UTC
If you don't explicitly list a 'bridge=XXXX' value in the 'vif' config, then it
is pot-luck which bridge device your guest will be connected to. There's nothing
that can be done about this - it is simply an incomplete config. The 'bridge='
param should always be specified.