Bug 646403

+++ This bug was initially created as a clone of Bug #532710 +++

Description of problem:
Setup:
HOST: F11 x86_64
GUEST: F12beta x86_64

I have a number of guests setup on this host and all connect without a problem with the 'br0' bridge on the host.  I created a new guest and loaded F12beta on it.  I defined it to also use 'br0' networking but this guest cannot ping anything on the LAN.  All the other guests have no problems communicating to addresses on the LAN.
Details:

Version-Release number of selected component (if applicable):


How reproducible:
always

Steps to Reproduce:
1. In F11 Host, create an F12beta guest and use bridged-networking (br0)
2.
3.
  
Actual results:
F12beta guest cannot address the LAN

Expected results:
F12beta guest can address the LAN


Additional info:

firewall is down in guest

### on HOST:
# brctl showmacs br0
port no mac addr                is local?       ageing timer
 1     00:0b:82:11:bc:cb       no                15.88
 3     00:0c:01:30:50:00       no                16.37
 2     00:0c:01:40:50:00       no                 2.51
 1     00:0c:29:e3:bc:ee       no                 0.49
 1     00:18:f8:48:27:15       no                 2.51
 1     00:18:f8:9b:ba:b4       no               116.39
 1     00:18:f8:9b:ba:b5       no                20.60
 1     00:1a:4d:5e:f6:36       no               156.43
 1     00:24:1d:19:05:cf       yes                0.00
 3     06:1d:1d:7a:31:9b       yes                0.00
 2     06:ba:11:dd:7b:08       yes                0.00
 1     54:52:00:24:5e:cf       no                28.93

# EXISTING GUEST: network interface in xml file:
   <interface type='bridge'>
     <mac address='00:0c:01:30:50:00'/>
     <source bridge='br0'/>
     <target dev='vnet0'/>
     <model type='virtio'/>
   </interface>

# NEW GUEST: network interface in xml file:
   <interface type='bridge'>
     <mac address='00:0c:01:40:50:00'/>
     <source bridge='br0'/>
     <target dev='vnet0'/>
     <model type='virtio'/>
   </interface>

### on EXISTING GUEST:
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
HWADDR=00:0c:01:30:50:00
IPADDR=192.168.1.200
NETMASK=255.255.255.0
NETWORK=192.168.1.0

### on NEW GUEST:
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
HWADDR=00:0c:01:40:50:00
IPADDR=192.168.1.210
NETMASK=255.255.255.0
GATEWAY=192.168.1.1
NETWORK=192.168.1.0
TYPE=Ethernet
IPV6INIT=no
USERCTL=no

### on EXISTING GUEST:
# ping 192.168.1.1
succeeds

### on NEW GUEST:
# ping 192.168.1.1
fails with Destination Host Unreachable

--- Additional comment from greno on 2009-11-04 11:54:13 EST ---

F11 Host:
libvirt-python-0.6.2-18.fc11.x86_64
libvirt-devel-0.6.2-18.fc11.x86_64
libvirt-0.6.2-18.fc11.x86_64

F12beta Guest:
libvirt-client-0.7.1-11.fc12.x86_64
libvirt-python-0.7.1-11.fc12.x86_64

# EXISTING WORKING F11 GUEST
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin /usr/bin/qemu-kvm -S -M pc -cpu qemu32 -m 2048 -smp 4 -name WEB_1 -uuid 092f3a1f-238e-4975-1b1b-4017503d02ba -monitor pty -pidfile /var/run/libvirt/qemu//WEB_1.pid -boot c -drive file=/var/vm/vm0/qemu/images/WEB_1/WEB_1.img,if=ide,index=0,boot=on -net nic,macaddr=00:0c:01:30:50:00,vlan=0,model=virtio -net tap,fd=25,vlan=0 -serial none -parallel none -usb -vnc 127.0.0.1:1 -k en-us 
char device redirected to /dev/pts/6

# NEW F12beta GUEST
LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin /usr/bin/qemu-kvm -S -M pc -m 512 -smp 4 -name CLOUD_1 -uuid 367b5bca-132d-4d50-23d8-87b09d9cceae -monitor pty -pidfile /var/run/libvirt/qemu//CLOUD_1.pid -boot c -drive file=,if=ide,media=cdrom,index=2 -drive file=/var/lib/libvirt/images/CLOUD_1.img,if=virtio,index=0,boot=on -net nic,macaddr=00:0c:01:40:50:00,vlan=0,model=virtio -net tap,fd=23,vlan=0 -serial pty -parallel none -usb -vnc 127.0.0.1:0 -soundhw es1370


The only thing I can find is that the new F12beta guest does not have 'bridge' module loaded in kernel as compared with a working existing F11 guest.  I tried loading the 'bridge' module but this did not make any difference after restarting the network.  I then tried rebooting the guest but then the 'bridge' module is not loaded again.  Is the 'bridge' module necessary?  I thought I had read where this was no longer necessary for guest bridging.

--- Additional comment from greno on 2009-11-04 22:52:40 EST ---

The only way I can get bridged networking to work in the F12beta guest is by selecting "Hypervisor default" when adding Network device.  If I select "virtio" it fails.

--- Additional comment from fedora-triage-list on 2009-11-16 09:59:48 EST ---


This bug appears to have been reported against 'rawhide' during the Fedora 12 development cycle.
Changing version to '12'.

More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

--- Additional comment from markmc on 2009-11-19 11:39:11 EST ---

Thanks for the report Gene

Could you try again with a F12-GA guest? I've just installed a guest connected to a bridge and it seems to be fine

--- Additional comment from greno on 2009-11-19 17:49:55 EST ---

The issue is with F11 host trying to install F12 guest with a bridge and virtio.  It will work only if you use "Hypervisor default" but not when you try to use "virtio".

The whole point was to setup a VM to test out new Fedora developments.  So the host will always be at least one release back.  And the throughput without virtio is too slow so we need virtio.

Did you test using F11 as the host and F12 guest with virtio bridge?

If yes, then I will be glad to retest.

--- Additional comment from markmc on 2009-11-20 12:51:47 EST ---

Ah, I missed that the host is F-11 - since the bug is against F-12 libvirt

I have a F-12 GA guest installing over a bridge on an F-11 host here now, it seems to be fine

I think trying a new F-12 install is a good first step

Next step, if it's still happening with that guest, is to try and get more data e.g.

  - log file from /var/log/libvirt/

  - output of the commands in

     http://fedoraproject.org/wiki/How_to_debug_Virtualization_problems#Networking

  - anything in dmesg in the guest?

  - does using tcpdump/wireshark on vnet0/br0 in the host and eth0 in the
    guest show the packets? if not, at which stage are the packets dropped?

(Just to note - one difference between the two guests is '-m 2048 -cpu qemu32' for the F-11 guest and '-m 512' for the F-12 guest. I doubt that's related, though)

--- Additional comment from kimmo.vuorinen on 2009-11-23 14:46:11 EST ---

There might be a problem with bridging in F11 too when using virtio. I just upgraded some of my systems from F9 to F11 and I have setup with four guests which all have at least 2 virtio nic:s. Bridges connect these interfaces to physical interfaces at dom0.

First bridge has been working ok, but second one doesn't pass network traffic to guests at all. Second bridge worked ok for a while without any configuration changes but suddenly stopped working, reboot didn't help.

My configuration has dom0 F11 with kernel-2.6.31.5-127.fc12.i686, three F11 guests and 1 F9 guest.

brctl showmac displays something strange:
bridge eth0, works ok:
port no	mac addr		is local?	ageing timer
  1	00:15:99:03:60:19	no		   1.05
  3	00:16:3e:17:29:19	no		   0.02
  2	00:16:3e:17:45:81	no		   0.49
  4	00:16:3e:18:99:13	no		   0.39
  1	00:19:db:f5:cb:8f	yes		   0.00
  1	00:1d:92:63:3e:ea	no		  34.65
  1	00:23:54:3a:06:ce	no		   0.00
  3	72:8e:45:17:e4:3e	yes		   0.00
  2	86:4a:2c:66:54:1a	yes		   0.00
  4	9e:bf:e6:26:2b:81	yes		   0.00

bridge eth1, look at the port numbers:
port no	mac addr		is local?	ageing timer
  1	00:03:fa:91:0c:be	no		   1.24
  1	00:0f:bb:19:75:7e	no		  20.42
  1	00:10:a7:05:a8:c2	yes		   0.00
  1	00:16:3e:16:13:92	no		   2.10     <- 4?
  1	00:16:3e:26:12:30	no		   3.05     <- 3?
  1	00:16:3e:32:22:93	no		   5.10     <- 2?
  3	02:ae:ce:e4:6b:4c	yes		   0.00
  2	96:cf:ec:15:ee:49	yes		   0.00
  4	a2:99:62:16:cf:34	yes		   0.00

I haven't found out if I can change there ports in userspace.

dmesg from dom0:
peth1: received packet with own address as source address

dmesg from guest doesn't tell anything regarding this and I am not using libvirt.

--- Additional comment from kimmo.vuorinen on 2009-11-24 08:12:25 EST ---

I upgraded host and one of the guests to F12 and problem remains, so problem lies most likely in kernel-2.6.31.5-127.fc12.i686.

More info about my configuration:

ifcfg-peth1:
DEVICE=peth1
TYPE=Ethernet
ONBOOT=yes
HWADDR=00:10:a7:05:a8:c2
BRIDGE=eth1

ifcfg-eth1:
DEVICE=eth1
TYPE=Bridge
DELAY=0
ONBOOT=yes
BOOTPROTO=dhcp
PEERDNS=no

/etc/qemu-ifup:
#!/bin/sh
if [ `echo $1 | egrep "vif[[:digit:]].0"` ]; then
	ifconfig $1 0.0.0.0 up
	brctl addif eth0 $1
fi

if [ `echo $1 | egrep "vif[[:digit:]].1"` ]; then
	ifconfig $1 0.0.0.0 up
	brctl addif eth1 $1
fi

if [ `echo $1 | egrep "vif[[:digit:]].2"` ]; then
	ifconfig $1 0.0.0.0 up
	brctl addif wlan0 $1
fi

guest commandline:
#!/bin/sh
qemu-kvm -name joshua.xxxxx.xxx -m 512 -k fi -drive file=/dev/Enigma/Joshua,if=virtio,boot=on \
-net nic,vlan=0,macaddr=00:16:3E:18:99:13,model=virtio -net tap,vlan=0,ifname=vif3.0 \
-net nic,vlan=1,macaddr=00:16:3E:26:12:30,model=virtio -net tap,vlan=1,ifname=vif3.1 \
-vnc :3 -usb -usbdevice tablet -localtime -daemonize

--- Additional comment from kimmo.vuorinen on 2009-12-01 16:56:31 EST ---

I tested latest kernels for releases F10, F11 and F12 in F12 but problem persists.

my hosts sysctl.conf contains

# Controls IP packet forwarding
net.ipv4.ip_forward = 1

# Do not filter bridge traffic
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

so problem should't be in iptables configuration. Host is protected by firewalling bridge device and guests are protected by their own firewalls. I have also tried without firewalls.

--- Additional comment from kimmo.vuorinen on 2009-12-01 17:14:24 EST ---

I forgot to mention that traffic from quests goes out but responses stop at the bridge. Tested by dumping DHCP traffic in host from physical interface, bridge and virtual interface.

--- Additional comment from andy on 2009-12-29 08:50:40 EST ---

I had/have similar experience with an F11 guest on an F11 host and also an F12 guest on an F12 host.

In both cases, the install was a NFS install so the guests were able to read from the n/w for a sustained amount of time.

I now find that the n/w works for a short time, but disappears after a while.

For F12 I've also recently discovered that if the guest pings the host's external IP address then the network connectivity is restored to the guest.  Stopping the ping will bring the network down on the host again within a minute or two.

Setting a ping interval of 2s doesn't seem to be enough to keep the network alive.

I don't have the F11/F11 combination any more, so can't confirm the "workaround".

--- Additional comment from andy on 2009-12-31 18:31:11 EST ---

Sigh, it appears that pinging the hosts external IP only allows pinging other hosts, not other IP traffic.

--- Additional comment from fedora-admin-xmlrpc on 2010-03-09 11:53:12 EST ---

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

--- Additional comment from fedora-admin-xmlrpc on 2010-03-09 12:17:08 EST ---

This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

--- Additional comment from jforbes on 2010-03-11 13:45:01 EST ---

Is this still a problem with the most recent updates?  Note that qemu 0.12.3 should be in updates testing as well.

--- Additional comment from kimmo.vuorinen on 2010-03-13 18:54:38 EST ---

I am still facing this issue with kernel-2.6.32.9-70.fc12.i686 and qemu-0.12.3-3.fc14.i686. One of the three bridges that are configured work.

--- Additional comment from fedora-triage-list on 2010-04-28 07:04:10 EDT ---


This message is a reminder that Fedora 11 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 11.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '11'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 11's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 11 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

--- Additional comment from fedora-triage-list on 2010-06-28 11:22:44 EDT ---


Fedora 11 changed to end-of-life (EOL) status on 2010-06-25. Fedora 11 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

--- Additional comment from kimmo.vuorinen on 2010-10-19 15:02:51 EDT ---

I am still experiencing this problem with:

kernel-2.6.35.6-43.fc14.i686
qemu-0.13.0-0.7.rc1.fc14.i686

Guests are Fedora 9,13 and Windows. Two out of three bridges work, I still think that problem has something to do with brctl showmacs showing strange port numbers for the bridge which is not working.