Bug 542405

Summary: KVM guests on bridged network cannot ping each other when using e1000 vNIC
Product: [Fedora] Fedora Reporter: James Rankin <rankin.james>
Component: kvmAssignee: Glauber Costa <gcosta>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 12CC: akong, berrange, clalance, ehabkost, gcosta, jforbes, markmc, quintela, tburke, thiagofborn, virt-maint
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-24 03:11:29 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 514891    

Description James Rankin 2009-11-29 17:33:14 UTC
Description of problem:
Apologies if this should be filed against a different package.

I'm using KVM on Fedora 12 x86_64 to run several virtual machines. I've set up a bridged network interface, as I need these virtual machines to co-mingle on the same network as the physical machines.

The bridge is setup as per the scripts below. I'm using the default F12 sysctl.conf.

The problem is this... although virtual machines CAN access other machines on the network, including the Internet, virtual machines CANNOT communicate with each other. No pings, no file sharing, etc. The VM's have different MAC addresses.

-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-

# cat /etc/sysconfig/network-scripts/ifcfg-br0 
DEVICE=br0
TYPE=Bridge
BOOTPROTO=dhcp
ONBOOT=yes
DELAY=0

-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-

# cat /etc/sysconfig/network-scripts/ifcfg-eth0 
# Intel Corporation 82573L Gigabit Ethernet Controller
DEVICE=eth0
HWADDR=00:19:D1:31:E9:E3
ONBOOT=yes
TYPE=Ethernet
#BOOTPROTO=dhcp
NAME="System eth0"
UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
PEERDNS=yes
PEERROUTES=yes
BRIDGE=br0
MTU=9000

-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-

# cat /etc/sysctl.conf 
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# Disable netfilter on bridges.
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0

-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-

# brctl show
bridge name	bridge id		STP enabled	interfaces
br0		8000.0019d131e9e3	no		eth0
							vnet0
							vnet1
virbr0		8000.000000000000	yes

-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-

# brctl showmacs br0
port no	mac addr		is local?	ageing timer
  1	00:0f:b5:23:73:05	no		 194.94
  1	00:15:af:75:9d:bb	no		 209.88
  1	00:19:d1:31:e9:e3	yes		   0.00
  1	00:22:6b:5f:1f:3d	no		   0.01
  1	00:23:32:c9:e9:f8	no		 159.23
  1	00:30:18:a9:b2:76	no		   0.00
  2	32:d2:69:48:a6:64	yes		   0.00
  2	52:54:00:07:ca:6b	no		 159.30
  3	52:54:00:6f:7a:12	no		 208.36
  3	f6:ac:62:68:f5:8b	yes		   0.00


-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-

Version-Release number of selected component (if applicable):
qemu-kvm-0.11.0-11.fc12.x86_64
bridge-utils-1.2-8.fc12.x86_64



Steps to Reproduce:
Setup bridge, place two virtual machines with no firewalls on bridge, attempt to ping one another.
  
Actual results:
No ping, no IP communication.

Expected results:
L2 bridging with the ability to ping, etc.

Additional info:

Comment 1 James Rankin 2009-11-30 21:28:34 UTC
I solved this issue by removing the emulated Intel e1000 NIC's in my Windows Server 2003 guests and replacing it with the Realtek 8139 NIC instead.

Please see more detailed info in my post here:
http://forums.fedoraforum.org/showpost.php?p=1299215&postcount=8

I still think this is a bug... maybe in the emulated e1000 NIC. Should this be filed against qemu instead?

Thanks

Comment 2 Justin M. Forbes 2009-12-03 19:37:26 UTC
Thanks for the report.  I will look into it.

Comment 3 Amos Kong 2010-03-20 04:00:54 UTC
(In reply to comment #0)
> Description of problem:
> Apologies if this should be filed against a different package.
> 
> I'm using KVM on Fedora 12 x86_64 to run several virtual machines. I've set up
> a bridged network interface, as I need these virtual machines to co-mingle on
> the same network as the physical machines.
> 
> The bridge is setup as per the scripts below. I'm using the default F12
> sysctl.conf.
> 
> The problem is this... although virtual machines CAN access other machines on
> the network, including the Internet, virtual machines CANNOT communicate with
> each other. No pings, no file sharing, etc. The VM's have different MAC
> addresses.
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # cat /etc/sysconfig/network-scripts/ifcfg-br0 
> DEVICE=br0
> TYPE=Bridge
> BOOTPROTO=dhcp
> ONBOOT=yes
> DELAY=0
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # cat /etc/sysconfig/network-scripts/ifcfg-eth0 
> # Intel Corporation 82573L Gigabit Ethernet Controller
> DEVICE=eth0
> HWADDR=00:19:D1:31:E9:E3
> ONBOOT=yes
> TYPE=Ethernet
> #BOOTPROTO=dhcp
> NAME="System eth0"
> UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
> PEERDNS=yes
> PEERROUTES=yes
> BRIDGE=br0
> MTU=9000
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # cat /etc/sysctl.conf 
> # Kernel sysctl configuration file for Red Hat Linux
> #
> # For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
> # sysctl.conf(5) for more details.
> 
> # Controls IP packet forwarding
> net.ipv4.ip_forward = 0
> 
> # Controls source route verification
> net.ipv4.conf.default.rp_filter = 1
> 
> # Do not accept source routing
> net.ipv4.conf.default.accept_source_route = 0
> 
> # Controls the System Request debugging functionality of the kernel
> kernel.sysrq = 0
> 
> # Controls whether core dumps will append the PID to the core filename.
> # Useful for debugging multi-threaded applications.
> kernel.core_uses_pid = 1
> 
> # Disable netfilter on bridges.
> net.bridge.bridge-nf-call-ip6tables = 0
> net.bridge.bridge-nf-call-iptables = 0
> net.bridge.bridge-nf-call-arptables = 0
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # brctl show
> bridge name bridge id  STP enabled interfaces
> br0  8000.0019d131e9e3 no  eth0
>        vnet0
>        vnet1
> virbr0  8000.000000000000 yes
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # brctl showmacs br0
> port no mac addr  is local? ageing timer
>   1 00:0f:b5:23:73:05 no   194.94
>   1 00:15:af:75:9d:bb no   209.88
>   1 00:19:d1:31:e9:e3 yes     0.00
>   1 00:22:6b:5f:1f:3d no     0.01
>   1 00:23:32:c9:e9:f8 no   159.23
>   1 00:30:18:a9:b2:76 no     0.00
>   2 32:d2:69:48:a6:64 yes     0.00
>   2 52:54:00:07:ca:6b no   159.30
>   3 52:54:00:6f:7a:12 no   208.36
>   3 f6:ac:62:68:f5:8b yes     0.00
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> Version-Release number of selected component (if applicable):
> qemu-kvm-0.11.0-11.fc12.x86_64
> bridge-utils-1.2-8.fc12.x86_64
> 
> 
> 
> Steps to Reproduce:
> Setup bridge, place two virtual machines with no firewalls on bridge, attempt
> to ping one another.

Comment 4 Amos Kong 2010-03-20 04:02:30 UTC
(In reply to comment #0)
> Description of problem:
> Apologies if this should be filed against a different package.
> 
> I'm using KVM on Fedora 12 x86_64 to run several virtual machines. I've set up
> a bridged network interface, as I need these virtual machines to co-mingle on
> the same network as the physical machines.
> 
> The bridge is setup as per the scripts below. I'm using the default F12
> sysctl.conf.
> 
> The problem is this... although virtual machines CAN access other machines on
> the network, including the Internet, virtual machines CANNOT communicate with
> each other. No pings, no file sharing, etc. The VM's have different MAC
> addresses.
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # cat /etc/sysconfig/network-scripts/ifcfg-br0 
> DEVICE=br0
> TYPE=Bridge
> BOOTPROTO=dhcp
> ONBOOT=yes
> DELAY=0
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # cat /etc/sysconfig/network-scripts/ifcfg-eth0 
> # Intel Corporation 82573L Gigabit Ethernet Controller
> DEVICE=eth0
> HWADDR=00:19:D1:31:E9:E3
> ONBOOT=yes
> TYPE=Ethernet
> #BOOTPROTO=dhcp
> NAME="System eth0"
> UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03
> PEERDNS=yes
> PEERROUTES=yes
> BRIDGE=br0
> MTU=9000
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # cat /etc/sysctl.conf 
> # Kernel sysctl configuration file for Red Hat Linux
> #
> # For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
> # sysctl.conf(5) for more details.
> 
> # Controls IP packet forwarding
> net.ipv4.ip_forward = 0
> 
> # Controls source route verification
> net.ipv4.conf.default.rp_filter = 1
> 
> # Do not accept source routing
> net.ipv4.conf.default.accept_source_route = 0
> 
> # Controls the System Request debugging functionality of the kernel
> kernel.sysrq = 0
> 
> # Controls whether core dumps will append the PID to the core filename.
> # Useful for debugging multi-threaded applications.
> kernel.core_uses_pid = 1
> 
> # Disable netfilter on bridges.
> net.bridge.bridge-nf-call-ip6tables = 0
> net.bridge.bridge-nf-call-iptables = 0
> net.bridge.bridge-nf-call-arptables = 0
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # brctl show
> bridge name bridge id  STP enabled interfaces
> br0  8000.0019d131e9e3 no  eth0
>        vnet0
>        vnet1
> virbr0  8000.000000000000 yes
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> # brctl showmacs br0
> port no mac addr  is local? ageing timer
>   1 00:0f:b5:23:73:05 no   194.94
>   1 00:15:af:75:9d:bb no   209.88
>   1 00:19:d1:31:e9:e3 yes     0.00
>   1 00:22:6b:5f:1f:3d no     0.01
>   1 00:23:32:c9:e9:f8 no   159.23
>   1 00:30:18:a9:b2:76 no     0.00
>   2 32:d2:69:48:a6:64 yes     0.00
>   2 52:54:00:07:ca:6b no   159.30
>   3 52:54:00:6f:7a:12 no   208.36
>   3 f6:ac:62:68:f5:8b yes     0.00
> 
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-
> 
> Version-Release number of selected component (if applicable):
> qemu-kvm-0.11.0-11.fc12.x86_64
> bridge-utils-1.2-8.fc12.x86_64
> 
> 
> 
> Steps to Reproduce:
> Setup bridge, place two virtual machines with no firewalls on bridge, attempt
> to ping one another.


Hello James:
How about the reproduce ratio ?

I touched this problem.
But when I moved the test to other network environment, the problem disappear.

RHEL5.5
kvm version:  kvm-83-164.el5
# qemu-kvm --version
QEMU PC emulator version 0.11.0 (qemu-kvm-0.11.0), Copyright (c) 2003-2008 Fabrice Bellard

Comment 5 Dor Laor 2010-03-21 13:57:49 UTC
Please re-test when the spanning tree (STP) is off. When it is on, the new port on the destination bridge is blocked for 45 seconds +- until learning finished.
Also, supplying tcpdump on the destination tap interface would be helpful.
(Does it work with virtio nic?).

Comment 6 James Rankin 2010-03-24 03:11:29 UTC
When I initially filed this bug, I could reproduce it literally every single time... I no longer can. 

I've made no networking configuration changed on my Fedora host (nor on the physical network). My KVM version changed at least once, and is currently at:
qemu-kvm-0.11.0-13.fc12.x86_64

In any case, though, I just created two new VM's with e1000 vNICs, and I can't duplicate this problem any longer. The e1000 NIC is working normally for me now.

Comment 7 Thiago Born 2019-12-25 16:00:32 UTC
Hello Team, 

This is very curious. I could reproduce the issue. 

e1000 was able to receive an IP from my router (NAT - Router Model HS8247W). 

However, from the guest OS it was not possible to reach any other device on the same network address space. 

Then, I have changed to device model: "rtl8139", and everything worked as expected. 

```shell
➜  network-scripts virsh version                                                                                                                                                                        Compiled against library: libvirt 4.7.0                                                                                                                                                                 Using library: libvirt 4.7.0                                                                                                                                                                            Using API: QEMU 4.7.0                                                                                                                                                                                   Running hypervisor: QEMU 3.0.1 
```