Bug 175499 - ARP sent with wrong source address
ARP sent with wrong source address
Product: Red Hat Enterprise Linux 4
Classification: Red Hat
Component: iproute (Show other bugs)
All Linux
medium Severity medium
: ---
: ---
Assigned To: Marcela Mašláňová
Brock Organ
Depends On:
  Show dependency treegraph
Reported: 2005-12-11 18:44 EST by Geoff Kingsmill
Modified: 2007-11-30 17:07 EST (History)
0 users

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2007-10-22 06:31:57 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Geoff Kingsmill 2005-12-11 18:44:59 EST
I am investigating a problem where network access to a machine with
multiple interfaces are intermittently failing. 
There appears to be a network ARP problem in Linux when a packet
arrives on one interface/subnet and is returned on a different 
interface/subnet. In this configuration, when Linux needs to send
out an ARP request, the ARP packet contains the correct Source MAC 
address of the interface which sent the ARP request however it sends
out the wrong Source IP address. Instead of sending the IP address
of interface which transmitted the ARP packet, it actually sends
the IP address of the interface which received the incoming packet.
The CISCO router does not respond to the ARP request due to the
fact that the Source IP address is on a different subnet.

The follow simplified network diagram illustrates the problem.
|                        CISCO Router                              |
|                         10.20.x.x/24                             | 
|                                                                  |
|    101.253                                 102.253     105.253   |
        |                                       |           |
        |                                       |           |
        |                                       |           |
+-------+-------+                       +-------+-----------+------+
|    101.1      |                       |    102.245     105.245   |
|               |                       |      eth0        eth1    |
|    HostA      |                       |           HostB          |
+---------------+                       +-------+-----+-----+------+
                                                |     |     |
                                                |     |     |
                                              other interfaces
-- HostA pings HostB 105.245 
[HostA]# ping
PING ( 56(84) bytes of data.
<hangs here>
-- The network interfaces on HostB
[HostB]# ifconfig
eth0      Link encap:Ethernet  HWaddr 00:11:0A:53:7A:3A
          inet addr:  Bcast:  Mask:
eth1      Link encap:Ethernet  HWaddr 00:11:0A:53:7A:3B
          inet addr:  Bcast:  Mask:
eth2      Link encap:Ethernet  HWaddr 00:11:85:66:E7:5D
          inet addr:  Bcast:  Mask:
eth3      Link encap:Ethernet  HWaddr 00:11:85:66:E7:5C
          inet addr:  Bcast:  Mask:
eth4      Link encap:Ethernet  HWaddr 00:00:80:22:89:BC
          inet addr:  Bcast:  Mask:
eth5      Link encap:Ethernet  HWaddr 00:02:A5:45:00:BA
          inet addr:  Bcast:  Mask:
-- The routing table on HostB shows that packets destined for the 
-- 101 subnet will be directed to the eth0 102 subnet gateway.
[HostB]# netstat -rn
Kernel IP routing table
Destination    Gateway         Genmask        Flags MSS Window  irtt Iface UGH   0 0          0 eth3   U     0 0          0 eth4   UG    0 0          0 eth2   UG    0 0          0 eth2   U     0 0          0 eth2   U     0 0          0 eth3   U     0 0          0 eth5   UG    0 0          0 eth3   U     0 0          0 eth1   UG    0 0          0 eth3   UG    0 0          0 eth2   UG    0 0          0 eth0   U     0 0          0 eth0   UG    0 0          0 eth0     U     0 0          0 eth5         UG    0 0          0 eth4
-- During the failure HostB does not have a MAC address for the 102.253
-- gateway
[HostB]# arp -a | grep 102
t2cat1-h102 ( at <incomplete> on eth0
-- If on HostB I then ping the 102.253 gateway the ARP request completes
-- and the ping on HostA starts working
[HostB]# ping -c 5
PING ( 56(84) bytes of data.
From icmp_seq=0 Destination Host Unreachable
From icmp_seq=3 Destination Host Unreachable
64 bytes from icmp_seq=4 ttl=255 time=2.00 ms
--- ping statistics ---
5 packets transmitted, 1 received, +2 errors, 80% packet loss, time 4001ms
rtt min/avg/max/mdev = 2.005/2.005/2.005/0.000 ms, pipe 2
-- The ping on HostB to 102.253 triggered an ARP request which worked 
-- as expected.
[HostB]# arp -a | grep 102
t2cat1-h102 ( at 00:00:0C:07:AC:66 [ether] on eth0
-- When the ARP entry times out , the ping on HostA again fails.
[HostB]# arp -a | grep 102
t2cat1-h102 ( at <incomplete> on eth0
-- So the question is why doesn't the ping in HostA cause the ARP entry on 
-- HostB to be refreshed.
-- A network trace on the 102 subnet shows that HostB sent out an ARP 
-- request with the correct "Target IP address" and "Sender MAC address"
-- but with the WRONG "Sender IP address". The ARP packet contained the
-- "Sender IP address" of the 105.245 receiving interface rather that the 
-- 102.245 IP address of the returning interface. The CISCO router ignores
-- the ARP request as the returning IP address is on a different subnet.
Ethereal Packet Capture
No. Time     Source            Destination Protocol Info
140 0.129057 HewlettP_53:7a:3a Broadcast   ARP      Who has  
Address Resolution Protocol (request)
    Hardware type: Ethernet (0x0001)
    Protocol type: IP (0x0800)
    Hardware size: 6
    Protocol size: 4
    Opcode: request (0x0001)
    Sender MAC address: HewlettP_53:7a:3a (00:11:0a:53:7a:3a)
    Sender IP address: (
                       ^^^^^^^^^^^^^^ WRONG - needs to be
    Target MAC address: 00:00:00_00:00:00 (00:00:00:00:00:00)
    Target IP address: (
-- If I define a 101.0 network route to go via the 102.245 interface but
-- also specify a src address of 102.245 then I still see the ARP request  
-- sent with a Sender IP address of the receiving interface 105.245 rather 
-- than the 102.245 IP address of either the interface which sent out the ARP 
-- request or the source address defined on the route.
[HostB]# ip route add via \
                           dev eth0 src
[HostB]# ip route show via dev eth3 dev eth4  proto kernel  scope link  src via dev eth2 via dev eth2 dev eth2  proto kernel  scope link  src dev eth3  proto kernel  scope link  src dev eth5  proto kernel  scope link  src dev eth1  proto kernel  scope link  src via dev eth3 via dev eth3 via dev eth2 via dev eth0  src
                                           ^^^^^^^^^^^^^^^^^ dev eth0  proto kernel  scope link  src via dev eth0 dev eth5  scope link
default via dev eth4
-- This is on a RedHat WS4-U1 system
[HostB]# lsb_release -a
LSB Version:    1.3
Distributor ID: RedHatEnterpriseWS
Description:    Red Hat Enterprise Linux WS release 4 (Nahant Update 1)
Release:        4
Codename:       NahantUpdate1

 I looked at the tuning parameter arp_filter and rp_filter in 
/proc/sys/net/ipv4/config/*/ but this made no difference and appears
to related to how incoming arp requests are handled rather than changing
the behaviour of outgoing ARP requests.

Is this a bug in ARP or is there a configuration parameter to change
the current behaviour.

Geoff Kingsmill
Comment 1 Geoff Kingsmill 2005-12-12 17:10:47 EST
I should also mention that I can work around the problem by adding a permanent
arp entry or by using arptables (although this machine is WS which does not
include arptables).
Comment 2 Radek Vokal 2006-02-03 09:28:53 EST
I've tried to reproduce this issue but with no luck. This is more a question for
our support team. I guess you're subscriber so if you can please contact someone
from http://www.redhat.com/apps/support/, they'll be glad to help you. 

Note You need to log in before you can comment on or make changes to this bug.