Summary:

When trying to use VLAN interfaces, the VLAN interface does not seem to be
brought up correctly:
No traffic is sent through that interface, even when explicitely sent there. The
VLAN interface statistic (in /proc) says that no packages where transmitted.
As soon as the network is restarted, things work fine. However, simply deleting
and adding the IP is not good enough.

Version:
- Software:
  Latest RHEL 4, all updates applied
  kernel-2.6.9-34.EL
- Hardware:
  2 onboard Broadcom Corporation NetXtreme BCM5721
  is is for the "normal" traffic, 1 for the VLANs


Details:

/etc/sysconfig/network-scripts/ifcfg-eth0:
# this is the "default" interface
# does not seem to have anything to do with the problem, JIC
DEVICE=eth0
BOOTPROTO=static
IPADDR=10.0.1.101
NETMASK=255.255.255.192
ONBOOT=yes
TYPE=Ethernet


/etc/sysconfig/network-scripts/ifcfg-eth1.261
DEVICE=eth1.261
IPADDR=10.0.3.251
NETMASK=255.255.255.192
VLAN=yes
ONBOOT=no
BOOTPROTO=none


After configuring this and restarting the system, I try to ping my router:
$ ping -I eth1.261 10.0.3.254
PING 10.0.3.254 (10.0.3.254) from 10.0.3.251 eth1.261: 56(84) bytes of data.
From 10.0.3.251 icmp_seq=1 Destination Host Unreachable
From 10.0.3.251 icmp_seq=2 Destination Host Unreachable
...


ifconfig and ip addr show output looks ok, eg.:
eth1.261  Link encap:Ethernet  HWaddr 00:16:35:9F:18:18
          inet addr:10.0.3.251  Bcast:10.0.3.255  Mask:255.255.255.192
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:229 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:70640 (68.9 KiB)  TX bytes:0 (0.0 b)




However, /proc says that no packages where transmitted:
# cat /proc/net/vlan/eth1.261
eth1.261  VID: 261       REORDER_HDR: 1  dev->priv_flags: 1
         total frames received          578
          total bytes received        27766
      Broadcast/Multicast Rcvd            0

      total frames transmitted            0
       total bytes transmitted            0
            total headroom inc            0
           total encap on xmit            0
Device: eth1
INGRESS priority mappings: 0:0  1:0  2:0  3:0  4:0  5:0  6:0 7:0
EGRESSS priority Mappings:


When I run tcpdump on the VLAN interface, I can see (broadcast) packages - so
the /proc output seems ok.


arp for that interface does not seem to work either:
# arp -a
? (10.0.3.254) at <incomplete> on eth1.261
[snip]


When I now do a
/sbin/service network restart
Everything works fine:
- I can ping the gateway
- arp looks ok
- /proc/net/vlan/eth1.261 shows outgoing traffic
- my apps work fine on the interface



I tried to reproduce this a bit better:

- When I delete and add the IP address on the VLAN interface manually,
  the problem stays the same.
  (ip addr del ... ; ip addr add ...)
- When I delete the IP, take down the VLAN and re-add it, things work fine)
  (ip addr del ... ; vconfig rem ... ; vconfig add ... ; ip addr add ... )
- If I do not configure any VLAN interfaces at startup, but configure everything
  manually, the problem is stil the same.
  ( modprobe 8021q ; ifconfig eth1 up ; vconfig add ... ; ip addr add ... )
- Once this works for one VLAN interface, I can add other VLAN interfaces, and
  they work just fine.
- If I use eth1 as a "normal" interface, it works just fine.


Thanks,
Michael