This issue is being noticed in 22.06 (ovn22.06-22.06.0-24.el8fdp, ovn22.06-22.06.0-24.el9fdp). When a frame is sent having double vlan tags, it does not pass through the ports if the port security is enabled. It works OK if port security is removed or not set. I used python and scapy to generate a customised frame Here is the reproducer: ########## Server side setup ############# systemctl start openvswitch systemctl start ovn-northd ovn-nbctl set-connection ptcp:6641 ovn-sbctl set-connection ptcp:6642 ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:42.42.42.1:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=42.42.42.1 systemctl start ovn-controller sleep 5 ovn-nbctl ls-add ls ovn-nbctl lsp-add ls vm1 ovn-nbctl lsp-set-addresses vm1 "00:00:00:00:00:01 42.42.42.15 2000::15" ovn-nbctl lsp-add ls vm2 ovn-nbctl lsp-set-addresses vm2 "00:00:00:00:00:02 42.42.42.25 2000::25" ovn-nbctl lsp-set-port-security vm2 "00:00:00:00:00:02 42.42.42.25 2000::25" ovn-nbctl lsp-set-port-security vm1 "00:00:00:00:00:01 42.42.42.15 2000::15" ovn-nbctl set logical_switch ls other_config:vlan-passthru=true #ovn-nbctl set logical_switch ls other_config:subnet=42.42.42.0/24 other_config:mcast_querier=true other_config:mcast_snoop=true other_config:mcast_flood_unregistered=true other_config:mcast_eth_src=00:00:00:00:00:05 other_config:mcast_ip4_src=42.42.42.5 other_config:mcast_ip6_src=fe80::1 ip netns add vm1 ovs-vsctl add-port br-int vm1 -- set interface vm1 type=internal ip link set vm1 netns vm1 ip netns exec vm1 ip link set vm1 address 00:00:00:00:00:01 ip netns exec vm1 ip link set vm1 up ip netns exec vm1 ip link set lo up ovs-vsctl set Interface vm1 external_ids:iface-id=vm1 ip netns exec vm1 ip link add link vm1 name vm1.5 type vlan id 5 ip netns exec vm1 ip link set vm1.5 up ip netns exec vm1 ip addr add 42.42.42.15/24 dev vm1.5 ip netns exec vm1 ip addr add 2000::15/64 dev vm1.5 ip netns exec vm1 ping 42.42.42.25 -c 3 ip netns exec vm1 ping6 2000::25 -c 3 #send packets with two vlan tags ip netns exec vm1 python3 vlan.py ## reset port security ovn-nbctl lsp-set-port-security vm2 ovn-nbctl lsp-set-port-security vm1 #send packets with two vlan tags ip netns exec vm1 python3 vlan.py ################################################################ ########## Client side setup ########################## systemctl start ovn-northd systemctl start openvswitch ovs-vsctl set open . external_ids:system-id=hv0 #ifconfig ens1f0 42.42.42.2 netmask 255.255.255.0 ovs-vsctl set open . external_ids:ovn-remote=tcp:42.42.42.1:6642 ovs-vsctl set open . external_ids:ovn-encap-type=geneve ovs-vsctl set open . external_ids:ovn-encap-ip=42.42.42.2 systemctl start ovn-controller ip netns add vm2 ovs-vsctl add-port br-int vm2 -- set interface vm2 type=internal ip link set vm2 netns vm2 ip netns exec vm2 ip link set vm2 address 00:00:00:00:00:02 ip netns exec vm2 ip link set vm2 up ip netns exec vm2 ip link set lo up ovs-vsctl set Interface vm2 external_ids:iface-id=vm2 ip netns exec vm2 ip link add link vm2 name vm2.5 type vlan id 5 ip netns exec vm2 ip link set vm2.5 up ip netns exec vm2 ip addr add 42.42.42.25/24 dev vm2.5 ip netns exec vm2 ip addr add 2000::25/64 dev vm2.5 ip netns exec vm2 tcpdump -U -i vm2.5 -w vlan.pcap& sleep 2 ## Execute vlan.py on the server side then do the following pkill tcpdump [root@double-vlan-tag]# tcpdump -r vlan.pcap -nnle|grep "vlan 5.*vlan 10" reading from file vlan.pcap, link-type EN10MB (Ethernet) dropped privs to tcpdump [1]+ Done ip netns exec vm2 tcpdump -U -i vm2.5 -w vlan.pcap [root@double-vlan-tag]# ## Do the below again after removing port security on the server side ip netns exec vm2 tcpdump -U -i vm2.5 -w vlan.pcap& sleep 2 ## Execute vlan.py on the server side then do the following pkill tcpdump [root@double-vlan-tag]# tcpdump -r vlan.pcap -nnle|grep "vlan 5.*vlan 10" reading from file vlan.pcap, link-type EN10MB (Ethernet) dropped privs to tcpdump 09:19:07.420233 00:00:00:00:00:01 > 00:00:00:00:00:02, ethertype 802.1Q (0x8100), length 50: vlan 5, p 0, ethertype 802.1Q, vlan 10, p 0, ethertype IPv4, 42.42.42.15 > 42.42.42.25: ICMP echo request, id 0, seq 0, length 8 [1]+ Done ip netns exec vm2 tcpdump -U -i vm2.5 -w vlan.pcap [root@double-vlan-tag]# ######### vlan.py ######### #!/usr/bin/python import sys from scapy.all import * mac="00:00:00:00:00:01" a=Ether(dst="00:00:00:00:00:02")/Dot1Q(vlan=5)/Dot1Q(vlan=10)/IP(dst="42.42.42.25")/ICMP() a.show() sendp(a, iface="vm1.5")
I checked out branch-22.06 from upstream, then tried to reproduce the issue with the following test case for OVN test suite: +OVN_FOR_EACH_NORTHD([ +AT_SETUP([ovn -- port security works with vlan-passthru=true switches]) +AT_KEYWORDS([ovntest]) +ovn_start + +net_add n + +# two hypervisors, each connected to the same network +for i in 1 2; do + sim_add hv$i + as hv$i + ovs-vsctl add-br br-phys + ovs-vsctl set open . external-ids:ovn-bridge-mappings=phys:br-phys + ovn_attach n br-phys 192.168.0.$i +done + +ovn-nbctl ls-add ls +ovn-nbctl lsp-add ls vm1 +ovn-nbctl lsp-add ls vm2 +ovn-nbctl lsp-set-addresses vm1 "00:00:00:00:00:01 42.42.42.15 2000::15" +ovn-nbctl lsp-set-addresses vm2 "00:00:00:00:00:02 42.42.42.25 2000::25" +ovn-nbctl lsp-set-port-security vm1 "00:00:00:00:00:01 42.42.42.15 2000::15" +ovn-nbctl lsp-set-port-security vm2 "00:00:00:00:00:02 42.42.42.25 2000::25" +ovn-nbctl set logical_switch ls other_config:vlan-passthru=true + +for i in 1 2; do + as hv$i + check ovs-vsctl add-port br-int vif$i -- set Interface vif$i \ + external-ids:iface-id=vm$i \ + options:tx_pcap=vif$i-tx.pcap \ + options:rxq_pcap=vif$i-rx.pcap + > $i.expected +done + +wait_for_ports_up +OVN_POPULATE_ARP + +send_icmp_packet() { + local inport=$1 outport=$2 eth_src=$3 eth_dst=$4 ipv4_src=$5 ipv4_dst=$6 ip_chksum=$7 data=$8 + local ip_ttl=ff + local ip_len=001c + tag1=81000005 + tag2=8100000a + local packet=${eth_dst}${eth_src}${tag1}${tag2}08004500${ip_len}00004000${ip_ttl}01${ip_chksum}${ipv4_src}${ipv4_dst}${data} + as hv$inport ovs-appctl netdev-dummy/receive vif$inport $packet + echo $packet > $outport.expected +} + +send_icmp_packet 1 2 000000000001 000000000002 2a2a2a0f 2a2a2a19 0000 0800000000000000000000 +send_icmp_packet 2 1 000000000002 000000000001 2a2a2a19 2a2a2a0f 0000 0000000000000000000000 + +for i in 1 2; do + OVN_CHECK_PACKETS_REMOVE_BROADCAST([vif$i-tx.pcap], [$i.expected]) +done + +AT_CLEANUP +]) + But the test passes, confirming that adding port-security that is identical to addresses set for the port doesn't block double tagged packets. (.1Q VLAN tag type is used; ICMP request and reply are sent; IP addresses / OVN commands used are identical; the only potential difference I see is in cluster setup - though it seems standard - and the fact that scapy is used to send ICMP packets instead of injecting a packet into the switch directly through userspace). I will explore that discrepancy, but on first sight double tagged packets don't seem to conflict with port security.
The difference between OVN test case (from Ihar) and initial reproducer is that three VLAN tags are used in the initial reproducer. Two tags are added by vlan.py, and one is added as generating the packet on vm1.5. Hence the following openflow, in CHK_IN_PORT_SEC table: table=73, priority=90,ip,reg14=0x1,metadata=0x1,dl_src=00:00:00:00:00:01,nw_src=42.42.42.15 actions=load:0->NXM_NX_REG10[12] is not hit, as the ethertype is seen as 0x8100 (the type of the third vlan) Dual tagging works fine. Do you need to have OVN/OVS support triple vlan tagging or is this a typo/test mis-configuration ? Thanks
Very good point Xavier. This, in fact, is the case of triple vlan tags. I noted a similar Bug 2133902 as well. But the problem is with the port-security. When it is disabled, we can use as many vlan tags as we want: see below tcpdump of the received packet with 4 vlan tags: 18:39:14.798329 00:00:00:00:00:01 > 00:00:00:00:00:02, ethertype 802.1Q (0x8100), length 58: vlan 5, p 0, ethertype 802.1Q, vlan 10, p 0, ethertype 802.1Q, vlan 15, p 0, ethertype 802.1Q, vlan 20, p 0, ethertype IPv4, 42.42.42.15 > 42.42.42.25: ICMP echo request, id 0, seq 0, length 8 However, when port-security is enabled with the configuration given in c0, as you pointed, only double vlan tags are allowed. Can you please explain what happens with the port-security when 3 vlan tags are used?