Description of problem: ping from bond brdige to peer side failed. Check the ovs log find below message: 2023-03-30T07:04:44.153Z|00062|bond(handler3)|DBG|Dropped 5 log messages in last 41 seconds (most recently, 18 seconds ago) due to excessive rate 2023-03-30T07:04:44.153Z|00063|bond(handler3)|DBG|bond balance-slb: member ens4f1v25: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1.Each PF creation can support the maximum number of vf. Select one of the vfs as slave of the bond. ip link set ens4f0 mtu 9200 ip link set ens4f1 mtu 9200 echo 64 > /sys/class/net/ens4f0/device/sriov_numvfs echo 64 > /sys/class/net/ens4f1/device/sriov_numvfs 221: ens4f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9200 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:bd:1c:a4 brd ff:ff:ff:ff:ff:ff vf 0 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 1 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 2 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 3 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 4 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 5 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 6 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 7 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 8 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 9 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 10 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 11 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 12 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 13 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 14 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 15 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 16 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 17 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 18 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 19 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 20 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 21 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 22 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 23 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 24 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 25 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 26 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 27 link/ether 6e:9a:14:f9:7b:18 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust on vf 28 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 29 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 30 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 31 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 32 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 33 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 34 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 35 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 36 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 37 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 38 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 39 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 40 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 41 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 42 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 43 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 44 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 45 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 46 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 47 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 48 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 49 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 50 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 51 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 52 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 53 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 54 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 55 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 56 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 57 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 58 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 59 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 60 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 61 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 62 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 63 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off altname enp175s0f0 # ip link show ens4f1 222: ens4f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9200 qdisc mq state UP mode DEFAULT group default qlen 1000 link/ether 3c:fd:fe:bd:1c:a5 brd ff:ff:ff:ff:ff:ff vf 0 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 1 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 2 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 3 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 4 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 5 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 6 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 7 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 8 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 9 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 10 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 11 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 12 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 13 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 14 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 15 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 16 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 17 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 18 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 19 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 20 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 21 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 22 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 23 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 24 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 25 link/ether 02:f2:12:92:34:15 brd ff:ff:ff:ff:ff:ff, spoof checking off, link-state auto, trust on vf 26 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 27 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 28 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 29 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 30 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 31 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 32 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 33 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 34 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 35 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 36 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 37 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 38 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 39 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 40 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 41 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 42 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 43 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 44 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 45 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 46 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 47 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 48 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 49 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 50 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 51 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 52 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 53 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 54 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 55 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 56 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 57 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 58 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 59 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 60 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 61 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 62 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off vf 63 link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff, spoof checking on, link-state auto, trust off altname enp175s0f1 choose vf 25 from ens4f0 and vf 27 from ens4f1 as slave port. ip link set ens4f0 vf 25 spoofchk off ip link set ens4f0 vf 25 trust on ip link set ens4f1 vf 27 spoofchk off ip link set ens4f1 vf 25 trust on 2. On switch side, corresponding switch port of test PF wasn added in channel group without lacp. sw-cisco9364# show interface brief -------------------------------------------------------------------------------- Port VRF Status IP Address Speed MTU -------------------------------------------------------------------------------- mgmt0 -- up 10.73.88.13 1000 1500 -------------------------------------------------------------------------------- Ethernet VLAN Type Mode Status Reason Speed Port Interface Ch # -------------------------------------------------------------------------------- .....(skip)....... Eth1/7/1 1000 eth trunk up none 25G(D) 64 Eth1/7/2 1000 eth trunk up none 25G(D) 64 ------------------------------------------------------------------------------------------ sw-cisco9364# show port-channel summary Flags: D - Down P - Up in port-channel (members) I - Individual H - Hot-standby (LACP only) s - Suspended r - Module-removed b - BFD Session Wait S - Switched R - Routed U - Up (port-channel) p - Up in delay-lacp mode (member) M - Not in use. Min-links not met -------------------------------------------------------------------------------- Group Port- Type Protocol Member Ports Channel -------------------------------------------------------------------------------- 64 Po64(SU) Eth NONE Eth1/7/1(P) Eth1/7/2(P) sw-cisco9364# show port-channel traffic interface port-channel 64 NOTE: Clear the port-channel member counters to get accurate statistics ChanId Port Rx-Ucst Tx-Ucst Rx-Mcst Tx-Mcst Rx-Bcst Tx-Bcst ------ --------- ------- ------- ------- ------- ------- ------- 64 Eth1/7/1 0.0% 0.0% 50.00% 48.93% 0.0% 0.0% 64 Eth1/7/2 0.0% 0.0% 50.00% 51.06% 100.00% 0.0% 3. create a ovs bridge and attach vf port as slave interface for balance-slb. systemctl restart openvswitch ovs-appctl vlog/set console:lacp:dbg syslog:lacp:dbg file:lacp:dbg ovs-appctl vlog/set console:bond:dbg syslog:bond:dbg file:bond:dbg ovs-appctl vlog/set console:ovs_rcu:dbg syslog:ovs_rcu:dbg file:ovs_rcu:dbg ovs-appctl vlog/set console:memory:dbg syslog:memory:dbg file:memory:dbg ovs-vsctl set Open_vSwitch . other_config={} ovs-vsctl --may-exist add-br bondbridge ovs-vsctl set int bondbridge mtu_request=9200 ovs-ofctl mod-port bondbridge bondbridge up ovs-vsctl add-bond bondbridge balance-slb ens4f1v25 ens4f0v27 lacp=off bond_mode=balance-slb -- set Interface ens4f1v25 mtu_request=9200 -- set Interface ens4f0v27 mtu_request=9200 # ovs-vsctl show d6152eff-ea84-4e6e-b452-4ad832befd72 Bridge bondbridge Port balance-slb Interface ens4f1v25 Interface ens4f0v27 Port bondbridge Interface bondbridge type: internal ovs_version: "3.1.1" # ovs-appctl bond/show ---- balance-slb ---- bond_mode: balance-slb bond may use recirculation: no, Recirc-ID : -1 bond-hash-basis: 0 lb_output action: disabled, bond-id: -1 all members active: false updelay: 0 ms downdelay: 0 ms next rebalance: 4012 ms lacp_status: off lacp_fallback_ab: false active-backup primary: <none> active member mac: 02:f2:12:92:34:15(ens4f1v25) member ens4f0v27: enabled may_enable: true member ens4f1v25: enabled active member may_enable: true 4. configure ip address on bond bridge and ping peer side # ip addr show bondbridge 1523: bondbridge: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9200 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether ba:32:1e:c1:71:40 brd ff:ff:ff:ff:ff:ff inet 172.31.152.210/24 scope global bondbridge valid_lft forever preferred_lft forever inet6 fe80::b832:1eff:fec1:7140/64 scope link valid_lft forever preferred_lft forever # ping -I bondbridge 172.31.152.1 PING 172.31.152.1 (172.31.152.1) from 172.31.152.210 bondbridge: 56(84) bytes of data. ^C --- 172.31.152.1 ping statistics --- 12 packets transmitted, 0 received, 100% packet loss, time 11259ms 6. check ovs log as below: 2023-03-30T07:17:58.777Z|00086|bond(handler3)|DBG|Dropped 3 log messages in last 113 seconds (most recently, 80 seconds ago) due to excessive rate 2023-03-30T07:17:58.777Z|00087|bond(handler3)|DBG|bond balance-slb: member ens4f1v25: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-03-30T07:17:59.272Z|00018|bond(revalidator2)|DBG|bond balance-slb: member ens4f1v25: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-03-30T07:18:06.280Z|01371|bond|DBG|bond balance-slb: ens4f1v25 0kB (h76: 0kB + h227: 0kB), ens4f0v27 0kB 2023-03-30T07:18:16.292Z|01372|bond|DBG|bond balance-slb: ens4f1v25 0kB (h76: 0kB + h227: 0kB), ens4f0v27 0kB 2023-03-30T07:18:26.301Z|01373|bond|DBG|bond balance-slb: ens4f1v25 1kB (h76: 1kB + h227: 0kB), ens4f0v27 0kB 2023-03-30T07:18:36.312Z|01374|bond|DBG|bond balance-slb: ens4f1v25 0kB (h76: 0kB), ens4f0v27 0kB 2023-03-30T07:18:41.388Z|00088|bond(handler3)|DBG|Dropped 1 log messages in last 19 seconds (most recently, 19 seconds ago) due to excessive rate 2023-03-30T07:18:41.388Z|00089|bond(handler3)|DBG|bond balance-slb: member ens4f1v25: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 7. peer side can capture icmp packets. # tcpdump -i ens3f0 -ne dropped privs to tcpdump tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on ens3f0, link-type EN10MB (Ethernet), snapshot length 262144 bytes 03:19:40.525669 ba:32:1e:c1:71:40 > 40:a6:b7:3e:a5:60, ethertype IPv4 (0x0800), length 98: 172.31.152.210 > 172.31.152.1: ICMP echo request, id 549, seq 64, length 64 03:19:40.525685 40:a6:b7:3e:a5:60 > ba:32:1e:c1:71:40, ethertype IPv4 (0x0800), length 98: 172.31.152.1 > 172.31.152.210: ICMP echo reply, id 549, seq 64, length 64 03:19:41.549670 ba:32:1e:c1:71:40 > 40:a6:b7:3e:a5:60, ethertype IPv4 (0x0800), length 98: 172.31.152.210 > 172.31.152.1: ICMP echo request, id 549, seq 65, length 64 03:19:41.549684 40:a6:b7:3e:a5:60 > ba:32:1e:c1:71:40, ethertype IPv4 (0x0800), length 98: 172.31.152.1 > 172.31.152.210: ICMP echo reply, id 549, seq 65, length 64 03:19:42.573661 ba:32:1e:c1:71:40 > 40:a6:b7:3e:a5:60, ethertype IPv4 (0x0800), length 98: 172.31.152.210 > 172.31.152.1: ICMP echo request, id 549, seq 66, length 64 03:19:42.573668 40:a6:b7:3e:a5:60 > ba:32:1e:c1:71:40, ethertype IPv4 (0x0800), length 98: 172.31.152.1 > 172.31.152.210: ICMP echo reply, id 549, seq 66, length 64 03:19:43.597676 ba:32:1e:c1:71:40 > 40:a6:b7:3e:a5:60, ethertype IPv4 (0x0800), length 98: 172.31.152.210 > 172.31.152.1: ICMP echo request, id 549, seq 67, length 64 03:19:43.597683 40:a6:b7:3e:a5:60 > ba:32:1e:c1:71:40, ethertype IPv4 (0x0800), length 98: 172.31.152.1 > 172.31.152.210: ICMP echo reply, id 549, seq 67, length 64 03:19:44.621670 ba:32:1e:c1:71:40 > 40:a6:b7:3e:a5:60, ethertype IPv4 (0x0800), length 98: 172.31.152.210 > 172.31.152.1: ICMP echo request, id 549, seq 68, length 64 03:19:44.621677 40:a6:b7:3e:a5:60 > ba:32:1e:c1:71:40, ethertype IPv4 (0x0800), length 98: 172.31.152.1 > 172.31.152.210: ICMP echo reply, id 549, seq 68, length 64 ^C 10 packets captured 10 packets received by filter 0 packets dropped by kernel On Dut side, Only icmp on tx can be capture # tcpdump -i bondbridge dropped privs to tcpdump tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on bondbridge, link-type EN10MB (Ethernet), snapshot length 262144 bytes 03:20:15.340374 IP hp-dl388g10-03.rhts.eng.pek2.redhat.com > 172.31.152.1: ICMP echo request, id 549, seq 98, length 64 03:20:16.364374 IP hp-dl388g10-03.rhts.eng.pek2.redhat.com > 172.31.152.1: ICMP echo request, id 549, seq 99, length 64 03:20:17.388367 IP hp-dl388g10-03.rhts.eng.pek2.redhat.com > 172.31.152.1: ICMP echo request, id 549, seq 100, length 64 03:20:18.412375 IP hp-dl388g10-03.rhts.eng.pek2.redhat.com > 172.31.152.1: ICMP echo request, id 549, seq 101, length 64 03:20:19.436367 IP hp-dl388g10-03.rhts.eng.pek2.redhat.com > 172.31.152.1: ICMP echo request, id 549, seq 102, length 64 03:20:20.460372 IP hp-dl388g10-03.rhts.eng.pek2.redhat.com > 172.31.152.1: ICMP echo request, id 549, seq 103, length 64 Actual results: 1. ping peer side failed 2. check 'admissibility verdict is to drop pkt as different port is learned' in ovs log Expected results: 1. ping peer side as well Additional info: If remove channel group from switch site(didn't used static link aggregation), icmp reply can be received as well.
I see: > bond balance-slb: member ens4f1v25: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off This indicates that OVS thinks it's getting a packet on the wrong port. This could be due to the switch, or the nic driver, or OVS. You run "tcpdump -i bondbridge" but in this case I'd recommend "tcpdump -i ens4f1v25". Or whatever one leg of the bond member is. This would indicate if we actually do see the packets coming in on the correct leg. I'll try and set up a reproducer, Intel nic right?
Hello Mike > I'll try and set up a reproducer, Intel nic right? yep, I use xxv710 NIC. I try run same test on 5.14.0-318.el9.x86_64 + openvswitch3.1-3.1.0-24.el9fdp.x86_64 and still can see below message on ovs-vswitchd.log # tail -f /var/log/openvswitch/ovs-vswitchd.log 2023-06-14T12:22:50.559Z|00041|bridge|INFO|bridge guestbridge: using datapath ID 0000d629ba153d42 2023-06-14T12:22:50.559Z|00042|connmgr|INFO|guestbridge: added service controller "punix:/var/run/openvswitch/guestbridge.mgmt" 2023-06-14T12:22:50.624Z|00043|bridge|INFO|bridge guestbridge: added interface patchguest on port 1 2023-06-14T12:22:51.448Z|00044|bridge|INFO|bridge guestbridge: added interface 740ab41a0b764_l on port 2 2023-06-14T12:22:51.720Z|00045|bridge|INFO|bridge guestbridge: added interface c2d003d22df84_l on port 3 2023-06-14T12:22:51.973Z|00046|bridge|INFO|bridge guestbridge: added interface 65434c82dd954_l on port 4 2023-06-14T12:22:52.756Z|00047|connmgr|INFO|guestbridge<->unix#39: 1 flow_mods in the last 0 s (1 deletes) 2023-06-14T12:22:52.792Z|00048|connmgr|INFO|guestbridge<->unix#42: 1 flow_mods in the last 0 s (1 adds) 2023-06-14T12:22:58.775Z|00049|memory|INFO|172224 kB peak resident set size after 10.1 seconds 2023-06-14T12:22:58.775Z|00050|memory|INFO|handlers:3 idl-cells-Open_vSwitch:460 ports:7 revalidators:1 rules:10 udpif keys:21 2023-06-14T12:27:31.062Z|00051|netdev_linux|WARN|ethtool command ETHTOOL_GFLAGS on network device ens4f1v11 failed: No such device 2023-06-14T12:27:31.063Z|00052|netdev_linux|WARN|ethtool command ETHTOOL_GSET on network device ens4f1v11 failed: No such device 2023-06-14T12:27:31.063Z|00053|bridge|INFO|bridge bondbridge: added interface ens4f1v11 on port 2 2023-06-14T12:27:31.063Z|00054|netdev_linux|WARN|ethtool command ETHTOOL_GDRVINFO on network device ens4f1v11 failed: No such device 2023-06-14T12:27:31.064Z|00055|netdev_linux|WARN|ethtool command ETHTOOL_GFLAGS on network device ens4f0v5 failed: No such device 2023-06-14T12:27:31.064Z|00056|netdev_linux|WARN|ethtool command ETHTOOL_GSET on network device ens4f0v5 failed: No such device 2023-06-14T12:27:31.064Z|00057|bridge|INFO|bridge bondbridge: added interface ens4f0v5 on port 3 2023-06-14T12:27:31.064Z|00058|netdev_linux|WARN|ethtool command ETHTOOL_GDRVINFO on network device ens4f0v5 failed: No such device 2023-06-14T12:27:31.064Z|00059|bond|INFO|member ens4f1v11: enabled 2023-06-14T12:27:31.064Z|00060|bond|INFO|member ens4f0v5: enabled 2023-06-14T12:27:31.065Z|00061|bond|INFO|bond balance-slb: active member is now ens4f0v5 2023-06-14T12:27:31.065Z|00062|netdev_linux|WARN|ethtool command ETHTOOL_GDRVINFO on network device ens4f1v11 failed: No such device 2023-06-14T12:27:31.065Z|00063|netdev_linux|WARN|ethtool command ETHTOOL_GDRVINFO on network device ens4f0v5 failed: No such device 2023-06-14T12:27:31.070Z|00064|netdev_linux|WARN|ethtool command ETHTOOL_GSET on network device ens4f1v11 failed: No such device 2023-06-14T12:27:31.070Z|00065|netdev_linux|WARN|ethtool command ETHTOOL_GSET on network device ens4f0v5 failed: No such device 2023-06-14T12:27:31.073Z|00066|netdev_linux|WARN|ethtool command ETHTOOL_GDRVINFO on network device ens4f1v11 failed: No such device 2023-06-14T12:27:31.073Z|00067|netdev_linux|WARN|ethtool command ETHTOOL_GDRVINFO on network device ens4f0v5 failed: No such device 2023-06-14T12:27:31.166Z|00068|bond|INFO|member ens4f0v5: link state down 2023-06-14T12:27:31.166Z|00069|bond|INFO|member ens4f0v5: disabled 2023-06-14T12:27:31.166Z|00070|bond|INFO|member ens4f1v11: link state down 2023-06-14T12:27:31.166Z|00071|bond|INFO|member ens4f1v11: disabled 2023-06-14T12:27:31.166Z|00072|bond|INFO|bond balance-slb: all members disabled 2023-06-14T12:27:31.270Z|00073|bond|INFO|member ens4f0v5: link state up 2023-06-14T12:27:31.271Z|00074|bond|INFO|member ens4f0v5: enabled 2023-06-14T12:27:31.271Z|00075|bond|INFO|member ens4f1v11: link state up 2023-06-14T12:27:31.271Z|00076|bond|INFO|member ens4f1v11: enabled 2023-06-14T12:27:31.271Z|00077|bond|INFO|bond balance-slb: active member is now ens4f0v5 2023-06-14T12:27:31.910Z|00001|bond(handler4)|DBG|bond balance-slb: member ens4f1v11: admissibility verdict is to drop pkt as different port is learned, active member: false, may_enable: true, enabled: true, LACP status: off 2023-06-14T12:27:31.910Z|00001|bond(handler3)|DBG|bond balance-slb: member ens4f0v5: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-06-14T12:27:31.910Z|00002|bond(handler3)|DBG|bond balance-slb: member ens4f0v5: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-06-14T12:27:31.910Z|00003|bond(handler3)|DBG|bond balance-slb: member ens4f0v5: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-06-14T12:27:32.283Z|00001|bond(revalidator5)|DBG|bond balance-slb: member ens4f0v5: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-06-14T12:27:41.295Z|00078|bond|DBG|bond balance-slb: ens4f0v5 1kB (h40: 1kB), ens4f1v11 0kB 2023-06-14T12:27:51.696Z|00079|bond|DBG|bond balance-slb: ens4f0v5 2kB (h40: 2kB), ens4f1v11 0kB 2023-06-14T12:28:01.707Z|00080|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:28:11.720Z|00081|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:28:21.731Z|00082|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:28:31.743Z|00083|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:28:39.686Z|00002|bond(handler4)|DBG|Dropped 5 log messages in last 67 seconds (most recently, 15 seconds ago) due to excessive rate 2023-06-14T12:28:39.686Z|00003|bond(handler4)|DBG|bond balance-slb: member ens4f1v11: admissibility verdict is to drop pkt as different port is learned, active member: false, may_enable: true, enabled: true, LACP status: off 2023-06-14T12:28:41.754Z|00084|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:28:51.764Z|00085|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:29:01.778Z|00086|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:29:11.788Z|00087|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:29:21.799Z|00088|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB 2023-06-14T12:29:31.810Z|00089|bond|DBG|bond balance-slb: ens4f0v5 3kB (h40: 3kB), ens4f1v11 0kB use tcpdump to capture packets from each slave interface. # tcpdump -i ens4f0v5 dropped privs to tcpdump tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on ens4f0v5, link-type EN10MB (Ethernet), snapshot length 262144 bytes 08:28:11.846088 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 912, seq 270, length 64 08:28:11.846117 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 912, seq 270, length 64 08:28:12.038082 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 913, seq 36, length 64 08:28:12.038108 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 913, seq 36, length 64 08:28:12.870082 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 912, seq 271, length 64 08:28:12.870109 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 912, seq 271, length 64 08:28:13.062084 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 913, seq 37, length 64 08:28:13.062111 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 913, seq 37, length 64 08:28:13.894081 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 912, seq 272, length 64 08:28:13.894101 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 912, seq 272, length 64 08:28:14.086084 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 913, seq 38, length 64 08:28:14.086104 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 913, seq 38, length 64 08:28:14.918081 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 912, seq 273, length 64 08:28:14.918102 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 912, seq 273, length 64 08:28:15.110085 IP 172.31.150.42 > 172.31.150.1: ICMP echo request, id 913, seq 39, length 64 08:28:15.110104 IP 172.31.150.1 > 172.31.150.42: ICMP echo reply, id 913, seq 39, length 64 08:28:15.442378 LLDP, length 270: sw-cisco9364 # tcpdump -i ens4f1v11 dropped privs to tcpdump tcpdump: verbose output suppressed, use -v[v]... for full protocol decode listening on ens4f1v11, link-type EN10MB (Ethernet), snapshot length 262144 bytes 08:28:29.071321 LLDP, length 270: sw-cisco9364 08:28:34.073341 LLDP, length 270: sw-cisco9364 08:28:39.075216 LLDP, length 270: sw-cisco9364 08:28:39.686229 ARP, Reply 172.31.150.1 is-at 40:a6:b7:3e:a5:60 (oui Unknown), length 46 08:28:44.076963 LLDP, length 270: sw-cisco9364 08:28:49.079388 LLDP, length 270: sw-cisco9364 08:28:54.080940 LLDP, length 270: sw-cisco9364 08:28:59.082342 LLDP, length 270: sw-cisco9364 08:29:04.084252 LLDP, length 270: sw-cisco9364 08:29:09.086490 LLDP, length 270: sw-cisco9364 08:29:14.088800 LLDP, length 270: sw-cisco9364 08:29:15.290478 ARP, Request who-has 172.31.150.42 tell 172.31.150.1, length 46 08:29:19.090864 LLDP, length 270: sw-cisco9364 08:29:22.694129 ARP, Reply 172.31.150.1 is-at 40:a6:b7:3e:a5:60 (oui Unknown), length 46 08:29:24.092902 LLDP, length 270: sw-cisco9364 08:29:29.094960 LLDP, length 270: sw-cisco9364 08:29:34.096839 LLDP, length 270: sw-cisco9364 08:29:39.098052 LLDP, length 270: sw-cisco9364 08:29:44.100178 LLDP, length 270: sw-cisco9364 08:29:49.101688 LLDP, length 270: sw-cisco9364 08:29:54.103838 LLDP, length 270: sw-cisco9364 08:29:59.105299 LLDP, length 270: sw-cisco9364 08:30:02.118068 IP6 hp-dl388g10-03.rhts.eng.pek2.redhat.com > ff02::2: ICMP6, router solicitation, length 16 08:30:04.106882 LLDP, length 270: sw-cisco9364 08:30:04.442470 ARP, Request who-has 172.31.150.42 tell 172.31.150.1, length 46
smae issue was found on juniper switch and cx5 nic. [root@dell-per740-09 ovs_bond_function]# ethtool -i ens3f0np0 driver: mlx5_core version: 5.14.0-333.el9.x86_64 firmware-version: 26.33.1048 (MT_0000000531) expansion-rom-version: bus-info: 0000:5e:00.0 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes [root@dell-per740-09 ovs_bond_function]# ethtool -i ens3f1np1 driver: mlx5_core version: 5.14.0-333.el9.x86_64 firmware-version: 26.33.1048 (MT_0000000531) expansion-rom-version: bus-info: 0000:5e:00.1 supports-statistics: yes supports-test: yes supports-eeprom-access: no supports-register-dump: no supports-priv-flags: yes # ovs-appctl bond/show ---- balance-slb ---- bond_mode: balance-slb bond may use recirculation: no, Recirc-ID : -1 bond-hash-basis: 0 lb_output action: disabled, bond-id: -1 all members active: false updelay: 0 ms downdelay: 0 ms next rebalance: 9967 ms lacp_status: off lacp_fallback_ab: false active-backup primary: <none> active member mac: b8:3f:d2:43:28:94(ens3f0np0) member ens3f0np0: enabled active member may_enable: true hash 4: 15159948 kB load member ens3f1np1: enabled may_enable: true hash 24: 15249907 kB load 2023-07-16T10:29:49.853Z|00067|bond|DBG|bond balance-slb: ens3f0np0 0kB (h4: 0kB + h24: 0kB + h108: 0kB + h223: 0kB), ens3f1np1 0kB (h34: 0kB) 2023-07-16T10:29:59.863Z|00068|bond|DBG|bond balance-slb: ens3f0np0 0kB (h4: 0kB + h24: 0kB + h108: 0kB + h223: 0kB), ens3f1np1 0kB (h34: 0kB) 2023-07-16T10:30:09.874Z|00069|bond|DBG|bond balance-slb: ens3f0np0 0kB (h108: 0kB + h4: 0kB + h24: 0kB + h223: 0kB), ens3f1np1 0kB (h34: 0kB) 2023-07-16T10:30:10.392Z|00002|bond(handler1)|DBG|bond balance-slb: member ens3f0np0: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-07-16T10:30:10.881Z|00002|bond(revalidator7)|DBG|bond balance-slb: member ens3f0np0: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-07-16T10:30:15.858Z|00001|bond(handler8)|DBG|bond balance-slb: member ens3f0np0: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-07-16T10:30:15.892Z|00003|bond(revalidator7)|DBG|bond balance-slb: member ens3f0np0: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-07-16T10:30:19.897Z|00070|bond|DBG|bond balance-slb: ens3f0np0 0kB (h4: 0kB + h24: 0kB + h223: 0kB + h108: 0kB), ens3f1np1 0kB 2023-07-16T10:30:29.908Z|00071|bond|DBG|bond balance-slb: ens3f0np0 7154567kB (h4: 3783174kB + h24: 3371392kB + h223: 0kB + h108: 0kB), ens3f1np1 0kB (h34: 0kB) 2023-07-16T10:30:29.908Z|00072|bond|INFO|bond balance-slb: shift 3371392kB of load (with hash 24) from ens3f0np0 to ens3f1np1 (now carrying 3783174kB and 3371392kB load, respectively) 2023-07-16T10:30:39.918Z|00073|bond|DBG|bond balance-slb: ens3f0np0 17554033kB (h4: 17554033kB + h223: 0kB + h108: 0kB), ens3f1np1 16632498kB (h24: 16632498kB + h34: 0kB) 2023-07-16T10:30:43.161Z|00003|bond(handler1)|DBG|Dropped 22 log messages in last 27 seconds (most recently, 13 seconds ago) due to excessive rate 2023-07-16T10:30:43.161Z|00004|bond(handler1)|DBG|bond balance-slb: member ens3f0np0: admissibility verdict is to drop pkt as different port is learned, active member: true, may_enable: true, enabled: true, LACP status: off 2023-07-16T10:30:49.930Z|00074|bond|DBG|bond balance-slb: ens3f0np0 23941097kB (h4: 23941097kB + h223: 0kB + h108: 0kB), ens3f1np1 23878170kB (h24: 23878170kB + h34: 0kB)