Bug 2221359

Summary: [ovs bond balance-tcp] bond-primary option didn't work after remove lacp active channel group
Product: Red Hat Enterprise Linux Fast Datapath Reporter: mhou <mhou>
Component: openvswitchAssignee: Mike Pattrick <mpattric>
openvswitch sub component: daemons and tools QA Contact: mhou <mhou>
Status: CLOSED DUPLICATE Docs Contact:
Severity: high    
Priority: unspecified CC: ctrautma, jhsiao
Version: RHEL 9.0   
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-07-08 10:07:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description mhou 2023-07-08 09:56:37 UTC 
Description of problem:

Configure lacp channel group as active-active or passive-active. That means configure balance-tcp with lacp active and configure peer physical switch site lacp channel group as active.

When remove lacp channel group from peer physical side, current active memeber of balance-tcp is not equal to which setting on other_config:bond-primary.  

Version-Release number of selected component (if applicable):
kernel version:5.14.0-332.el9.x86_64+rt
ovs version: openvswitch3.1-3.1.0-33.el9fdp.x86_64

How reproducible: 100%


Steps to Reproduce:
1. create balance-tcp port and configure lacp as active.
ovs-vsctl add-bond bondbridge balance-tcp ens4f0 ens4f1 lacp=passive bond_mode=balance-tcp  -- set Interface ens4f0 mtu_request=9000  -- set Interface ens4f1 mtu_request=9000 -- set Port balance-tcp other_config:lacp-fallback-ab=true other_config:bond-primary=ens4f0

2. configure active lacp channel group on peer switch site.
Running 'swcfg setup_port_channel 9364 'Eth1/7/1 Eth1/7/2' active'
spawn ssh admin.88.13
User Access Verification
(admin.88.13) Password: 

Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2022, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source.  This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or 
GNU General Public License (GPL) version 3.0  or the GNU
Lesser General Public License (LGPL) Version 2.1 or 
Lesser General Public License (LGPL) Version 2.0. 
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.
sw-cisco9364# \r
               ^
% Incomplete command at '^' marker.
sw-cisco9364# terminal length 0
sw-cisco9364# configure t
Enter configuration commands, one per line. End with CNTL/Z.
sw-cisco9364(config)# show running-config interface Eth1/7/1 

!Command: show running-config interface Ethernet1/7/1
!Running configuration last done at: Sat Jul  8 09:07:52 2023
!Time: Sat Jul  8 09:07:54 2023

version 9.3(10) Bios:version 05.47 

interface Ethernet1/7/1
  mtu 9216
  inherit port-profile mhou_profile

interface Eth1/7/1
sw-cisco9364(config)# interface Eth1/7/1
no inherit port-profile mhou_profile
sw-cisco9364(config-if)# no inherit port-profile mhou_profile
lacp rate normal
Message reported by command ::             no switchport mode
Warning: If the interface is configured with mappings, all the associated mappings would be removed when port mode is changed
sw-cisco9364(config-if)# lacp rate normal
channel-group 57 force mode active
sw-cisco9364(config-if)# channel-group 57 force mode active
show running-config interface Eth1/7/2 
sw-cisco9364(config-if)# show running-config interface Eth1/7/2 

!Command: show running-config interface Ethernet1/7/2
!Running configuration last done at: Sat Jul  8 09:07:59 2023
!Time: Sat Jul  8 09:07:59 2023

version 9.3(10) Bios:version 05.47 

interface Ethernet1/7/2
  mtu 9216
  inherit port-profile mhou_profile

interface Eth1/7/2
sw-cisco9364(config-if)# interface Eth1/7/2
no inherit port-profile mhou_profile
sw-cisco9364(config-if)# no inherit port-profile mhou_profile
lacp rate normal
Message reported by command ::             no switchport mode
Warning: If the interface is configured with mappings, all the associated mappings would be removed when port mode is changed
sw-cisco9364(config-if)# lacp rate normal
channel-group 57 force mode active
sw-cisco9364(config-if)# channel-group 57 force mode active
interface port-channel57
sw-cisco9364(config-if)# interface port-channel57
shutdown
sw-cisco9364(config-if)# shutdown
no lacp suspend-individual
sw-cisco9364(config-if)# no lacp suspend-individual
lacp graceful-convergence
Warning: !! Disable lacp suspend-individual only on port-channel with edge ports. Disabling this on network port port-channel could lead to loops.!
sw-cisco9364(config-if)# lacp graceful-convergence
no shutdown
sw-cisco9364(config-if)# no shutdown
inherit port-profile mhou_profile_channel
sw-cisco9364(config-if)# inherit port-profile mhou_profile_channel
mtu 9216
sw-cisco9364(config-if)# mtu 9216
end
sw-cisco9364(config-if)# end
sw-cisco9364# show port-channel summary
show port-channel internal info interface port-channel 57
Flags:  D - Down        P - Up in port-channel (members)
        I - Individual  H - Hot-standby (LACP only)
        s - Suspended   r - Module-removed
        b - BFD Session Wait
        S - Switched    R - Routed
        U - Up (port-channel)
        p - Up in delay-lacp mode (member)
        M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port-       Type     Protocol  Member Ports
      Channel
--------------------------------------------------------------------------------
57    Po57(SD)    Eth      LACP      Eth1/7/1(D)  Eth1/7/2(D)  
sw-cisco9364# show port-channel internal info interface port-channel 57
end

port-channel57
channel    : 57
bundle     : 65535
ifindex    : 0x16000038
admin mode : active
oper mode  : active
nports     : 2
active     : 0
pre cfg    : 0
ltl        : 0x2 (2)
lif        : 0x1059
iod        : 0x5b (91)
global id  : 1
flag       : 0
lock count : 0
num. of SIs: 0
ac mbrs    : 0 0
lacp graceful conv disable   : 0 
lacp suspend indiv disable   : 1 
pc min-links                 : 1 
pc max-bundle                : 32 
pc max active members        : 32 
pc is-suspend-minlinks       : 0 
port load defer enable       : 0 
port-channel bfd config enabled     : 0 
port-channel bfd config complete: 0 
port-channel bfd destination: null 
port-channel bfd start timeout: 0 
lacp fast-select-hot-standby disable   : 0 
port-channel port hash-distribution    : none
ethpm bundle lock count : 0
lacp delayed-enable fop none 0x0 
lacp delayed-enable : 0 
lacp delayed-enable cfg-port none
lacp delayed-enable oper-port none
lacp delayed-enable local best priority : 0xffffffff 
lacp delayed-enable remote best priority : 0xffffffff 
lacp vpc conv enabled   : 0 
gir conv enabled   : 0 
bundle number map:
Members: 
Ethernet1/7/1     is_ltl_programmed = 0
Port BFD session state: 5 (none)
Ethernet1/7/2     is_ltl_programmed = 0
Port BFD session state: 5 (none)
port-channel external lock: 
Lock Info: resource [eth-port-channel 57] 
  type[0] p_gwrap[(nil)]
      FREE @ 916395 usecs after Sat Jul  8 09:07:57 2023
  type[1] p_gwrap[(nil)]
      FREE @ 196102 usecs after Sat Jul  8 09:08:05 2023
  type[2] p_gwrap[(nil)]
      FREE @ 580008 usecs after Sat Jul  8 09:08:03 2023
0x16000038
internal (ethpm bundle) lock: 
Lock Info: resource [eth-port-channel 57] 
  type[0] p_gwrap[(nil)]
      FREE @ 916382 usecs after Sat Jul  8 09:07:57 2023
  type[1] p_gwrap[(nil)]
      FREE @ 320113 usecs after Sat Jul  8 09:08:02 2023
  type[2] p_gwrap[(nil)]
      FREE @ 916382 usecs after Sat Jul  8 09:07:57 2023
0x16000038
sw-cisco9364# end


3. check the bond-primary and lacp-fallback-ab already effect on balance-tcp port.
# ovs-appctl bond/show
---- balance-tcp ----
bond_mode: balance-tcp
bond may use recirculation: yes, Recirc-ID : 1
bond-hash-basis: 0
lb_output action: disabled, bond-id: -1
updelay: 0 ms
downdelay: 0 ms
next rebalance: 6024 ms
lacp_status: negotiated
lacp_fallback_ab: true
active-backup primary: ens4f0
active member mac: 3c:fd:fe:bd:1c:a4(ens4f0)

member ens4f0: enabled
  active member
  may_enable: true

member ens4f1: enabled
  may_enable: true


4. remove lacp channel group from peer siwtch site.
# swcfg cleanup_port_channel 9364 'Eth1/7/1 Eth1/7/2'
spawn ssh admin.88.13
User Access Verification
(admin.88.13) Password: 

Cisco Nexus Operating System (NX-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (C) 2002-2022, Cisco and/or its affiliates.
All rights reserved.
The copyrights to certain works contained in this software are
owned by other third parties and used and distributed under their own
licenses, such as open source.  This software is provided "as is," and unless
otherwise stated, there is no warranty, express or implied, including but not
limited to warranties of merchantability and fitness for a particular purpose.
Certain components of this software are licensed under
the GNU General Public License (GPL) version 2.0 or 
GNU General Public License (GPL) version 3.0  or the GNU
Lesser General Public License (LGPL) Version 2.1 or 
Lesser General Public License (LGPL) Version 2.0. 
A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://opensource.org/licenses/gpl-3.0.html and
http://www.opensource.org/licenses/lgpl-2.1.php and
http://www.gnu.org/licenses/old-licenses/library.txt.
sw-cisco9364# \r
               ^
% Incomplete command at '^' marker.
sw-cisco9364# terminal length 0
sw-cisco9364# clear counters interface Eth1/7/1
sw-cisco9364# clear counters interface Eth1/7/2
sw-cisco9364# show running-config interface port-channel 57 

!Command: show running-config interface port-channel57
!Running configuration last done at: Sat Jul  8 09:08:07 2023
!Time: Sat Jul  8 09:10:50 2023

version 9.3(10) Bios:version 05.47 

interface port-channel57
  mtu 9216
  no lacp suspend-individual
  inherit port-profile mhou_profile_channel

configure t
sw-cisco9364# configure t
Enter configuration commands, one per line. End with CNTL/Z.
sw-cisco9364(config)# interface port-channel 57
no inherit port-profile mhou_profile_channel
sw-cisco9364(config-if)# no inherit port-profile mhou_profile_channel
exit
Message reported by command ::             no switchport mode
Warning: If the interface is configured with mappings, all the associated mappings would be removed when port mode is changed
sw-cisco9364(config-if)# exit
sw-cisco9364(config)# no interface port-channel57
interface Eth1/7/1
sw-cisco9364(config)# interface Eth1/7/1
no lacp rate fast
sw-cisco9364(config-if)# no lacp rate fast
inherit port-profile mhou_profile
sw-cisco9364(config-if)# inherit port-profile mhou_profile
no shutdown
sw-cisco9364(config-if)# no shutdown
mtu 9216
sw-cisco9364(config-if)# mtu 9216
interface Eth1/7/2
sw-cisco9364(config-if)# interface Eth1/7/2
no lacp rate fast
sw-cisco9364(config-if)# no lacp rate fast
inherit port-profile mhou_profile
sw-cisco9364(config-if)# inherit port-profile mhou_profile
no shutdown
sw-cisco9364(config-if)# no shutdown
mtu 9216
sw-cisco9364(config-if)# mtu 9216
end
sw-cisco9364(config-if)# end
show port-channel summary
sw-cisco9364# show port-channel summary
end
sw-cisco9364# end

5. check the current balance-tcp status
# ovs-appctl bond/show
---- balance-tcp ----
bond_mode: balance-tcp
bond may use recirculation: no, Recirc-ID : -1
bond-hash-basis: 0
lb_output action: disabled, bond-id: -1
updelay: 0 ms
downdelay: 0 ms
lacp_status: configured
lacp_fallback_ab: true
active-backup primary: ens4f0
active member mac: 3c:fd:fe:bd:1c:a5(ens4f1)

member ens4f0: enabled
  may_enable: true

member ens4f1: enabled
  active member
  may_enable: true

check the ovs-vswitchd.log
2023-07-08T09:20:23.762Z|00067|bond|DBG|bond balance-tcp: ens4f0 0kB (h194: 0kB), ens4f1 0kB
2023-07-08T09:20:33.775Z|00068|bond|DBG|bond balance-tcp: ens4f0 0kB (h194: 0kB), ens4f1 0kB
2023-07-08T09:20:43.788Z|00069|bond|DBG|bond balance-tcp: ens4f0 0kB (h194: 0kB), ens4f1 0kB
2023-07-08T09:20:53.797Z|00070|bond|DBG|bond balance-tcp: ens4f0 0kB (h194: 0kB), ens4f1 0kB
2023-07-08T09:21:03.808Z|00071|bond|DBG|bond balance-tcp: ens4f0 0kB (h194: 0kB), ens4f1 0kB
2023-07-08T09:21:13.817Z|00072|bond|DBG|bond balance-tcp: ens4f0 0kB (h63: 0kB + h194: 0kB), ens4f1 0kB
2023-07-08T09:21:23.456Z|00073|bond|INFO|member ens4f0: link state down
2023-07-08T09:21:23.457Z|00074|bond|INFO|member ens4f0: disabled
2023-07-08T09:21:23.457Z|00075|bond|INFO|bond balance-tcp: active member is now ens4f1
2023-07-08T09:21:23.966Z|00076|bond|DBG|bond balance-tcp: ens4f1 0kB (h80: 0kB + h207: 0kB + h63: 0kB + h194: 0kB)
2023-07-08T09:21:24.347Z|00077|bond|INFO|member ens4f1: link state down
2023-07-08T09:21:24.347Z|00078|bond|INFO|member ens4f1: disabled
2023-07-08T09:21:24.347Z|00079|bond|INFO|bond balance-tcp: all members disabled
2023-07-08T09:21:51.960Z|00080|bond|INFO|member ens4f1: link state up
2023-07-08T09:21:51.960Z|00081|bond|INFO|member ens4f1: enabled
2023-07-08T09:21:51.960Z|00082|bond|INFO|bond balance-tcp: active member is now ens4f1
2023-07-08T09:21:53.414Z|00083|bond|INFO|member ens4f0: link state up
2023-07-08T09:21:53.414Z|00084|bond|INFO|member ens4f0: enabled
2023-07-08T09:23:00.470Z|00001|bond(handler3)|DBG|bond balance-tcp: member ens4f0: admissibility verdict is to drop pkt, active member: false, may_enable: true, enabled: true, LACP status: configured
2023-07-08T09:23:29.142Z|00002|bond(handler3)|DBG|bond balance-tcp: member ens4f0: admissibility verdict is to drop pkt, active member: false, may_enable: true, enabled: true, LACP status: configured
2023-07-08T09:23:33.238Z|00003|bond(handler3)|DBG|bond balance-tcp: member ens4f0: admissibility verdict is to drop pkt, active member: false, may_enable: true, enabled: true, LACP status: configured
2023-07-08T09:24:06.007Z|00004|bond(handler3)|DBG|bond balance-tcp: member ens4f0: admissibility verdict is to drop pkt, active member: false, may_enable: true, enabled: true, LACP status: configured
2023-07-08T09:24:06.682Z|00001|bond(revalidator5)|DBG|bond balance-tcp: member ens4f0: admissibility verdict is to drop pkt, active member: false, may_enable: true, enabled: true, LACP status: configured
2023-07-08T09:25:11.542Z|00005|bond(handler3)|DBG|bond balance-tcp: member ens4f0: admissibility verdict is to drop pkt, active member: false, may_enable: true, enabled: true, LACP status: configured


Actual results:
1. from balance-tcp mode, Configuring bond-primary parameters cannot take effect after triggering lacp-fallback-ab 

Expected results:
1. bond-primary should take effect when triger lacp-fallback-ab

Additional info:
Comment 1 mhou 2023-07-08 10:07:47 UTC 
close this bug as DUP of 2219562

*** This bug has been marked as a duplicate of bug 2219562 ***