Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 2173635

Summary: Geneve tunnel interfaces are not moved when ovn-bridge is changed
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Jakub Libosvar <jlibosva>
Component: ovn22.12Assignee: Ales Musil <amusil>
Status: CLOSED ERRATA QA Contact: Ehsan Elahi <eelahi>
Severity: medium Docs Contact:
Priority: medium    
Version: FDP 22.LCC: amusil, ctrautma, jiji, jishi, mmichels
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovn22.12-22.12.0-44.el8fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-18 00:16:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jakub Libosvar 2023-02-27 14:55:22 UTC 
Description of problem:
This is in a context of ml2/ovs to ml2/ovn OpenStack migration. What we do during the data plane migration is that new br-migration ovn-bridge is introduced and ovn-controller is started. Once everything is ready for the data plane cutoff, the ovn-migration bridge is changed to br-int and OVN takes over. However, the Geneve interfaces (ovn-*) remain on the former br-migration until the bridge is removed. Then the interfaces are correctly re-created on the br-int bridge.

The problem is that traffic that goes through the tunnels is disrupted for a longer period of time. 


Version-Release number of selected component (if applicable):
ovn22.12-22.12.0-20.el9fdp.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Create OVS bridge B
2. Set external_ids:ovn-bridge=B
3. Make sure tunnel interfaces are on B
4. Change external_ids:ovn-bridge back to br-int

Actual results:
Tunnel interfaces remain on B

Expected results:
Tunnel interfaces are re-created on br-int


Additional info:
Comment 1 Ales Musil 2023-03-30 07:53:58 UTC 
Patch posted u/s: https://patchwork.ozlabs.org/project/ovn/patch/20230330075259.371472-1-amusil@redhat.com/
Comment 2 OVN Bot 2023-04-12 04:01:22 UTC 
ovn23.03 fast-datapath-rhel-8 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2186081
ovn23.03 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2186082
ovn22.12 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2186083
Comment 6 Ehsan Elahi 2023-04-26 17:30:59 UTC 
Reproduced On:
# rpm -qa | grep -E 'ovn|openvswitch'
ovn22.12-22.12.0-35.el8fdp.x86_64
ovn22.12-central-22.12.0-35.el8fdp.x86_64
ovn22.12-host-22.12.0-35.el8fdp.x86_64
openvswitch2.17-2.17.0-74.el8fdp.x86_64
openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch

Verified On:
# rpm -qa | grep -E 'ovn|openvswitch'
ovn22.12-22.12.0-51.el8fdp.x86_64
ovn22.12-central-22.12.0-51.el8fdp.x86_64
ovn22.12-host-22.12.0-51.el8fdp.x86_64
openvswitch2.17-2.17.0-74.el8fdp.x86_64
openvswitch-selinux-extra-policy-1.0-29.el8fdp.noarch

Here is the reproducer:

######### HV1 ###########
systemctl start ovn-northd
ovn-nbctl set-connection ptcp:6641
ovn-sbctl set-connection ptcp:6642
systemctl start openvswitch
ovs-vsctl set open . external_ids:system-id=hv1
ifconfig ens1f0 192.168.20.1 netmask 255.255.255.0
ovs-vsctl set open . external_ids:ovn-remote=tcp:192.168.20.1:6642
ovs-vsctl set open . external_ids:ovn-encap-type=geneve
ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.20.1
ovs-vsctl set open . external_ids:ovn-monitor-all=true
systemctl start ovn-controller

ovs-vsctl add-br br-ex
ovs-vsctl set open . external_ids:ovn-bridge-mappings=physnet1:br-ex

ovn-nbctl ls-add public
ovn-nbctl ls-add internal

ovn-nbctl lsp-add public ln_port
ovn-nbctl lsp-set-addresses ln_port unknown
ovn-nbctl lsp-set-type ln_port localnet
ovn-nbctl lsp-set-options ln_port network_name=physnet1

# should be 1
ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv0.20.2 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv0.20.2)
ovs-vsctl --bare --columns ports find bridge name=br-int | grep "$tunnel_id"
prev_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv0.20.2")
# Stop ovn-controller on hv1
ovn-appctl -t ovn-controller exit --restart
# The tunnel should remain intact
ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv0.20.2 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv0.20.2)
ovs-vsctl --bare --columns ports find bridge name=br-int | grep "$tunnel_id"
current_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv0.20.2")
# The tunnel should be the same after restart
# "$current_id" == "$prev_id"

# Change the bridge to br-int1 on hv1
ovs-vsctl add-br br-int1
ovs-vsctl set open . external_ids:ovn-bridge="br-int1"
systemctl restart ovn-controller
ovn-appctl -t ovn-controller vlog/set file:dbg
ovn-nbctl --wait=hv sync
# Check that the tunnel was created on br-int1 instead
ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv0.20.2 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv0.20.2)
ovs-vsctl --bare --columns ports find bridge name=br-int1 | grep "$tunnel_id"
### tunnel_id does not changed in non-fixed version. But changed in fixed release. 

cat /var/log/ovn/ovn-controller.log |grep "Clearing old tunnel port \"ovn-hv0-0\" (hv0.20.2) from bridge \"br-int\""
### ovn-controller does not log in non-fixed version. 

ovn-appctl -t ovn-controller exit --restart

# The tunnel should remain intact
ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv0.20.2 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv0.20.2)
ovs-vsctl --bare --columns ports find bridge name=br-int1 | grep "$tunnel_id"

prev_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv0.20.2")

systemctl restart ovn-controller
ovn-appctl -t ovn-controller vlog/set file:dbg
ovn-nbctl --wait=hv sync
ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv0.20.2 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv0.20.2)
ovs-vsctl --bare --columns ports find bridge name=br-int1 | grep "$tunnel_id"
current_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id="hv0.20.2")
# The tunnel should be the same after restart
# "$current_id" == "$prev_id"

ovs-vsctl del-br br-ex
ovs-vsctl del-br br-int1

######### HV0 ###########
systemctl start ovn-northd
systemctl start openvswitch
ovs-vsctl set open . external_ids:system-id=hv0
ifconfig ens5f0 192.168.20.2 netmask 255.255.255.0
ovs-vsctl set open . external_ids:ovn-remote=tcp:192.168.20.1:6642
ovs-vsctl set open . external_ids:ovn-encap-type=geneve
ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.20.2
ovs-vsctl set open . external_ids:ovn-monitor-all=true
systemctl start ovn-controller

ovs-vsctl add-br br-ex
ovs-vsctl set open . external_ids:ovn-bridge-mappings=physnet1:br-ex
ovn-appctl -t ovn-controller vlog/set file:dbg
# should be 1
ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv1.20.1 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv1.20.1)
ovs-vsctl --bare --columns ports find bridge name=br-int | grep "$tunnel_id"

ovs-vsctl add-br br-int1
ovs-vsctl set open . external_ids:ovn-bridge="br-int1"
ovn-nbctl --wait=hv sync

ovs-vsctl --format=table --no-headings find port external_ids:ovn-chassis-id=hv1.20.1 | wc -l
tunnel_id=$(ovs-vsctl --bare --columns _uuid find port external_ids:ovn-chassis-id=hv1.20.1)
ovs-vsctl --bare --columns ports find bridge name=br-int1 | grep "$tunnel_id"
### tunnel_id does not changed in non-fixed version. But changed in fixed release. 

cat /var/log/ovn/ovn-controller.log |grep "Clearing old tunnel port \"ovn-hv1-0\" (hv1.20.1) from bridge \"br-int\""
### ovn-controller does not log in non-fixed version. 
ovs-vsctl del-br br-int1
Comment 8 errata-xmlrpc 2023-05-18 00:16:36 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ovn22.12 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:3203