Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 1779110

Summary:	[OVN] ovn-controller virtual port unbinding might trigger multiple SB DB updates
Product:	Red Hat Enterprise Linux Fast Datapath	Reporter:	Dumitru Ceara <dceara>
Component:	ovn2.12	Assignee:	Dumitru Ceara <dceara>
Status:	CLOSED ERRATA	QA Contact:	Jianlin Shi <jishi>
Severity:	low	Docs Contact:
Priority:	unspecified
Version:	RHEL 8.0	CC:	ctrautma, jishi, kfida, ralongi
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1779112 1779114 (view as bug list)		Environment:
Last Closed:	2020-01-21 06:20:15 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1779112, 1779114

Description Dumitru Ceara 2019-12-03 10:39:24 UTC

Description of problem:
In the case when a virtual port binding is processed by ovn-controller before its virtual_parent, consider_local_datapath might not release it in the current     ovn-controller iteration even though the virtual_parent gets released.

Backport of upstream commit:
https://github.com/ovn-org/ovn/commit/5309099ec38cf41f4e41f1929c408741a3146dac

Version-Release number of selected component (if applicable):

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 3 Jianlin Shi 2019-12-23 01:03:10 UTC

Hi Dumitru,

any suggestions about how to test the issue? thanks

BR,
Jianlin Shi

Comment 4 Dumitru Ceara 2020-01-03 10:25:15 UTC

Hi Jianlin Shi,

One way to validate the fix is to test as below.

Regards,
Dumitru

# Configure a logical switch with two VMs attached to it:
ovn-nbctl ls-add ls
ovn-nbctl lsp-add ls vm1
ovn-nbctl lsp-set-addresses vm1 00:00:00:00:00:01
ovn-nbctl lsp-add ls vm2
ovn-nbctl lsp-set-addresses vm1 00:00:00:00:00:02
ip netns add vm1
ovs-vsctl add-port br-int vm1 -- set interface vm1 type=internal
ip link set vm1 netns vm1
ip netns exec vm1 ip link set vm1 address 00:00:00:00:00:01
ip netns exec vm1 ip addr add 42.42.42.2/24 dev vm1
ip netns exec vm1 ip link set vm1 up
ovs-vsctl set Interface vm1 external_ids:iface-id=vm1
ip netns add vm2
ovs-vsctl add-port br-int vm2 -- set interface vm2 type=internal
ip link set vm2 netns vm2
ip netns exec vm2 ip link set vm2 address 00:00:00:00:00:02
ip netns exec vm2 ip addr add 42.42.42.3/24 dev vm2
ip netns exec vm2 ip link set vm2 up
ovs-vsctl set Interface vm2 external_ids:iface-id=vm2

# Configure a virtual port
ovn-nbctl lsp-add ls vm-virt
ovn-nbctl lsp-set-type vm-virt virtual
ovn-nbctl set logical_switch_port vm-virt options:virtual-ip=42.42.42.42
ovn-nbctl set logical_switch_port vm-virt options:virtual-parents=vm1,vm2

# Bind the vport
ip netns exec vm1 ip addr add dev vm1 42.42.42.42/24
ip netns exec vm1 arping -I vm1 -s 42.42.42.42 42.42.42.42

# Enable logs for I-P engine runs and bindings
ovn-appctl -t ovn-controller vlog/set main::DBG
ovn-appctl -t ovn-controller vlog/set binding::DBG

# Release the vport
ovs-vsctl clear interface vm1 external_ids

# Check the logs and make sure there's no engine run (no log in "main") betwen:
# "Releasing lport vm1 from this chassis." and
# "Releasing lport vm-virt from this chassis"

Comment 5 Jianlin Shi 2020-01-06 08:50:20 UTC

reproduced on ovn2.12-2.12.0-14.el8fdp.x86_64 with reproducer in comment 4:

[root@dell-per740-12 bz1779110]# bash -x setup.sh                                                     
+ systemctl start openvswitch
+ systemctl start ovn-northd
+ ovn-nbctl set-connection ptcp:6641
+ ovn-sbctl set-connection ptcp:6642
+ ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:20.0.30.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=20.0.30.25
+ systemctl restart ovn-controller
+ ovn-nbctl ls-add ls
+ ovn-nbctl lsp-add ls vm1
+ ovn-nbctl lsp-set-addresses vm1 00:00:00:00:00:01
+ ovn-nbctl lsp-add ls vm2
+ ovn-nbctl lsp-set-addresses vm1 00:00:00:00:00:02                                                   
+ ip netns add vm1
+ ovs-vsctl add-port br-int vm1 -- set interface vm1 type=internal                                    
+ ip link set vm1 netns vm1
+ ip netns exec vm1 ip link set vm1 address 00:00:00:00:00:01
+ ip netns exec vm1 ip addr add 42.42.42.2/24 dev vm1                                                 
+ ip netns exec vm1 ip link set vm1 up
+ ovs-vsctl set Interface vm1 external_ids:iface-id=vm1                                               
+ ip netns add vm2
+ ovs-vsctl add-port br-int vm2 -- set interface vm2 type=internal                                    
+ ip link set vm2 netns vm2                                                                           
+ ip netns exec vm2 ip link set vm2 address 00:00:00:00:00:02                                         
+ ip netns exec vm2 ip addr add 42.42.42.3/24 dev vm2                                                 
+ ip netns exec vm2 ip link set vm2 up                                                                
+ ovs-vsctl set Interface vm2 external_ids:iface-id=vm2                                               
+ ovn-nbctl lsp-add ls vm-virt                                                                        
+ ovn-nbctl lsp-set-type vm-virt virtual                                                              
+ ovn-nbctl set logical_switch_port vm-virt options:virtual-ip=42.42.42.42                            
+ ovn-nbctl set logical_switch_port vm-virt options:virtual-parents=vm1,vm2                           
+ ip netns exec vm1 ip addr add dev vm1 42.42.42.42/24                                                
+ ip netns exec vm1 arping -I vm1 -s 42.42.42.42 42.42.42.42 -c 10                                    
ARPING 42.42.42.42 from 42.42.42.42 vm1                                                               
Sent 10 probes (10 broadcast(s))                                                                      
Received 0 response(s)                                                                                
+ ovn-appctl -t ovn-controller vlog/set main::DBG                                                     
+ ovn-appctl -t ovn-controller vlog/set binding::DBG                                                  
+ ovs-vsctl clear interface vm1 external_ids

log in /var/log/ovn/ovn-controller.log:

2020-01-06T08:46:28.035Z|00026|binding|INFO|Releasing lport vm1 from this chassis.
2020-01-06T08:46:28.035Z|00027|main|DBG|removing ct zone 2 for 'vm1'
2020-01-06T08:46:28.037Z|00028|main|DBG|engine did not run, force recompute next time: br_int 0x560a400
d51ae0, chassis 0x560a40d50f80
2020-01-06T08:46:28.038Z|00029|binding|INFO|Releasing lport vm-virt from this chassis.

Verified on ovn2.12-2.12.0-19.el8fdp.x86_64:

2020-01-06T08:49:34.106Z|00026|binding|INFO|Releasing lport vm1 from this chassis.
2020-01-06T08:49:34.106Z|00027|binding|INFO|Releasing lport vm-virt from this chassis.
2020-01-06T08:49:34.106Z|00028|main|DBG|removing ct zone 2 for 'vm1'

Comment 6 Jianlin Shi 2020-01-06 09:28:30 UTC

reproduced on ovn2.12-host-2.12.0-14.el7fdp.x86_64:

2020-01-06T09:22:48.647Z|00026|binding|INFO|Releasing lport vm1 from this chassis.                    
2020-01-06T09:22:48.647Z|00027|main|DBG|removing ct zone 2 for 'vm1'                                  
2020-01-06T09:22:48.648Z|00028|main|DBG|engine did not run, and it was not needed either: br_int 0x56 
528ea722f0, chassis 0x56528ea8d140                                                                    
2020-01-06T09:22:48.649Z|00029|main|DBG|engine did not run, force recompute next time: br_int 0x56528 
ea722f0, chassis 0x56528ea8d140                                                                   

<=== log between two releasing log
    
2020-01-06T09:22:48.649Z|00030|binding|INFO|Releasing lport vm-virt from this chassis. 

Verified on 2.12.0-19:

2020-01-06T09:27:28.039Z|00026|binding|INFO|Releasing lport vm1 from this chassis.                    
2020-01-06T09:27:28.039Z|00027|binding|INFO|Releasing lport vm-virt from this chassis.

[root@hp-dl380pg8-12 bz1779110]# rpm -qa | grep -E "openvswitch|ovn"
openvswitch2.12-2.12.0-12.el7fdp.x86_64                                                               
ovn2.12-central-2.12.0-19.el7fdp.x86_64                                                               
openvswitch-selinux-extra-policy-1.0-14.el7fdp.noarch                                                 
ovn2.12-2.12.0-19.el7fdp.x86_64                                                                       
ovn2.12-host-2.12.0-19.el7fdp.x86_64

Comment 8 errata-xmlrpc 2020-01-21 06:20:15 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0167