Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 1939524

Summary: Changing rate on meter does not take effect unless ovn-controller is restarted
Product: Red Hat Enterprise Linux Fast Datapath Reporter: ffernand <ffernand>
Component: ovn2.13Assignee: lorenzo bianconi <lorenzo.bianconi>
Status: CLOSED ERRATA QA Contact: Ehsan Elahi <eelahi>
Severity: high Docs Contact:
Priority: high    
Version: FDP 20.HCC: akaris, akatz, ctrautma, eelahi, jishi, lorenzo.bianconi, mmichels, ralongi
Target Milestone: ---   
Target Release: FDP 22.C   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: ovn-2021-21.12.0-42.el8fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2073462 (view as bug list) Environment:
Last Closed: 2022-04-25 14:26:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1619266, 2051954, 2073462    

Description ffernand 2021-03-16 15:08:57 UTC
Using ovs/ovn master as well as openvswitch2.13-2.13.0-79.5.el8fdp.x86_64, I notice what while I am able to successfully change the rate attribute on nb and sb dbs, the update is not making its way to ofctl by the ovn-controller(s).

NOTE: **even "ovn-appctl -t ovn-controller recompute" doesn't "fix" it.**, which makes this issue a high priority.

The only work around at this point is to restart the ovn-controller, which can be a very disruptive approach.

Reference deployment taken from this document:
https://docs.google.com/document/d/1L7u_VPRSvSiD4wUMNAgTctagLq72ar7ZwtVvo2QG04o/edit?usp=sharing

[root@compute-1 ~]# rpm -qa | grep openvswitch2
openvswitch2.13-2.13.0-79.5.el8fdp.x86_64
network-scripts-openvswitch2.13-2.13.0-79.5.el8fdp.x86_64

[root@compute-1 ~]# sudo podman exec ovn_controller rpm -qa | grep ovn2.
ovn2.13-20.12.0-17.el8fdp.x86_64
ovn2.13-host-20.12.0-17.el8fdp.x86_64

looking at meter from controller
[root@controller-0 ~]# ovn-nbctl list meter
_uuid               : 10f37ada-576e-45a7-932b-bc1f3ff2e828
bands               : [e5528fca-0d33-4b7d-aa28-e524c889f570]
external_ids        : {}
fair                : []
name                : sgm
unit                : pktps
[root@controller-0 ~]# ovn-nbctl list meter-band ;  # north
_uuid               : e5528fca-0d33-4b7d-aa28-e524c889f570
action              : drop
burst_size          : 0
external_ids        : {}
rate                : 100
[root@controller-0 ~]# ovn-sbctl list meter-band   ; # SOUTH
_uuid               : b7f598a1-d207-4230-9ca5-3906e877042e
action              : drop
burst_size          : 0
rate                : 100

## looking at the meter at compute
[root@compute-1 ~]# sudo ovs-ofctl -OOpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=100    ; # CURRENTLY SET VALUE

## changing the meter 
[root@controller-0 ~]# MB=$(ovn-nbctl --bare --column _uuid list meter-band) ; echo $MB
e5528fca-0d33-4b7d-aa28-e524c889f570
[root@controller-0 ~]# ovn-nbctl set meter-band ${MB} rate=222
[root@controller-0 ~]# ovn-nbctl list meter
_uuid               : 10f37ada-576e-45a7-932b-bc1f3ff2e828
bands               : [e5528fca-0d33-4b7d-aa28-e524c889f570]
external_ids        : {}
fair                : []
name                : sgm
unit                : pktps
[root@controller-0 ~]# ovn-nbctl list meter-band
_uuid               : e5528fca-0d33-4b7d-aa28-e524c889f570
action              : drop
burst_size          : 0
external_ids        : {}
rate                : 222
[root@controller-0 ~]# ovn-sbctl list meter-band
_uuid               : f564c7af-64e6-4567-8c8f-b02c4ad99890
action              : drop
burst_size          : 0
rate                : 222

## compute node(s) never the get the memo
[root@compute-0 ~]# sudo ovs-ofctl -OOpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=100    ; # OLD VALUE

[root@compute-1 ~]# sudo ovs-ofctl -OOpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=100    ; # OLD VALUE

## I also tried:
sudo podman exec ovn_controller ovn-appctl -t ovn-controller recompute

## And on computes
ovs-vsctl set open . external_ids:ovn-monitor-all=true

but only this seemed to make it take effect:

[root@compute-0 ~]# systemctl restart tripleo_ovn_controller.service


[root@compute-0 ~]# sudo ovs-ofctl -OOpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=222   ; # EXPECTED value after ovn-controller restart

Comment 1 Mark Michelson 2021-09-24 13:48:51 UTC
Hi Flavio, this came up during OVN issue prioritization. Given the age of this bug report, do you know if this is still relevant?

Comment 2 Alex Katz 2021-09-29 06:44:02 UTC
Hello Mark,

Flavio is not working on this project anymore.
I can confirm that we are still facing that issue in OpenStack environments.

Comment 4 Andreas Karis 2022-03-03 11:16:17 UTC
Hi there,

Just as an FYI: We are running into this in ovn-kubernetes, too: https://bugzilla.redhat.com/show_bug.cgi?id=2051954#c11

- Andreas

Comment 5 lorenzo bianconi 2022-03-10 16:44:20 UTC
upstream fix: https://github.com/ovn-org/ovn/commit/885655e16e63384484af4607e43fa4c69a1089c5

Comment 8 Jianlin Shi 2022-04-14 08:31:51 UTC
tested with following script:

systemctl start openvswitch                                                                           
systemctl start ovn-northd                                                                            
ovn-nbctl set-connection ptcp:6641                                                                    
ovn-sbctl set-connection ptcp:6642                                                                    
ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:20.0.177.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=20.0.177.25
systemctl restart ovn-controller                                                                      

ovn-nbctl ls-add ls1                                                                                  
ovn-nbctl lsp-add ls1 ls1p1                                                                           
ovn-nbctl lsp-set-addresses ls1p1 "00:00:00:01:01:01 192.168.1.1 2001::1"                             

ovs-vsctl add-port br-int ls1p1 -- set interface ls1p1 type=internal external_ids:iface-id=ls1p1      

ip netns add ls1p1                     
ip link set ls1p1 netns ls1p1              
ip netns exec ls1p1 ip link set ls1p1 address 00:00:00:01:01:01                                       
ip netns exec ls1p1 ip link set ls1p1 up
ip netns exec ls1p1 ip addr add 192.168.1.1/24 dev ls1p1                                              
ip netns exec ls1p1 ip addr add 2001::1/64 dev ls1p1
ip netns exec ls1p1 ip route add default via 192.168.1.254 dev ls1p1                                  
ip netns exec ls1p1 ip -6 route add default via 2001::a dev ls1p1                                     

ovn-nbctl meter-add meter0 drop 50 pktps
ovn-nbctl --log --severity=alert --name=bz1939524 --meter=meter0 acl-add ls1 to-lport 1000 'icmp4' drop
ovn-nbctl --wait=hv sync                                                                              
ovs-ofctl -O OpenFlow15 dump-meters br-int                                                            
band_id=$(ovn-nbctl list meter_band | awk '/_uuid/{print $3}' )                                       
for meter in 30 20 10 -1 5                                                                            
do                                                                                                    
        ovn-nbctl set meter_band $band_id rate=$meter                                                 
                                          
        ovn-nbctl --wait=hv sync                                                                      
        ovs-ofctl -O OpenFlow15 dump-meters br-int                                                    
done 

reproduced on ovn-2021-21.12.0-32.el8:

+ ovn-nbctl set meter_band 4f49cde0-febe-4dac-88d4-8e34657b6fcb rate=30
+ ovn-nbctl --wait=hv sync                  
+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=                         
type=drop rate=50                                                                                     
+ for meter in 30 20 10 -1 5                  
+ ovn-nbctl set meter_band 4f49cde0-febe-4dac-88d4-8e34657b6fcb rate=20
+ ovn-nbctl --wait=hv sync                         
+ ovs-ofctl -O OpenFlow15 dump-meters br-int       
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):        
meter=1 pktps stats bands=                                                                                                                                                                                  
type=drop rate=50

<==== rate not changed

...


Verified on ovn-2021-21.12.0-42:

+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=50
++ ovn-nbctl list meter_band
++ awk '/_uuid/{print $3}'
+ band_id=74c0eec8-0b1e-4c2b-a0bc-48c3391a5908
+ for meter in 30 20 10 -1 5
+ ovn-nbctl set meter_band 74c0eec8-0b1e-4c2b-a0bc-48c3391a5908 rate=30
+ ovn-nbctl --wait=hv sync
+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=30
+ for meter in 30 20 10 -1 5
+ ovn-nbctl set meter_band 74c0eec8-0b1e-4c2b-a0bc-48c3391a5908 rate=20
+ ovn-nbctl --wait=hv sync
+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=20
+ for meter in 30 20 10 -1 5
+ ovn-nbctl set meter_band 74c0eec8-0b1e-4c2b-a0bc-48c3391a5908 rate=10
+ ovn-nbctl --wait=hv sync
+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=10
+ for meter in 30 20 10 -1 5
+ ovn-nbctl set meter_band 74c0eec8-0b1e-4c2b-a0bc-48c3391a5908 rate=-1
ovn-nbctl: constraint violation: -1 is not in the valid range 1 to 4294967295 (inclusive)
+ ovn-nbctl --wait=hv sync
+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=10
+ for meter in 30 20 10 -1 5
+ ovn-nbctl set meter_band 74c0eec8-0b1e-4c2b-a0bc-48c3391a5908 rate=5
+ ovn-nbctl --wait=hv sync
+ ovs-ofctl -O OpenFlow15 dump-meters br-int
OFPST_METER_CONFIG reply (OF1.5) (xid=0x2):
meter=1 pktps stats bands=
type=drop rate=5

<=== rate changed

+ exit 0
[root@wsfd-advnetlab16 bz1939524]# rpm -qa | grep -E "openvswitch2.15|ovn-2021"
ovn-2021-host-21.12.0-42.el8fdp.x86_64
ovn-2021-central-21.12.0-42.el8fdp.x86_64
ovn-2021-21.12.0-42.el8fdp.x86_64

Comment 10 errata-xmlrpc 2022-04-25 14:26:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ovn-2021 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:1523