Bug 2112111

Summary:	ovn-controller is not installing the group mod flows properly for load balancers
Product:	Red Hat Enterprise Linux Fast Datapath	Reporter:	Surya Seetharaman <surya>
Component:	ovn22.06	Assignee:	OVN Team <ovnteam>
Status:	CLOSED ERRATA	QA Contact:	ying xu <yinxu>
Severity:	high	Docs Contact:
Priority:	high
Version:	FDP 22.L	CC:	anusaxen, ctrautma, jiji, jishi, jniu, mmichels, nusiddiq, rkhan, vpickard, zzhao
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	ovn22.06-22.06.0-27.el8fdp	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-09-06 18:48:49 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2111733

Description Surya Seetharaman 2022-07-28 20:23:24 UTC

Description of problem:

We are seeing:
2022-07-28T16:17:25.928Z|03812|ofctrl|INFO|OpenFlow error: OFPT_ERROR (OF1.5) (xid=0x48e43): OFPGMFC_GROUP_EXISTS
OFPT_BUNDLE_ADD_MESSAGE (OF1.5) (xid=0x48e43):
 bundle_id=0x6ea1 flags=atomic ordered
OFPT_GROUP_MOD (OF1.5) (xid=0x48e43):
 ADD group_id=4,type=select,selection_method=dp_hash,bucket=bucket_id:0,weight:100,actions=ct(commit,table=20,zone=NXM_NX_REG13[0..15],nat(dst=10.130.11.209:5432),exec(set_field:0x2/0x2->ct_mark))

on ovn-controller logs and another group already exists with id=4:

sh-4.4# ovs-ofctl dump-groups br-int | grep group_id=4,
 group_id=4,type=select,selection_method=dp_hash,bucket=bucket_id:0,weight:100,actions=ct(commit,table=15,zone=NXM_NX_REG11[0..15],nat(dst=10.131.11.44:5432),exec(load:0x1->NXM_NX_CT_MARK[1]))



Additional info:

See must-gather from https://bugzilla.redhat.com/show_bug.cgi?id=2111619#c5 and https://bugzilla.redhat.com/show_bug.cgi?id=2111733#c1 for more such sample failures.

Comment 2 OVN Bot 2022-07-29 16:07:12 UTC

ovn22.06 fast-datapath-rhel-8 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2112416
ovn22.06 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2112417

Comment 5 OVN Bot 2022-08-09 04:07:03 UTC

ovn22.03 fast-datapath-rhel-8 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2116620
ovn22.03 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2116621

Comment 6 ying xu 2022-08-09 11:03:18 UTC

I talked with devel,this bug is hardly to reproduce.
I tried many times, and couldn't reproduce it.

I can just set it as sanity only.

I add 99 LBS on a s1, and then restart ovs, then add a new lb,everything is ok.

my script:
 ovs-vsctl set open . external_ids:ovn-ofctrl-wait-before-clear=5000
ovs-vsctl list open_vswitch .
sleep 5
for i in `seq 100 200`;do
        systemctl restart ovs-vswitchd
        sleep 1
        ovn-nbctl lb-add lb$i 2.2.2.$i 10.1.2.$i -- ls-lb-add s1 lb$i
        sleep 5
        count=`ovn-appctl -t ovn-controller group-table-list | awk '{print $2}' | sort | uniq | wc -l`
        if [ $count == $i ];then
                echo $count
        else
                echo $count
                break
        fi
        if [ `cat /var/log/openvswitch/ovs-vswitchd.log |grep error` ];then
                break;
        fi
        sleep 10
done

no errors found.


# rpm -qa|grep ovn
ovn22.06-host-22.06.0-27.el8fdp.x86_64
ovn22.06-22.06.0-27.el8fdp.x86_64
ovn22.06-central-22.06.0-27.el8fdp.x86_64

Comment 7 OVN Bot 2022-08-10 04:07:35 UTC

ovn-2021 fast-datapath-rhel-8 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2117124
ovn-2021 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2117125

Comment 8 OVN Bot 2022-08-17 18:41:43 UTC

ovn-2021 fast-datapath-rhel-8 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2119165
ovn-2021 fast-datapath-rhel-9 clone created at https://bugzilla.redhat.com/show_bug.cgi?id=2119166

Comment 10 errata-xmlrpc 2022-09-06 18:48:49 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ovn22.06 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:6362