Description of problem: Unfortunately the backport of BZ#1957786 made the northd and controller incompatible between version ovn-2021-21.12.0-66 (northd) and ovn-2021-21.12.0-59 (ovn-controller), the other way around is fine. This is causing issues on RHV environments as some hosts might have older version of ovn-controller than the northd.
Fix posted for review upstream: https://patchwork.ozlabs.org/project/ovn/list/?series=302899&state=*
A follow-up is required, v2 posted upstream: https://patchwork.ozlabs.org/project/ovn/list/?series=303236&state=* Moving back to POST.
Hi Dumitru, We ran RHV with ovn-2021-21.12.0-73.el8fdp.x86_64 and it looks ok. Is this the latest minor version? Is there a reason the bug is still in POST - are there more changes in the pipline? Thanks a lot, Eitan RHV\networking
(In reply to eraviv from comment #10) > Hi Dumitru, > Hi Eitan, > We ran RHV with ovn-2021-21.12.0-73.el8fdp.x86_64 and it looks ok. > Is this the latest minor version? Yes and all the required patches are in it. > Is there a reason the bug is still in POST - are there more changes in the > pipline? I'm moving it to MODIFIED manually now. I'm not sure why our automation didn't. > > Thanks a lot, > Eitan > RHV\networking Thanks, Dumitru
Actually, moving to ON_QA, the BZ is already part of the errata.
Initially installed following packages: [root@wsfd-2091565 ~]# rpm -qa |grep ovn ovn-2021-21.12.0-59.el8fdp.x86_64 ovn-2021-host-21.12.0-59.el8fdp.x86_64 ovn-2021-central-21.12.0-59.el8fdp.x86_64 Here is the script for the reproducer: systemctl start ovn-northd ovn-nbctl set-connection ptcp:6641 ovn-sbctl set-connection ptcp:6642 systemctl start openvswitch ovs-vsctl set open . external_ids:system-id=hv1 ovs-vsctl set open . external_ids:ovn-remote=tcp:192.168.1.1:6642 ovs-vsctl set open . external_ids:ovn-encap-type=geneve ovs-vsctl set open . external_ids:ovn-encap-ip=192.168.1.1 #ovs-vsctl set open . external_ids:ovn-monitor-all=true systemctl start ovn-controller ovn-nbctl lr-add R1 ovn-nbctl ls-add sw0 ovn-nbctl lsp-add sw0 sw02 -- lsp-set-addresses sw02 'f0:00:00:01:02:02 192.168.1.2 2001::2' ovn-nbctl ls-add sw1 ovn-nbctl ls-add public ovn-nbctl lrp-add R1 rp-sw0 00:00:01:01:02:03 192.168.1.254/24 2001::254/64 ovn-nbctl lrp-add R1 rp-sw1 00:00:03:01:02:03 192.168.2.254/24 2002::254/64 ovn-nbctl lrp-add R1 rp-public 00:00:02:01:02:03 172.16.1.254/24 1000::254/64 -- lrp-set-gateway-chassis rp-public hv1 ovs-vsctl add-br br-ext ovn-nbctl lsp-add sw0 sw0-rp -- set Logical_Switch_Port sw0-rp type=router options:router-port=rp-sw0 -- lsp-set-addresses sw0-rp router ovn-nbctl lsp-add sw1 sw1-rp -- set Logical_Switch_Port sw1-rp type=router options:router-port=rp-sw1 -- lsp-set-addresses sw1-rp router ovn-nbctl lsp-add public public-rp -- set Logical_Switch_Port public-rp type=router options:router-port=rp-public -- lsp-set-addresses public-rp router ovs-vsctl add-port br-int sw01 -- set interface sw01 type=internal external_ids:iface-id=sw01 ip netns add sw01 ip link set sw01 netns sw01 ip netns exec sw01 ip link set sw01 address f0:00:00:01:02:03 ip netns exec sw01 ip link set sw01 up ip netns exec sw01 ip addr add 192.168.1.1/24 dev sw01 ip netns exec sw01 ip route add default via 192.168.1.254 dev sw01 ip netns exec sw01 ip addr add 2001::1/64 dev sw01 ip netns exec sw01 ip -6 route add default via 2001::254 dev sw01 ovn-nbctl lsp-add sw0 sw01 -- lsp-set-addresses sw01 "f0:00:00:01:02:03 192.168.1.1 2001::1" ovs-vsctl add-port br-int sw11 -- set interface sw11 type=internal external_ids:iface-id=sw11 ip netns add sw11 ip link set sw11 netns sw11 ip netns exec sw11 ip link set sw11 address f0:00:00:02:02:03 ip netns exec sw11 ip link set sw11 up ip netns exec sw11 ip addr add 192.168.2.1/24 dev sw11 ip netns exec sw11 ip route add default via 192.168.2.254 dev sw11 ip netns exec sw11 ip addr add 2002::1/64 dev sw11 ip netns exec sw11 ip -6 route add default via 2002::254 dev sw11 ovn-nbctl lsp-add sw1 sw11 -- lsp-set-addresses sw11 "f0:00:00:02:02:03 192.168.2.1 2002::1" ovs-vsctl add-port br-int sw12 -- set interface sw12 type=internal external_ids:iface-id=sw12 ip netns add sw12 ip link set sw12 netns sw12 ip netns exec sw12 ip link set sw12 address f0:00:00:02:02:02 ip netns exec sw12 ip link set sw12 up ip netns exec sw12 ip addr add 192.168.2.2/24 dev sw12 ip netns exec sw12 ip route add default via 192.168.2.254 dev sw12 ip netns exec sw12 ip addr add 2002::2/64 dev sw12 ip netns exec sw12 ip -6 route add default via 2002::254 dev sw12 ovn-nbctl lsp-add sw1 sw12 -- lsp-set-addresses sw12 "f0:00:00:02:02:02 192.168.2.2 2002::2" ovs-vsctl add-port br-ext server -- set interface server type=internal ip netns add server ip netns exec server ip link set lo up ip link set server netns server ip netns exec server ip link set server up ip netns exec server ip addr add 172.16.1.50/24 dev server ip netns exec server ip route add default via 172.16.1.1 dev server ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=phynet:br-ext ovn-nbctl lsp-add public public1 -- lsp-set-addresses public1 unknown -- lsp-set-type public1 localnet -- lsp-set-options public1 network_name=phynet ovs-vsctl add-port br-ext ens1f0 ovn-nbctl lb-add lb0 30.0.0.1 192.168.2.1,192.168.2.2 ovn-nbctl ls-lb-add sw0 lb0 ovn-nbctl --wait=hv sync ip netns exec sw01 ping 30.0.0.1 -c 3 ### Ping successful <========================== Now upgraded ovn-central (ovn-controller): [root@wsfd-2091565 ~]# rpm -qa |grep ovn ovn-2021-21.12.0-59.el8fdp.x86_64 ovn-2021-host-21.12.0-59.el8fdp.x86_64 ovn-2021-central-21.12.0-66.el8fdp.x86_64 This time ovn-controller complained: [root@wsfd-advnetlab155 ~]# cat /var/log/ovn/ovn-controller.log |grep mark 2022-06-15T11:00:44.602Z|00032|lflow|WARN|error parsing match "ct.est && ct_mark.blocked == 0": Syntax error at `ct_mark.blocked' expecting field name. 2022-06-15T11:00:44.603Z|00033|lflow|WARN|error parsing actions "xxreg1 = ip6.dst; reg2[0..15] = tcp.dst; ct_lb_mark;": Syntax error at `ct_lb_mark' expecting action. ip netns exec sw01 ping 30.0.0.1 -c 3 ### Ping failed <================================= Now upgraded ovn-central (ovn-controller) again: [root@wsfd-2091565 ~]# rpm -qa |grep ovn ovn-2021-21.12.0-59.el8fdp.x86_64 ovn-2021-host-21.12.0-59.el8fdp.x86_64 ovn-2021-central-21.12.0-73.el8fdp.x86_64 ovn-controller resumed working as normal ip netns exec sw01 ping 30.0.0.1 -c 3 ### Ping successful <=================================
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (ovn bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2022:5446