Created attachment 1713581 [details] OVN NB DB from master-1 Description of problem: Seen on an IPv6 single-stack Openshift cluster: In ovn-controller we have a lot of logs complaining about: 2020-09-03T04:21:56Z|00058|lflow|WARN|error parsing actions "trigger_event(event = "empty_lb_backends", meter = "", vip = "fd02::3a5a:50051", protocol = "tcp", load_balancer = "52e4aa4a-456b-4ee6-81a2-90b67bcd8b55");": Invalid load balancer VIP 'fd02::3a5a:50051' This load-balancer VIP is however properly defined in the OVN NB DB, see: $ ovn-nbctl -p /ovn-cert/tls.key -c /ovn-cert/tls.crt -C /ovn-ca/ca-bundle.crt --db 'ssl:[2620:52:0:1386::91]:9641,ssl:[2620:52:0:1386::92]:9641,ssl:[2620:52:0:1386::93]:9641' list load_balancer | grep fd02::3a5a vips : {.... "[fd02::3a5a]:50051"="",...} Version-Release number of selected component (if applicable): rpm -qa | grep ovn ovn2.13-2.13.0-39.el7fdp.x86_64 ovn2.13-central-2.13.0-39.el7fdp.x86_64 ovn2.13-host-2.13.0-39.el7fdp.x86_64 ovn2.13-vtep-2.13.0-39.el7fdp.x86_64 How reproducible: - Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Assigning to Dumitru directly, since I already talked to him and he's aware of the context.
Created attachment 1713582 [details] OVN SB DB master-1
Created attachment 1713584 [details] ovn-controller logs from master-1
reproduced on ovn2.13.0-39 with following script: systemctl start openvswitch systemctl start ovn-northd ovn-nbctl set-connection ptcp:6641 ovn-sbctl set-connection ptcp:6642 ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:1.1.23.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=1.1.23.25 systemctl restart ovn-controller ip netns add server0 ip link add veth0_s0 netns server0 type veth peer name veth0_s0_p ip netns exec server0 ip link set lo up ip netns exec server0 ip link set veth0_s0 up ip netns exec server0 ip link set veth0_s0 address 00:00:00:01:01:02 ip netns exec server0 ip addr add 192.168.1.1/24 dev veth0_s0 ip netns exec server0 ip -6 addr add 2001::1/64 dev veth0_s0 ip netns exec server0 ip route add default via 192.168.1.254 dev veth0_s0 ip netns exec server0 ip -6 route add default via 2001::a dev veth0_s0 ovs-vsctl add-port br-int veth0_s0_p ip link set veth0_s0_p up ovs-vsctl set interface veth0_s0_p external_ids:iface-id=ls1p1 ovn-nbctl ls-add ls1 ovn-nbctl lsp-add ls1 ls1p1 #ovn-nbctl lsp-set-addresses ls1p1 "00:00:00:01:01:02 2001::1 192.168.1.1" ovn-nbctl lsp-set-addresses ls1p1 "00:00:00:01:01:02 192.168.1.1 2001::1" ovn-nbctl lsp-add ls1 ls1p2 ovn-nbctl lsp-set-addresses ls1p2 "00:00:00:01:02:02 192.168.1.2 2001::2" ovn-nbctl lr-add lr1 ovn-nbctl lrp-add lr1 lr1-ls1 00:00:00:00:00:01 192.168.1.254/24 2001::a/64 ovn-nbctl lsp-add ls1 ls1-lr1 ovn-nbctl lsp-set-addresses ls1-lr1 "00:00:00:00:00:01 192.168.1.254 2001::a" ovn-nbctl lsp-set-type ls1-lr1 router ovn-nbctl lsp-set-options ls1-lr1 router-port=lr1-ls1 ovn-nbctl lrp-add lr1 lr1-ls2 00:00:00:00:00:02 192.168.2.254/24 2002::a/64 ovn-nbctl ls-add ls2 ovn-nbctl lsp-add ls2 ls2-lr1 ovn-nbctl lsp-set-addresses ls2-lr1 "00:00:00:00:00:02 192.168.2.254 2002::a" ovn-nbctl lsp-set-type ls2-lr1 router ovn-nbctl lsp-set-options ls2-lr1 router-port=lr1-ls2 ovn-nbctl lsp-add ls2 ls2p1 ovn-nbctl lsp-set-addresses ls2p1 "00:00:00:02:01:02 192.168.2.1 2002::1" ovn-nbctl lsp-add ls1 ls1p3 ovn-nbctl lsp-set-addresses ls1p3 "00:00:00:01:03:02 192.168.1.3 2001::3" ip netns add server2 ip link add veth0_s2 netns server2 type veth peer name veth0_s2_p ip netns exec server2 ip link set lo up ip netns exec server2 ip link set veth0_s2 up ip netns exec server2 ip link set veth0_s2 address 00:00:00:01:03:02 ip netns exec server2 ip addr add 192.168.1.3/24 dev veth0_s2 ip netns exec server2 ip -6 addr add 2001::3/64 dev veth0_s2 ip netns exec server2 ip route add default via 192.168.1.254 dev veth0_s2 ip netns exec server2 ip -6 route add default via 2001::a dev veth0_s2 ovs-vsctl add-port br-int veth0_s2_p ip link set veth0_s2_p up ovs-vsctl set interface veth0_s2_p external_ids:iface-id=ls1p3 ovn-nbctl lb-add lb0 192.168.1.100 192.168.1.1,192.168.1.2 ovn-nbctl lb-add lb0 [2001::100]:50051 [2001::1]:50051,[2001::2]:50051 ovn-nbctl ls-lb-add ls2 lb0 ovn-nbctl set nb_global . options:controller_event=true ovn-nbctl create load_balancer name="foo" protocol="tcp" vips='"[2001::101]:50052"=""' ovn-nbctl ls-lb-add ls2 foo [root@wsfd-advnetlab18 bz1875337]# rpm -qa | grep -E "openvswitch|ovn" openvswitch-selinux-extra-policy-1.0-23.el8fdp.noarch ovn2.13-2.13.0-39.el8fdp.x86_64 openvswitch2.13-2.13.0-61.el8fdp.x86_64 ovn2.13-central-2.13.0-39.el8fdp.x86_64 ovn2.13-host-2.13.0-39.el8fdp.x86_64 + ovn-nbctl ls-lb-add ls2 lb0 + ovn-nbctl set nb_global . options:controller_event=true + ovn-nbctl create load_balancer name=foo protocol=tcp 'vips="[2001::101]:50052"=""' 240e552e-1f34-478a-9b22-46347f12c6ad + ovn-nbctl ls-lb-add ls2 foo [root@wsfd-advnetlab18 bz1875337]# tail -5 /var/log/ovn/ovn-controller.log 2020-10-14T07:55:55.638Z|00015|binding|INFO|ls1p1: Claiming 00:00:00:01:01:02 192.168.1.1 2001::1 2020-10-14T07:55:55.638Z|00016|binding|INFO|Claiming lport ls1p3 for this chassis. 2020-10-14T07:55:55.638Z|00017|binding|INFO|ls1p3: Claiming 00:00:00:01:03:02 192.168.1.3 2001::3 2020-10-14T07:55:55.653Z|00003|rconn(ovn_pinctrl0)|INFO|unix:/run/openvswitch/br-int.mgmt: connected 2020-10-14T07:55:55.658Z|00018|lflow|WARN|error parsing actions "trigger_event(event = "empty_lb_backends", meter = "", vip = "2001::101:50052", protocol = "tcp", load_balancer = "240e552e-1f34-478a-9b22-46347f12c6ad");": Invalid load balancer VIP '2001::101:50052' <=== error in ovn-controller.log Verified on ovn20.09.0-2: [root@wsfd-advnetlab18 bz1875337]# tail -5 /var/log/ovn/ovn-controller.log 2020-10-14T07:57:18.900Z|00014|binding|INFO|Claiming lport ls1p3 for this chassis. 2020-10-14T07:57:18.900Z|00015|binding|INFO|ls1p3: Claiming 00:00:00:01:03:02 192.168.1.3 2001::3 2020-10-14T07:57:18.901Z|00016|binding|INFO|Claiming lport ls1p1 for this chassis. 2020-10-14T07:57:18.901Z|00017|binding|INFO|ls1p1: Claiming 00:00:00:01:01:02 192.168.1.1 2001::1 2020-10-14T07:57:18.915Z|00003|rconn(ovn_pinctrl0)|INFO|unix:/run/openvswitch/br-int.mgmt: connected [root@wsfd-advnetlab18 bz1875337]# rpm -qa | grep -E "openvswitch|ovn" openvswitch-selinux-extra-policy-1.0-23.el8fdp.noarch ovn2.13-central-20.09.0-2.el8fdp.x86_64 openvswitch2.13-2.13.0-61.el8fdp.x86_64 ovn2.13-20.09.0-2.el8fdp.x86_64 ovn2.13-host-20.09.0-2.el8fdp.x86_64
also verified on rhel7 version: [root@wsfd-advnetlab16 bz1875337]# tail -5 /var/log/ovn/ovn-controller.log 2020-10-14T07:59:17.982Z|00014|binding|INFO|Claiming lport ls1p1 for this chassis. 2020-10-14T07:59:17.982Z|00015|binding|INFO|ls1p1: Claiming 00:00:00:01:01:02 192.168.1.1 2001::1 2020-10-14T07:59:17.982Z|00016|binding|INFO|Claiming lport ls1p3 for this chassis. 2020-10-14T07:59:17.982Z|00017|binding|INFO|ls1p3: Claiming 00:00:00:01:03:02 192.168.1.3 2001::3 2020-10-14T07:59:17.996Z|00003|rconn(ovn_pinctrl0)|INFO|unix:/run/openvswitch/br-int.mgmt: connected <=== no error [root@wsfd-advnetlab16 bz1875337]# ovn-nbctl list load_balancer _uuid : b53934e1-06cc-4331-a2fa-132dcb177fcf external_ids : {} health_check : [] ip_port_mappings : {} name : foo protocol : tcp selection_fields : [] vips : {"[2001::101]:50052"=""} _uuid : c34a02ab-4fea-4f23-9739-65680a06c889 external_ids : {} health_check : [] ip_port_mappings : {} name : lb0 protocol : tcp selection_fields : [] vips : {"192.168.1.100"="192.168.1.1,192.168.1.2", "[2001::100]:50051"="[2001::1]:50051,[2001::2]:50051"} [root@wsfd-advnetlab16 bz1875337]# rpm -qa | grep -E "openvswitch|ovn" kernel-kernel-networking-openvswitch-ovn-common-1.0-11.noarch python3-openvswitch2.13-2.13.0-51.el7fdp.x86_64 kernel-kernel-networking-openvswitch-ovn-acl-1.0-19.noarch openvswitch2.13-2.13.0-51.el7fdp.x86_64 ovn2.13-20.09.0-2.el7fdp.x86_64 ovn2.13-host-20.09.0-2.el7fdp.x86_64 openvswitch-selinux-extra-policy-1.0-15.el7fdp.noarch ovn2.13-central-20.09.0-2.el7fdp.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (ovn2.13 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:4356