Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1673023

Summary: ovs-vswitchd crash in a deployment with ovn - OSP14
Product: Red Hat OpenStack Reporter: Jon Uriarte <juriarte>
Component: openvswitchAssignee: Numan Siddique <nusiddiq>
Status: CLOSED EOL QA Contact: Roman Safronov <rsafrono>
Severity: urgent Docs Contact:
Priority: high    
Version: 14.0 (Rocky)CC: apevec, chrisw, dalvarez, jschluet, lmartins, nusiddiq, rheslop, rhos-maint, tredaelli
Target Milestone: ---Keywords: TestOnly, Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch2.10-2.10.0-44.el7fdn Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-08-26 20:59:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1699124, 1720215, 1720219    
Bug Blocks:    

Description Jon Uriarte 2019-02-06 13:34:25 UTC 
Same issue described in [1], but for OSP14.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1671347



#0  tun_metadata_to_geneve__ (flow=flow@entry=0x7ffc8183c5d8, b=b@entry=0x7ffc81820288, crit_opt=crit_opt@entry=0x7ffc8181ff67) at ../lib/tun-metadata.c:676
#1  0x00005624864a8f6b in tun_metadata_to_geneve_nlattr_flow (b=0x7ffc81820288, flow=0x7ffc8183c590) at ../lib/tun-metadata.c:706
#2  tun_metadata_to_geneve_nlattr (tun=tun@entry=0x7ffc8183c590, flow=flow@entry=0x7ffc8183c590, key=key@entry=0x0, b=b@entry=0x7ffc81820288) at ../lib/tun-metadata.c:810
#3  0x000056248642a4cc in tun_key_to_attr (a=a@entry=0x7ffc81820288, tun_key=tun_key@entry=0x7ffc8183c590, tun_flow_key=tun_flow_key@entry=0x7ffc8183c590, key_buf=key_buf@entry=0x0, tnl_type=<optimized out>, 
    tnl_type@entry=0x0) at ../lib/odp-util.c:2885
#4  0x0000562486435ce2 in odp_key_from_dp_packet (buf=buf@entry=0x7ffc81820288, packet=0x7ffc8183c480) at ../lib/odp-util.c:5908
#5  0x00005624864b8df0 in dpif_netlink_encode_execute (buf=0x7ffc81820288, d_exec=0x7ffc8183c0e8, dp_ifindex=<optimized out>) at ../lib/dpif-netlink.c:1742
#6  dpif_netlink_operate__ (dpif=dpif@entry=0x56248833ca50, ops=ops@entry=0x7ffc8183c0d8, n_ops=n_ops@entry=1) at ../lib/dpif-netlink.c:1828
#7  0x00005624864b95ce in dpif_netlink_operate_chunks (n_ops=1, ops=0x7ffc8183c0d8, dpif=<optimized out>) at ../lib/dpif-netlink.c:2120
#8  dpif_netlink_operate (dpif_=0x56248833ca50, ops=0x7ffc8183c0d8, n_ops=<optimized out>) at ../lib/dpif-netlink.c:2156
#9  0x00005624863fac53 in dpif_operate (dpif=0x56248833ca50, ops=ops@entry=0x7ffc8183c0d8, n_ops=n_ops@entry=1) at ../lib/dpif.c:1349
#10 0x00005624863fb438 in dpif_execute (dpif=<optimized out>, execute=execute@entry=0x7ffc8183c170) at ../lib/dpif.c:1314
#11 0x00005624863a8951 in nxt_resume (ofproto_=0x56248835fea0, pin=0x7ffc8183cbe0) at ../ofproto/ofproto-dpif.c:5085
#12 0x00005624863946f6 in handle_nxt_resume (ofconn=ofconn@entry=0x5624882fe690, oh=oh@entry=0x562488326ce0) at ../ofproto/ofproto.c:3619
#13 0x00005624863a08cb in handle_openflow__ (msg=0x5624883262c0, ofconn=0x5624882fe690) at ../ofproto/ofproto.c:8167
#14 handle_openflow (ofconn=0x5624882fe690, ofp_msg=0x5624883262c0) at ../ofproto/ofproto.c:8288
#15 0x00005624863d2583 in ofconn_run (handle_openflow=0x5624863a0600 <handle_openflow>, ofconn=0x5624882fe690) at ../ofproto/connmgr.c:1446
#16 connmgr_run (mgr=0x562488329bd0, handle_openflow=handle_openflow@entry=0x5624863a0600 <handle_openflow>) at ../ofproto/connmgr.c:365
#17 0x000056248639a4ce in ofproto_run (p=0x56248835fea0) at ../ofproto/ofproto.c:1825
#18 0x0000562486387bac in bridge_run__ () at ../vswitchd/bridge.c:2944
#19 0x000056248638da08 in bridge_run () at ../vswitchd/bridge.c:3002
#20 0x00005624861cce15 in main (argc=12, argv=0x7ffc8183e088) at ../vswitchd/ovs-vswitchd.c:125

Version-Release number of selected component (if applicable):
OSP 14 2019-01-17.2 puddle

Controller (RHEL 7.6):
[root@controller-0 tmp]# rpm -qa | grep openvswitch
openvswitch2.10-2.10.0-28.el7fdp.1.x86_64
rhosp-openvswitch-2.10-0.1.el7ost.noarch
openvswitch2.10-debuginfo-2.10.0-28.el7fdp.1.x86_64
openvswitch-selinux-extra-policy-1.0-9.el7fdp.noarch

[heat-admin@controller-0 ~]$ sudo docker images | grep ovn
192.168.24.1:8787/rhosp14/openstack-ovn-northd                2019-01-16.1        03bdcefcab6c        8 days ago          752 MB
192.168.24.1:8787/rhosp14/openstack-ovn-northd                pcmklatest          03bdcefcab6c        8 days ago          752 MB
192.168.24.1:8787/rhosp14/openstack-neutron-server-ovn        2019-01-16.1        29207e903690        8 days ago          920 MB
192.168.24.1:8787/rhosp14/openstack-nova-novncproxy           2019-01-16.1        9803d4995c22        2 weeks ago         880 MB
192.168.24.1:8787/rhosp14/openstack-ovn-controller            2019-01-16.1        7cde11f5e83e        2 weeks ago         626 MB


neutron_api container:
python-neutron-fwaas-13.0.1-0.20180910012802.3c6cfbc.el7ost.noarch
openstack-neutron-13.0.2-0.20180929022427.c7f970c.el7ost.noarch
openstack-neutron-common-13.0.2-0.20180929022427.c7f970c.el7ost.noarch
openstack-neutron-ml2-13.0.2-0.20180929022427.c7f970c.el7ost.noarch
puppet-neutron-13.3.1-0.20181013115834.el7ost.noarch
python-neutron-13.0.2-0.20180929022427.c7f970c.el7ost.noarch
openstack-neutron-lbaas-13.0.1-0.20180831185310.e0cca6e.el7ost.noarch
python2-neutronclient-6.9.1-0.20180925041810.7eba94e.el7ost.noarch
python-neutron-lbaas-13.0.1-0.20180831185310.e0cca6e.el7ost.noarch
openstack-neutron-fwaas-13.0.1-0.20180910012802.3c6cfbc.el7ost.noarch
python2-neutron-lib-1.18.0-0.20180816094046.67865c7.el7ost.noarch


ovn_controller container:
puppet-ovn-13.3.1-0.20181013120724.38e2e33.el7ost.noarch
openvswitch2.10-ovn-common-2.10.0-28.el7fdp.1.x86_64
rhosp-openvswitch-ovn-host-2.10-0.1.el7ost.noarch
rhosp-openvswitch-ovn-common-2.10-0.1.el7ost.noarch
openvswitch2.10-ovn-host-2.10.0-28.el7fdp.1.x86_64
Comment 1 Jon Uriarte 2019-02-11 10:14:54 UTC 
Verified with openvswitch2.10-2.10.0-44.el7fdn, the ovs-vswitchd process does not crash any more.

The networking issues with this rpm are due to the BZ [0] and they're expected as openvswitch2.10-2.10.0-44.el7fdn doesn't contain the fix.

[0] https://bugzilla.redhat.com/show_bug.cgi?id=1643902
Comment 2 Jon Uriarte 2019-02-11 10:34:17 UTC 
Sorry, openvswitch2.10-2.10.0-44.el7fdn contains the fix for [0] but the OVN images I was using didn't.


[0] https://bugzilla.redhat.com/show_bug.cgi?id=1643902