Bug 2171896

Summary: ovn agents are dead after ovn-controller update.
Product: Red Hat OpenStack Reporter: Sofer Athlan-Guyot <sathlang>
Component: openstack-tripleo-heat-templatesAssignee: Sofer Athlan-Guyot <sathlang>
Status: CLOSED ERRATA QA Contact: Khomesh Thakre <kthakre>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 17.1 (Wallaby)CC: dhill, kthakre, mburns, mciecier, prgutier
Target Milestone: rcKeywords: Triaged
Target Release: 17.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-14.3.1-1.20230519151015.el9ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-16 01:13:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sofer Athlan-Guyot 2023-02-20 19:11:16 UTC 
Description of problem: Update of 17.1 from 17.0:

Hi,

after updating the ovn-controller the agent are deads:

| ID                                   | Agent Type                   | Host                      | Availability Zone | Alive | State | Binary                     |
+--------------------------------------+------------------------------+---------------------------+-------------------+-------+-------+----------------------------+
| bc58a243-c9e1-48bc-aa25-23bc6a18e956 | OVN Controller agent         | compute-0.redhat.local    |                   | XXX   | UP    | ovn-controller             |
| 32294417-48c4-507a-b7cd-5affaa62e009 | OVN Metadata agent           | compute-0.redhat.local    |                   | XXX   | UP    | neutron-ovn-metadata-agent |
| de3b2308-8cf1-4453-80a5-5bb7898c084f | OVN Controller Gateway agent | controller-0.redhat.local |                   | XXX   | UP    | ovn-controller             |
| af5eb2d5-d51e-40f7-9dab-04d31588c034 | OVN Controller Gateway agent | controller-1.redhat.local |                   | XXX   | UP    | ovn-controller             |
| 985726c7-09f8-45f7-bfe8-9e34cde5fe30 | OVN Controller Gateway agent | controller-2.redhat.local |                   | XXX   | UP    | ovn-controller             |


with:

/var/log/containers/stdouts/ovn_controller.log:2023-02-20T18:17:00.231170529+00:00 stderr F 2023-02-20T18:17:00Z|00157|main|WARN|controller version - 22.12.1-20.27.0-70.6 mismatch with northd version - 22.03.2-20.21.0-61.4
Comment 1 Sofer Athlan-Guyot 2023-02-20 19:45:48 UTC 
Note, that running:

    ansible -b -i overcloud-deploy/qe-Cloud-0/config-download/qe-Cloud-0/tripleo-ansible-inventory.yaml ovn_controller -mshell -a 'ovs-vsctl set open . external_ids:ovn-match-northd-version=false'

before update of the ovn-controller prevents the issue from happening.
Comment 19 errata-xmlrpc 2023-08-16 01:13:59 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2023:4577