Description of problem: One of our customers faced network outage during FFU steps after upgrading and restarting OVN controllers: controllers were unable to process existing OVN DB contents and connectivity was lost. Lukas Bezdicka helped to address the problem from DF perspective. OVN issue was reported to find a root cause of this problem: https://issues.redhat.com/browse/FDP-692 . This bug is reported to react when OVN team provides a feedback. Version-Release number of selected component (if applicable): RHOSP 16.2/17.1 How reproducible: no clear reproducer, problem was faced during OVN upgrade after OVN controllers were updated and restarted, but old RHOSP 16.2 OVN controlplane was still running.
The issue is in the way TripleO generates pid file paths for the systemd service file and conmon - they are not the same which leads to situation conmon generates a pid file that is not seen by the systemd. Then the systemd keeps restaring ovn-controller. Since the restarts are periodic at some point ovn-controller is restarted while OVN databases are being upgraded and hence down. ovn-controller loses the data about services such as DNS and DHCP. Then workloads using it can't resolve hostnames or if the DHCP lease expires they lose connectivity completely.
*** Bug 2298388 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: RHOSP 17.1.4 (openstack-tripleo-heat-templates) security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:9978
*** Bug 2321885 has been marked as a duplicate of this bug. ***
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days