Bug 2103545

Summary:	[OSP17] Unable to create a VM after migration to OVN, timeout on network-vif-plugged
Product:	Red Hat OpenStack	Reporter:	Roman Safronov <rsafrono>
Component:	openstack-neutron	Assignee:	Arnau Verdaguer <averdagu>
Status:	CLOSED ERRATA	QA Contact:	Roman Safronov <rsafrono>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	17.0 (Wallaby)	CC:	chrisw, egarciar, jlibosva, jschluet, scohen, skaplons, spower
Target Milestone:	---	Keywords:	AutomationBlocker, Regression, Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-neutron-18.4.1-0.20220705190433.5258354.el9ost openstack-tripleo-heat-templates-14.3.1-0.20220701162329.dd13d73.el9ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-09-21 12:23:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2075038

Description Roman Safronov 2022-07-04 04:41:25 UTC

Description of problem:
After OVN migration it's not possible to create a VM. VM remains in BUILD state for 300 seconds and then moves to ERROR state. In nova logs can be seen that it was not able to plug a VIF.

While OVN migration script completed successfully, br-migration bridge still present on nodes.

Version-Release number of selected component (if applicable):
RHOS-17.0-RHEL-9-20220701.n.1    Note, with previous OSP17 puddles the issue did not happen.
openstack-neutron-ovn-migration-tool-18.4.1-0.20220629010510.949942c.el9ost.noarch
openstack-neutron-18.4.1-0.20220629010510.949942c.el9ost.noarch
ovn22.03-22.03.0-52.el9fdp.x86_64


How reproducible:
100%
Happens in all ovs2ovn downstream jobs

Steps to Reproduce:
1. Deploy HA environment (3 controller, 3 compute nodes) with ML2/OVS backend
2. Create a workload: network, subnet, router, VM, connect internal network to external via the router. Spawn a VM connected to the internal network. 
3. Migrate to OVN according to the official documentation (link to 16.2 docs https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.2/html/migrating_the_networking_service_to_the_ml2ovn_mechanism_driver/migrating-ml2ovs-to-ovn)
4. Create a new VM

Actual results:
VM status is ERROR, timeout on network-vif-plugged

Expected results:
VM status is ACTIVE, VM is accessible

Additional info:

Comment 1 Arnau Verdaguer 2022-07-05 14:21:11 UTC

The problem recalls on the second time that tripleo update is called.

This second time is needed to set br-int as the bridge used on the cloud (instead of the br-migration that is currently using, after the first tripleo update).
The problem is that something is wrong with the task: "Generate ovn-extras environment file" which "fails" with this error:
Did not run command since '/home/stack/ovn-extras.yaml' exists. (It doesn't fail actually, since the return value is 0). 

The problem is that if the ovn-extras.yaml is not recreated it will use the br-migration (instead of the br-int).

Comment 2 Jakub Libosvar 2022-07-05 14:27:58 UTC

Looks like this patch https://review.opendev.org/c/openstack/neutron/+/848000 didn't get to the promotion yet.

Comment 8 Roman Safronov 2022-07-14 19:14:49 UTC

Was verified on RHOS-17.0-RHEL-9-20220711.n.1 with openstack-neutron-ovn-migration-tool-18.4.1-0.20220705190433.5258354.el9ost.noarch and  openstack-tripleo-heat-templates-14.3.1-0.20220706080800.feca772.el9ost.noarch
The issue did not happen during any of downstream CI ovs2ovn jobs.

Comment 14 errata-xmlrpc 2022-09-21 12:23:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Release of components for Red Hat OpenStack Platform 17.0 (Wallaby)), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:6543