Bug 2008425

Summary:	ovn migration create-resources is not stable
Product:	Red Hat OpenStack	Reporter:	Eduardo Olivares <eolivare>
Component:	python-networking-ovn	Assignee:	Eduardo Olivares <eolivare>
Status:	CLOSED ERRATA	QA Contact:	Roman Safronov <rsafrono>
Severity:	medium	Docs Contact:
Priority:	high
Version:	16.1 (Train)	CC:	apevec, ekuris, lhh, majopela, scohen
Target Milestone:	z8	Keywords:	Triaged
Target Release:	16.1 (Train on RHEL 8.2)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	python-networking-ovn-7.3.1-1.20210930103309.4e24f4c.el8ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	2008440 (view as bug list)		Environment:
Last Closed:	2022-03-24 11:01:35 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	2008440

Description Eduardo Olivares 2021-09-28 08:36:05 UTC

Description of problem:
As part of the ovs2ovn migration process, some workload is created before the migration begins. The intention is to verify that those workloads are healthy after the migration.
The script that creates those workloads checks they are pingable and ssh'able. Sometimes the ssh fails because the created VMs did not get the public keys during the cloud-init phase [1]:
+ ssh -i /home/stack/ovn_migration/pre_migration_resources/ovn_migration_ssh_key -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null cirros.0.243 date
Warning: Permanently added '10.0.0.243' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
cirros.0.243: Permission denied (publickey,password).
+ rc=255
+ echo 'Done with the resource creation : exiting with 255'
Done with the resource creation : exiting with 255
+ exit 255

It seems the issue is related to a problem in cirros 0.4.0, which was resolved in a later release of cirros [2].
So, using cirros 0.5.2 could stabilize this script.

This script can be found in two different formats:
https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration/tripleo_environment/playbooks/roles/resources/create/templates/create-resources.sh.j2
https://opendev.org/openstack/neutron/src/branch/master/tools/ovn_migration/infrared/tripleo-ovn-migration/roles/create-resources/templates/create-resources.sh.j2

[1] http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-network-networking-ovn-16.1_director-rhel-virthost-3cont_2comp_3net-ipv4-vxlan-ml2ovs-to-ovn-migration_composable/26/undercloud-0/home/stack/ovn_migration/pre_migration_resources/create-migration-resources.sh.log.gz
[2] https://github.com/cirros-dev/cirros/pull/11/commits/e40bcd2964aa496a9d03e1aaf95cf7a86938f129

Version-Release number of selected component (if applicable):

How reproducible:
Less than 10%

Steps to Reproduce:
1. run an ovs2ovn migration job
2.
3.

Comment 5 Roman Safronov 2022-02-07 13:38:57 UTC

Verified on RHOS-16.1-RHEL-8-20220203.n.1 with python3-networking-ovn-migration-tool-7.3.1-1.20220113183502.el8ost.noarch

Verified that the changes present in the rpm and there are no problems with accessing the workload VMs via ssh.

Comment 11 errata-xmlrpc 2022-03-24 11:01:35 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.8 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0986