Bug 1766575 - [OVN] After minor update from OSP13z8 to a puddle with Port Groups & OVS 2.11, Port Groups migration didn't happen
Summary: [OVN] After minor update from OSP13z8 to a puddle with Port Groups & OVS 2.11...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: python-networking-ovn
Version: 13.0 (Queens)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Lucas Alvares Gomes
QA Contact: Roman Safronov
URL:
Whiteboard:
: 1786135 (view as bug list)
Depends On:
Blocks: 1753533
TreeView+ depends on / blocked
 
Reported: 2019-10-29 12:32 UTC by Daniel Alvarez Sanchez
Modified: 2020-03-20 13:27 UTC (History)
6 users (show)

Fixed In Version: python-networking-ovn-4.0.3-16.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-03-10 11:26:15 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:0770 0 None None None 2020-03-10 11:26:58 UTC

Description Daniel Alvarez Sanchez 2019-10-29 12:32:37 UTC
When migrating an OSP13z8 cloud to a new puddle with Port Groups & OVS/OVN 2.11, Neutron Security Groups are still modelled as Address Sets and not as Port Groups. The maintenance task for migrating them [0] should kick in but looks like there's an issue that prevents that from happening:


- In one controller I can see this in neutron server logs:

2019-10-29 12:21:46.504 38 DEBUG futurist.periodics [-] Submitting periodic callback 'networking_ovn.common.maintenance.DBInconsistenciesPeriodics.migrate_to_port_groups' _process_scheduled /usr/lib/python2.7/site-packages/futurist/periodics.py:639
2019-10-29 12:21:56.504 38 DEBUG futurist.periodics [-] Submitting periodic callback 'networking_ovn.common.maintenance.DBInconsistenciesPeriodics.migrate_to_port_groups' _process_scheduled /usr/lib/python2.7/site-packages/futurist/periodics.py:639
2019-10-29 12:22:06.505 38 DEBUG futurist.periodics [-] Submitting periodic callback 'networking_ovn.common.maintenance.DBInconsistenciesPeriodics.migrate_to_port_groups' _process_scheduled /usr/lib/python2.7/site-packages/futurist/periodics.py:639


- In the other two controllers, no traces are shown

- Address sets are still used:

()[root@controller-0 /]# ovn-nbctl list address_set | grep -c _uuid
22
()[root@controller-0 /]# ovn-nbctl list port_group
_uuid               : 92c43d25-f19e-4d10-99ea-1d3c2badfa40
acls                : [0b2cc90c-87ba-48db-8364-9539c2121cdc, c91705a7-2dfd-4fde-98ae-9f012fe57e28]
external_ids        : {}
name                : neutron_pg_drop
ports               : [fed45d09-f9fd-410a-b7ff-cc82564a16b2]


Dataplane on existing workloads is fine but further creation of ports will fail.

A workaround is restarting neutron server or running the db sync script manually on one of the nodes so that they get created.



[0] https://github.com/openstack/networking-ovn/blob/stable/queens/networking_ovn/common/maintenance.py#L243

Comment 3 Daniel Alvarez Sanchez 2019-10-29 15:10:59 UTC
Just to clarify, we determined that this happens when neutron server containers are updated before a new version of OVN DB servers are promoted as master nodes with the new schema.
The fix will be such as upon reconnection from neutron-server to a newer schema, the port groups migration will be attempted if needed.

Comment 6 Eduardo Olivares 2020-01-08 09:10:02 UTC
*** Bug 1786135 has been marked as a duplicate of this bug. ***

Comment 8 Roman Safronov 2020-02-26 11:05:19 UTC
Was verified on 13.0-RHEL-7/2020-02-24.2 with python-networking-ovn-4.0.4-2.el7ost.noarch
Verified that after updating from OSP13z8 to 13.0-RHEL-7/2020-02-24.2 (including overcloud nodes reboot) Neutron Security Groups are modelled using Port_Groups and not Address_Sets.
No Address_Sets exist after the migration but all existing security groups are working.
When creating a new security groups no Address_Sets are created, only corresponding Port_Groups and ACLs.
It is possible to use and update existing security groups and create new ones.
Tried to launch a new instances using old and new security groups.

There was an issue that security groups migration was not applied until overcloud nodes are rebooted. Opened a separate bz on this issue https://bugzilla.redhat.com/show_bug.cgi?id=1806623

Comment 11 errata-xmlrpc 2020-03-10 11:26:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0770


Note You need to log in before you can comment on or make changes to this bug.