Description of problem: After executing the OVS>OVN migration process, tenant networks that existed pre-migration (as VXLAN network types) do not get their neutron provider:network_type updated to geneve from vxlan. This prevents any further manipulation/change to the tenant network settings. Version-Release number of selected component (if applicable): OSP14 How reproducible: Easily Steps to Reproduce: 1. Deploy OSP14 overcloud with OVS, create vxlan tenant networks 2. Run OVS>OVN migration process 3. View and try to modify tenant networks that were originally vxlan and were converted to geneve Actual results: This network is actually geneve, but still shows vxlan in output: (overcloud) [stack@joea-director14 ~]$ openstack network show tenant +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2019-02-06T15:21:41Z | | description | | | dns_domain | | | id | d87ec876-85ed-43cb-8535-d0c115e423bc | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | mtu | 1442 | | name | tenant | | port_security_enabled | True | | project_id | 093a314be2d24707a21fcb75a4bc65a6 | | provider:network_type | vxlan | | provider:physical_network | None | | provider:segmentation_id | 4 | | qos_policy_id | None | | revision_number | 7 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | c2909ab7-4617-4f44-845d-0d58bbea8c1a | | tags | adapted_mtu, test | | updated_at | 2019-02-07T16:50:42Z | +---------------------------+--------------------------------------+ (overcloud) [stack@joea-director14 ~]$ openstack network set --disable-port-security tenant BadRequestException: 400: Client Error for url: http://10.9.66.103:9696/v2.0/networks/d87ec876-85ed-43cb-8535-d0c115e423bc, {"NeutronError": {"message": "Invalid input for operation: Network type vxlan is not supported.", "type": "InvalidInput", "detail": ""}} Expected results: Neutron fields reporting network_type should be geneve, and we should be able to modify network configuration as per usual Additional info: OSP14 Hackfest
Creating new tenant networks after migration results in the correct network_type value and this network can be manipulated without issue: (overcloud) [stack@joea-director14 ~]$ openstack network show tenant2 +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2019-02-07T17:16:19Z | | description | | | dns_domain | | | id | df1f839c-958e-4984-8bbf-52cdfcfde6be | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | None | | is_vlan_transparent | None | | mtu | 1442 | | name | tenant2 | | port_security_enabled | True | | project_id | 093a314be2d24707a21fcb75a4bc65a6 | | provider:network_type | geneve | | provider:physical_network | None | | provider:segmentation_id | 32 | | qos_policy_id | None | | revision_number | 3 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | 79957257-6aaa-4814-b36d-97b4098697f5 | | tags | | | updated_at | 2019-02-07T17:16:20Z | +---------------------------+--------------------------------------+ (overcloud) [stack@joea-director14 ~]$ openstack network set --disable-port-security tenant2 (overcloud) [stack@joea-director14 ~]$
Good catch. We are updating the MTU but not the type of the existing networks. This is tricky as we're failing the precommit action of the network update due to an unsupported type [0]. I can think of a possible solution which would be: 1. After migration, add 'vxlan' as a temporary valid value for tenant network types [1]. Restart neutron server containers. 2. Update all existing networks to change the type to 'geneve' 3. Revert changes made in step 1. [0] https://github.com/openstack/networking-ovn/blob/fe8b9c75e7b8ede8a6e8c774fdbdd9448b0ef232/networking_ovn/ml2/mech_driver.py#L280 [1] https://github.com/openstack/networking-ovn/blob/85c227e1fff98fcbed3da057a1e02efbe176440f/migration/migrate-to-ovn.yml#L108
This Release is retired. If this bug is still relevant, please reopen and retarget to an open release.
Now ML2/OVN supports VXLAN overlay so we need to account for the case when we migrate to ML2/OVN but we want to keep VXLAN (eg. to not change MTU)
One customer now observes the same issue after they have completed migration from ovs to ovn in their RHOSP16.1 deployment which was upgraded from RHOSP13. I confirmed that their existing networks still have network_type: vxlan instead of geneve, and they can't update these vxlan networks because of the same error. I don't have any handy deployment to test this now but I don't think that neutron supports updating network_type of existing networks. There is a validation logic implemented in ml2 plugin layer which blocks updating provider network attribute except for segmentation id. https://github.com/openstack/neutron/blob/stable/train/neutron/plugins/ml2/plugin.py#L866-L884 So I'm afraid that only option is to allow vxlan for tenant networks. @Daniel Could you check above points ? Also, I'd like to ask you to confirm that using vxlan with ml2-ovn is currently supported in RHOSP16.1. As per upstream change to enable vxlan support, I see vxlan is supported with ovn>20.09 and we do have the version meeting that requirements. Also I noticed that we do enable vxlan during ovs->ovn migration. So I believe usage of vxlan is supported, but I could not find any clear documentation which explains it. https://review.opendev.org/c/openstack/tripleo-heat-templates/+/753453
@ Jakub Could you help me get some feedback here ? I've checked the networking-ovn package in RHOSP, but I'm afraid that it doesn't include the following change which added support for vxlan network to ovn drivers. So I'm afraid that it is not yet supported to use vxlan in ml2+ovn deployment... https://review.opendev.org/c/openstack/neutron/+/734889
(In reply to Takashi Kajinami from comment #8) > @ Jakub > > Could you help me get some feedback here ? > > I've checked the networking-ovn package in RHOSP, but I'm afraid that it > doesn't include the following change > which added support for vxlan network to ovn drivers. So I'm afraid that it > is not yet supported to use vxlan > in ml2+ovn deployment... > > https://review.opendev.org/c/openstack/neutron/+/734889 That's correct. The problem here is we should change from vxlan to Geneve during the migration. I just sent patches and will build an RPM soon.
Thanks Jakub for submitting the fix. If I understand the proposed fix correctly, the change makes migration process trigger sql command to update network_type column in the networksegments table and replaces 'vxlan' with 'geneve'. Can we apply the same approach to the delployment where ovn migration has been completed ? Side note: If that approach can be applied in such deployment, I'm thinking of stopping all neutron processes on controller nodes to avoid any possible conflict and then run sql query to apply the same change. Maybe it would be more safe to apply the change to only one network first then apply the same to the remaining networks if we don't see any problems.
Hi. May I ask any feedback about my previous comment ?
Verified on RHOS-16.1-RHEL-8-20210727.n.1 with python3-networking-ovn-7.3.1-1.20210714143305.4e24f4c.el8ost.noarch Created several vxlan networks before migration to OVN, migrated to OVN, then created several new networks and verified that after migration all geneve networks (old and newly created) have unique segmentation ids. Additionally verified that there are no issues when modifying a network that was created before the migration to OVN.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.7 (Train) bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:3762