Created attachment 1939483 [details] OVN NB DB Created attachment 1939483 [details] OVN NB DB In the next setup of openstack, the reside-on-redirect-chassis and ovn-chassis-mac-mappings are not behaving as expected, and is wrongly using geneve tunnels for VLAN networks, leading to MTU issues We have a vlan provider network (provisioning) connected to a router (bm_router) that is connected (gateway) to a external flat provider network (nova) $ openstack network show provisioning +---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2023-01-19T17:57:32Z | | description | | | dns_domain | | | id | e3804743-5366-45a0-8326-59190fd93ee9 | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | location | cloud='', project.domain_id=, project.domain_name=, project.id='68766b5c07fc4221bfc538823bf4676a', project.name=, region_name='regionOne', zone= | | mtu | 1500 | | name | provisioning | | port_security_enabled | True | | project_id | 68766b5c07fc4221bfc538823bf4676a | | provider:network_type | vlan | | provider:physical_network | tenant | | provider:segmentation_id | 246 | | qos_policy_id | None | | revision_number | 3 | | router:external | Internal | | segments | None | | shared | True | | status | ACTIVE | | subnets | a7186fe6-f5c6-449e-9232-1f62c7ed8309 | | tags | | | updated_at | 2023-01-20T14:52:16Z | +---------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------+ $ openstack router show bm_router +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2023-01-19T17:57:45Z | | description | | | external_gateway_info | {"network_id": "cb84816a-3a02-41f4-b7ec-8a5b0638da5b", "external_fixed_ips": [{"subnet_id": "fbba8651-a043-488e-9273-2733611aefe1", "ip_address": "10.46.44.97"}], "enable_snat": true} | | flavor_id | None | | id | e9719af0-bfc0-4218-9787-fd9a553c530b | | interfaces_info | [{"port_id": "6b0624c2-f317-408e-ac3f-65cc250b3566", "ip_address": "172.27.7.1", "subnet_id": "a7186fe6-f5c6-449e-9232-1f62c7ed8309"}] | | location | cloud='', project.domain_id=, project.domain_name=, project.id='68766b5c07fc4221bfc538823bf4676a', project.name=, region_name='regionOne', zone= | | name | bm_router | | project_id | 68766b5c07fc4221bfc538823bf4676a | | revision_number | 4 | | routes | | | status | ACTIVE | | tags | | | updated_at | 2023-01-19T17:57:49Z | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ openstack network show nova +---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | nova | | created_at | 2023-01-19T17:31:24Z | | description | | | dns_domain | | | id | cb84816a-3a02-41f4-b7ec-8a5b0638da5b | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | location | cloud='', project.domain_id=, project.domain_name='Default', project.id='381cffb54c8e402b8b7f7515aae5375f', project.name='admin', region_name='regionOne', zone= | | mtu | 1500 | | name | nova | | port_security_enabled | True | | project_id | 381cffb54c8e402b8b7f7515aae5375f | | provider:network_type | flat | | provider:physical_network | datacentre | | provider:segmentation_id | None | | qos_policy_id | None | | revision_number | 2 | | router:external | External | | segments | None | | shared | False | | status | ACTIVE | | subnets | fbba8651-a043-488e-9273-2733611aefe1 | | tags | | | updated_at | 2023-01-19T17:31:30Z | +---------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Then we create a VM on the provisioning network, on compute-0: $ openstack server list --all --long -c Name -c Networks -c Host +-----------------------------+---------------------------+---------------------------+ | Name | Networks | Host | +-----------------------------+---------------------------+---------------------------+ | ostest-hlksb-master-0 | provisioning=172.27.7.193 | compute-0.redhat.local | +-----------------------------+---------------------------+---------------------------+ This VM has MTU problems as the traffic is tunneled to the gateway port on controller-1, via geneve (even though none of the networks are of geneve type), and despite reside-on-redirect-chassis-flag being set to False and ovn-chassis-mac-mappings being configured: $ sudo ovs-vsctl list open . | grep mac-mappings external_ids : {hostname=compute-0.redhat.local, ovn-bridge=br-int, ovn-bridge-mappings="datacentre:br-ex,tenant:br-isolated", ovn-chassis-mac-mappings="datacentre:fa:16:3e:08:1b:64,tenant:fa:16:3e:6f:c6:30", ovn-encap-ip="172.17.2.79", ovn-encap-type=geneve, ovn-match-northd-version="true", ovn-monitor-all="true", ovn-ofctrl-wait-before-clear="8000", ovn-openflow-probe-interval="60", ovn-remote="tcp:172.17.1.122:6642", ovn-remote-probe-interval="60000", rundir="/var/run/openvswitch", system-id="1b3edb7b-ca70-49bf-9c08-1de2790ef2e4"} The expectation based on documentation here [1] are that: - If reside-on-redirect-chassis is set to false (as in this case), the traffic is tunneled to the node with the gateway port. However, it is not expected to be tunneled with geneve since the network is a vlan network, right? - In addition, as ovn-chassis-mac-mappings is configured, expectation is that traffic is not tunneled to the gateway port but directly out In addition, I tested changing the value of reside-on-redirect-chassis flag to True. In that case the traffic is not tunneled anymore (thus no MTU issue), but, as expected, it is centralized through the gateway port node. Version-Release number of selected component (if applicable): Tested with ovn-2021-21.12.0-94 and ovn-2021-21.12.0-104 [1] https://github.com/ovn-org/ovn/blob/main/ovn-nb.xml#L2922
Hi, I had a look at the configuration and documentation, and based on what I'm seeing, it appears there's a bit of configuration missing that should make this work. Since you do not have options:reside-on-redirect-chassis set to true on the gateway port, you have to set options:redirect-type=bridged in order for traffic to be redirected over the localnet port instead of being tunneled. According to the NB DB uploaded here, that option is not set on the gateway router port. Also, since the southbound database has not been uploaded, we can only assume the ovn-chassis-mac-mappings have been set correctly. Please do the following: 1) Try setting options:redirect-type=bridged on the gateway router port and retry your scenario. If the traffic is sent over the localnet port, then all is well and we can close this issue. 2) If (1) doesn't work, then please upload your southbound database so that we can attempt to reproduce with the same ovn-chassis-mac-mappings configured. Thanks!
(In reply to Mark Michelson from comment #1) > Hi, I had a look at the configuration and documentation, and based on what > I'm seeing, it appears there's a bit of configuration missing that should > make this work. Since you do not have options:reside-on-redirect-chassis set > to true on the gateway port, you have to set options:redirect-type=bridged > in order for traffic to be redirected over the localnet port instead of > being tunneled. According to the NB DB uploaded here, that option is not set > on the gateway router port. > > Also, since the southbound database has not been uploaded, we can only > assume the ovn-chassis-mac-mappings have been set correctly. > > Please do the following: > > 1) Try setting options:redirect-type=bridged on the gateway router port and > retry your scenario. If the traffic is sent over the localnet port, then all > is well and we can close this issue. > 2) If (1) doesn't work, then please upload your southbound database so that > we can attempt to reproduce with the same ovn-chassis-mac-mappings > configured. > > Thanks! Either I'm setting it wrong or it still does not work. If I set the option redirect-type=bridged [1] it does nothing and the traffic is still going tunneled to the controller. I'm attaching the new DBs for the new setup [1] ovn-nbctl set logical_router_port 099c0427-d457-4a4d-b4fc-58c0a20db544 options:redirect-type=bridged
Hi, to me it still seems like misconfiguration. Looking through the SB DB you did set the redirect-type to bridged, but for the other LRP. The DGP is "lrp-9dfdf3ed-9ace-4baf-a611-934cd9f0ec44". This is also mentioned in documentation that it works only for DGP: "If set to bridged on a distributed gateway port, this option causes OVN to redirect packets to the gateway chassis over a localnet port instead of a tunnel. The relevant chassis must share a localnet port." Can you please try to set it on the DGP instead and check if there is still an issue? We have a test for this scenario that is working fine, but there might be something wrong nevertheless. Thanks, Ales
Sorry, my bad, I'll try to reprovision and check again by setting it on the lrp-XXX asociated to the OVN router gateway port (the one that has the cr-lrp associated to it)
Testing it on the different port does not seems to work. Ales pointing out it may be due to https://bugzilla.redhat.com/show_bug.cgi?id=2007120
And yet another update, the "options:redirect-type=bridged" flag works as expected, the issue was that neutron is not aware of it and it is removing it automatically. So moving this bug to neutron so that this flag is managed properly
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Release of components for Red Hat OpenStack Platform 17.1 (Wallaby)), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2023:4577