Description of problem: When we try to disable snat of an existing router, neutron accepts the request and enable_snat is updated but snat rule is not removed form ovn and snat is still available. Initially a router is created with snat_enabled: true. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack router show router +-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2021-04-15T06:54:51Z | | description | | | external_gateway_info | {"network_id": "08012876-fb02-4f3e-9000-40810c433c3c", "external_fixed_ips": [{"subnet_id": "6661e943-1789-439e-b957-65d93748fa8c", "ip_address": "10.0.0.234"}, {"subnet_id": "20ed52a9-1788-4ad8-8e4a-8d0d40e6eb73", "ip_address": "2620:52:0:13b8::1000:45"}], "enable_snat": true} | | flavor_id | None | | id | 6f7b40aa-c1ec-4e07-972d-a71af103db7d | | interfaces_info | [{"port_id": "3cf4d04e-dfca-4a1a-b72e-56d10d422bc7", "ip_address": "192.168.10.1", "subnet_id": "1e717b5b-68e9-416c-990e-6d34390474bb"}] | | location | cloud='', project.domain_id=, project.domain_name='Default', project.id='4c9a7610e1b043be9ba5fcb530a964ad', project.name='admin', region_name='regionOne', zone= | | name | router | | project_id | 4c9a7610e1b043be9ba5fcb530a964ad | | revision_number | 33 | | routes | | | status | ACTIVE | | tags | | | updated_at | 2021-05-19T08:50:05Z | +-------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ~~~ I created an instance under the private network connected to the router. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack server show testinstance001 +-------------------------------------+---------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------------------+---------------------------------------------------------------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0.redhat.local | | OS-EXT-SRV-ATTR:hostname | testinstance001 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0.redhat.local | | OS-EXT-SRV-ATTR:instance_name | instance-00000002 | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-9jk0h49t | | OS-EXT-SRV-ATTR:root_device_name | /dev/vda | | OS-EXT-SRV-ATTR:user_data | None | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2021-04-15T07:03:21.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | private=192.168.10.28 | | config_drive | | | created | 2021-04-15T07:03:08Z | | description | None | | flavor | disk='1', ephemeral='0', extra_specs.hw_rng:allowed='True', original_name='m1.nano', ram='128', swap='0', vcpus='1' | | hostId | cc254519f2f506ec715c4d6693567a2ad9bf221949fd34691b5dbd8a | | host_status | UP | | id | 91e4846c-0a8f-4ebb-8591-17ca7d6a874c | | image | cirros-0.4.0-x86_64-disk.img (ad38b060-abdc-4570-a46a-1c799fb46898) | | key_name | None | | locked | False | | locked_reason | None | | name | testinstance001 | | progress | 0 | | project_id | 4c9a7610e1b043be9ba5fcb530a964ad | | properties | | | security_groups | name='icmp' | | server_groups | [] | | status | ACTIVE | | tags | [] | | trusted_image_certificates | None | | updated | 2021-04-15T07:03:22Z | | user_id | 492083d2deef4aaaae5dbd0cc4e3df19 | | volumes_attached | | +-------------------------------------+---------------------------------------------------------------------------------------------------------------------+ ~~~ Then ping from the instance to external network succeeds as expected. ~~~ $ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=111 time=12.516 ms 64 bytes from 8.8.8.8: seq=1 ttl=111 time=9.532 ms 64 bytes from 8.8.8.8: seq=2 ttl=111 time=9.063 ms ~~~ Then I updated the router to disabled snat. Request has been accepted and the router now has enable_snat: false. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack router set router --disable-snat --external-gateway nova (overcloud) [stack@undercloud-0 ~]$ openstack router show router +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2021-04-15T06:54:51Z | | description | | | external_gateway_info | {"network_id": "08012876-fb02-4f3e-9000-40810c433c3c", "external_fixed_ips": [{"subnet_id": "6661e943-1789-439e-b957-65d93748fa8c", "ip_address": "10.0.0.234"}, {"subnet_id": "20ed52a9-1788-4ad8-8e4a-8d0d40e6eb73", "ip_address": "2620:52:0:13b8::1000:45"}], "enable_snat": false} | | flavor_id | None | | id | 6f7b40aa-c1ec-4e07-972d-a71af103db7d | | interfaces_info | [{"port_id": "3cf4d04e-dfca-4a1a-b72e-56d10d422bc7", "ip_address": "192.168.10.1", "subnet_id": "1e717b5b-68e9-416c-990e-6d34390474bb"}] | | location | cloud='', project.domain_id=, project.domain_name='Default', project.id='4c9a7610e1b043be9ba5fcb530a964ad', project.name='admin', region_name='regionOne', zone= | | name | router | | project_id | 4c9a7610e1b043be9ba5fcb530a964ad | | revision_number | 34 | | routes | | | status | ACTIVE | | tags | | | updated_at | 2021-05-19T08:52:50Z | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ~~~ However snat still exists in ovn. ~~~ [heat-admin@controller-0 ~]$ sudo podman exec -it $(sudo podman ps -q -f name=ovn-dbs) ovn-nbctl lr-nat-list 60c80c50-2fde-4688-bcf1-e958ffc0f3ed TYPE EXTERNAL_IP EXTERNAL_PORT LOGICAL_IP EXTERNAL_MAC LOGICAL_PORT snat 10.0.0.234 192.168.10.0/24 ~~~ Then the ping still succeeds. ~~~ $ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=111 time=11.821 ms 64 bytes from 8.8.8.8: seq=1 ttl=111 time=9.556 ms 64 bytes from 8.8.8.8: seq=2 ttl=111 time=8.736 ms 64 bytes from 8.8.8.8: seq=3 ttl=111 time=9.217 ms ~~~ If I remove the gateway and attach it again with snat disabled. ~~~ (overcloud) [stack@undercloud-0 ~]$ openstack router unset router --external-gateway (overcloud) [stack@undercloud-0 ~]$ openstack router set router --external-gateway nova --disable-snat (overcloud) [stack@undercloud-0 ~]$ openstack router show router +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2021-04-15T06:54:51Z | | description | | | external_gateway_info | {"network_id": "08012876-fb02-4f3e-9000-40810c433c3c", "external_fixed_ips": [{"subnet_id": "6661e943-1789-439e-b957-65d93748fa8c", "ip_address": "10.0.0.236"}, {"subnet_id": "20ed52a9-1788-4ad8-8e4a-8d0d40e6eb73", "ip_address": "2620:52:0:13b8::1000:17"}], "enable_snat": false} | | flavor_id | None | | id | 6f7b40aa-c1ec-4e07-972d-a71af103db7d | | interfaces_info | [{"port_id": "3cf4d04e-dfca-4a1a-b72e-56d10d422bc7", "ip_address": "192.168.10.1", "subnet_id": "1e717b5b-68e9-416c-990e-6d34390474bb"}] | | location | cloud='', project.domain_id=, project.domain_name='Default', project.id='4c9a7610e1b043be9ba5fcb530a964ad', project.name='admin', region_name='regionOne', zone= | | name | router | | project_id | 4c9a7610e1b043be9ba5fcb530a964ad | | revision_number | 39 | | routes | | | status | ACTIVE | | tags | | | updated_at | 2021-05-19T08:56:54Z | +-------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ~~~ This time snat is not registered in ovn ~~~ [heat-admin@controller-0 ~]$ sudo podman exec -it $(sudo podman ps -q -f name=ovn-dbs) ovn-nbctl lr-nat-list 60c80c50-2fde-4688-bcf1-e958ffc0f3ed [heat-admin@controller-0 ~]$ ~~~ Thus ping now fails ~~~ $ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes ^C --- 8.8.8.8 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss $ ~~~ Version-Release number of selected component (if applicable): RHOSP 16.1.5 How reproducible: Always Steps to Reproduce: 1. Create a router and attach an external gateway with snat enabled 2. Disable snat of the router 3. Try ping from an instance on the private network attached to the router Actual results: ping still succeeds even after the router get snat disabled Expected results: ping still fails after the router get snat disabled Additional info:
Fixed in RHOS-16.1-RHEL-8-20211007.n.1 [root@controller-1 ~]# podman exec -it neutron_api /bin/bash ()[neutron@controller-1 /]$ rpm -qa | grep ovn puppet-ovn-15.4.1-1.20210528102649.192ac4e.el8ost.noarch python3-networking-ovn-7.3.1-1.20210714143309.el8ost.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.7 (Train) bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:3762