Description of problem: When we create a vip as a allowed-address-pair and attach a floating ip to it; the vip address is unreachable over it's floating ip Here are the details from my lab: The vxlan network on which the instance is created: +++ (overcloud) [stack@undercloud16 ~]$ neutron net-show net1 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | availability_zone_hints | | | availability_zones | nova | | created_at | 2021-04-22T04:12:48Z | | description | | | id | 1c6adde3-3959-4ae5-be0c-36cb71a41e1f | | ipv4_address_scope | | | ipv6_address_scope | | | l2_adjacency | True | | mtu | 1450 | | name | net1 | | port_security_enabled | True | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | provider:network_type | vxlan | | provider:physical_network | | | provider:segmentation_id | 1 | | qos_policy_id | | | revision_number | 2 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | 12b24308-c016-4617-a13d-c590586d382b | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-22T04:13:08Z | +---------------------------+--------------------------------------+ (overcloud) [stack@undercloud16 ~]$ neutron subnet-show 12b24308-c016-4617-a13d-c590586d382b neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +-------------------+------------------------------------------------+ | Field | Value | +-------------------+------------------------------------------------+ | allocation_pools | {"start": "172.16.2.2", "end": "172.16.2.254"} | | cidr | 172.16.2.0/24 | | created_at | 2021-04-22T04:13:08Z | | description | | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 172.16.2.1 | | host_routes | | | id | 12b24308-c016-4617-a13d-c590586d382b | | ip_version | 4 | | ipv6_address_mode | | | ipv6_ra_mode | | | name | subnet1 | | network_id | 1c6adde3-3959-4ae5-be0c-36cb71a41e1f | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | revision_number | 0 | | segment_id | | | service_types | | | subnetpool_id | | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-22T04:13:08Z | +-------------------+------------------------------------------------+ +++ Here's the instance that's created on this network: +++ (overcloud) [stack@undercloud16 ~]$ nova show 8ea2104f-e274-49e1-bc75-12d586b6da81 +--------------------------------------+----------------------------------------------------------+ | Property | Value | +--------------------------------------+----------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | overcloud-novacompute-0.site1.redhat.local | | OS-EXT-SRV-ATTR:hostname | test1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | overcloud-novacompute-0.site1.redhat.local | | OS-EXT-SRV-ATTR:instance_name | instance-00000003 | | OS-EXT-SRV-ATTR:kernel_id | | | OS-EXT-SRV-ATTR:launch_index | 0 | | OS-EXT-SRV-ATTR:ramdisk_id | | | OS-EXT-SRV-ATTR:reservation_id | r-jknt9fzp | | OS-EXT-SRV-ATTR:root_device_name | /dev/vda | | OS-EXT-SRV-ATTR:user_data | - | | OS-EXT-STS:power_state | 1 | | OS-EXT-STS:task_state | - | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2021-04-22T08:27:20.000000 | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | config_drive | | | created | 2021-04-22T08:27:04Z | | description | - | | flavor:disk | 10 | | flavor:ephemeral | 0 | | flavor:extra_specs | {} | | flavor:original_name | m1.small | | flavor:ram | 1024 | | flavor:swap | 0 | | flavor:vcpus | 1 | | hostId | 12cde44bad72a26453736eb64754bb7442275fff90de022d79fbf5b4 | | host_status | UP | | id | 8ea2104f-e274-49e1-bc75-12d586b6da81 | | image | rhel7 (58b0637f-5c8f-41a9-b0ef-62d7bb6ec5ac) | | key_name | key1 | | locked | False | | locked_reason | - | | metadata | {} | | name | test1 | | net1 network | 172.16.2.229 | +++ here are the security group rules for the security group for the instances: +++ (overcloud) [stack@undercloud16 ~]$ openstack security group rule list secgroup1 +--------------------------------------+-------------+-----------+-----------+------------+-----------------------+ | ID | IP Protocol | Ethertype | IP Range | Port Range | Remote Security Group | +--------------------------------------+-------------+-----------+-----------+------------+-----------------------+ | 5021ef1c-ba5e-4411-8583-5f012b3e72be | None | IPv4 | 0.0.0.0/0 | | None | | 50a8de1a-000c-44a8-b39a-03fe4e7abb2d | None | IPv6 | ::/0 | | None | | 87452278-fe04-4a33-9260-9b6175cda680 | icmp | IPv4 | 0.0.0.0/0 | | None | | e78fd417-fca6-438b-a6c8-18852acae20b | tcp | IPv4 | 0.0.0.0/0 | 22:22 | None | +--------------------------------------+-------------+-----------+-----------+------------+-----------------------+ +++ then I created a vip port with the same security group: +++ (overcloud) [stack@undercloud16 ~]$ neutron port-create --fixed-ip subnet_id=12b24308-c016-4617-a13d-c590586d382b,ip_address=172.16.2.25 1c6adde3-3959-4ae5-be0c-36cb71a41e1f neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Created a new port: +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | | | binding:profile | {} | | binding:vif_details | {} | | binding:vif_type | unbound | | binding:vnic_type | normal | | created_at | 2021-04-22T08:36:10Z | | description | | | device_id | | | device_owner | | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "12b24308-c016-4617-a13d-c590586d382b", "ip_address": "172.16.2.25"} | | id | c8f99338-4b15-407c-bfe1-3de559b5b8ac | | ip_allocation | immediate | | mac_address | fa:16:3e:7f:71:66 | | name | | | network_id | 1c6adde3-3959-4ae5-be0c-36cb71a41e1f | | port_security_enabled | True | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | qos_policy_id | | | resource_request | | | revision_number | 1 | | security_groups | 0de5a2ad-1521-4324-a66f-da1d61c34abc | | status | DOWN | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-22T08:36:11Z | +-----------------------+------------------------------------------------------------------------------------+ (overcloud) [stack@undercloud16 ~]$ neutron port-update --security-group 21ebe018-314b-4c85-88ef-1465b4f25653 c8f99338-4b15-407c-bfe1-3de559b5b8ac neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Updated port: c8f99338-4b15-407c-bfe1-3de559b5b8ac (overcloud) [stack@undercloud16 ~]$ neutron port-show c8f99338-4b15-407c-bfe1-3de559b5b8ac neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +-----------------------+------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | | | binding:host_id | | | binding:profile | {} | | binding:vif_details | {} | | binding:vif_type | unbound | | binding:vnic_type | normal | | created_at | 2021-04-22T08:36:10Z | | description | | | device_id | | | device_owner | | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "12b24308-c016-4617-a13d-c590586d382b", "ip_address": "172.16.2.25"} | | id | c8f99338-4b15-407c-bfe1-3de559b5b8ac | | ip_allocation | immediate | | mac_address | fa:16:3e:7f:71:66 | | name | | | network_id | 1c6adde3-3959-4ae5-be0c-36cb71a41e1f | | port_security_enabled | True | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | qos_policy_id | | | resource_request | | | revision_number | 2 | | security_groups | 21ebe018-314b-4c85-88ef-1465b4f25653 | | status | DOWN | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-22T08:37:21Z | +-----------------------+------------------------------------------------------------------------------------+ +++ then I added this port as a allowed-address-pair on the interface for the test1 instance: +++ (overcloud) [stack@undercloud16 ~]$ nova list +--------------------------------------+-------+--------+------------+-------------+-------------------+ | ID | Name | Status | Task State | Power State | Networks | +--------------------------------------+-------+--------+------------+-------------+-------------------+ | 8ea2104f-e274-49e1-bc75-12d586b6da81 | test1 | ACTIVE | - | Running | net1=172.16.2.229 | | a47fc4a2-68bc-4b4d-81ac-27d65b130860 | test2 | ACTIVE | - | Running | net2=192.24.5.184 | +--------------------------------------+-------+--------+------------+-------------+-------------------+ (overcloud) [stack@undercloud16 ~]$ nova interface-list 8ea2104f-e274-49e1-bc75-12d586b6da81 +------------+--------------------------------------+--------------------------------------+--------------+-------------------+-----+ | Port State | Port ID | Net ID | IP addresses | MAC Addr | Tag | +------------+--------------------------------------+--------------------------------------+--------------+-------------------+-----+ | ACTIVE | d667e73e-9ff3-4821-9858-8093cef4e142 | 1c6adde3-3959-4ae5-be0c-36cb71a41e1f | 172.16.2.229 | fa:16:3e:5d:0a:70 | - | +------------+--------------------------------------+--------------------------------------+--------------+-------------------+-----+ (overcloud) [stack@undercloud16 ~]$ neutron port-update --allowed-address-pair ip_address=172.16.2.25 d667e73e-9ff3-4821-9858-8093cef4e142 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Updated port: d667e73e-9ff3-4821-9858-8093cef4e142 (overcloud) [stack@undercloud16 ~]$ neutron port-show d667e73e-9ff3-4821-9858-8093cef4e142 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +-----------------------+--------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+--------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | True | | allowed_address_pairs | {"mac_address": "fa:16:3e:5d:0a:70", "ip_address": "172.16.2.25"} | | binding:host_id | overcloud-novacompute-0.site1.redhat.local | | binding:profile | {} | | binding:vif_details | {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": true, "datapath_type": "system", "bridge_name": "br-int"} | | binding:vif_type | ovs | | binding:vnic_type | normal | | created_at | 2021-04-22T08:27:11Z | | description | | | device_id | 8ea2104f-e274-49e1-bc75-12d586b6da81 | | device_owner | compute:nova | | extra_dhcp_opts | | | fixed_ips | {"subnet_id": "12b24308-c016-4617-a13d-c590586d382b", "ip_address": "172.16.2.229"} | | id | d667e73e-9ff3-4821-9858-8093cef4e142 | | ip_allocation | immediate | | mac_address | fa:16:3e:5d:0a:70 | | name | | | network_id | 1c6adde3-3959-4ae5-be0c-36cb71a41e1f | | port_security_enabled | True | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | qos_policy_id | | | resource_request | | | revision_number | 5 | | security_groups | 21ebe018-314b-4c85-88ef-1465b4f25653 | | status | ACTIVE | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-22T08:38:45Z | +-----------------------+--------------------------------------------------------------------------------------------------------------------------+ (overcloud) [stack@undercloud16 ~]$ +++ then inside the test1 instance; I set the vip address on the port: +++ [root@test1 ~]# nmcli conn show NAME UUID TYPE DEVICE System eth0 5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 ethernet eth0 [root@test1 ~]# nmcli conn mod "System eth0" +ipv4.address 172.16.2.25/24 [root@test1 ~]# nmcli conn up "System eth0" Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/2) [root@test1 ~]# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast state UP group default qlen 1000 link/ether fa:16:3e:5d:0a:70 brd ff:ff:ff:ff:ff:ff inet 172.16.2.229/24 brd 172.16.2.255 scope global noprefixroute dynamic eth0 valid_lft 86399sec preferred_lft 86399sec inet 172.16.2.25/24 brd 172.16.2.255 scope global secondary noprefixroute eth0 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe5d:a70/64 scope link tentative valid_lft forever preferred_lft forever [root@test1 ~]# +++ here's the router to which the instance subnet: +++ (overcloud) [stack@undercloud16 ~]$ neutron router-show 9ea43f9e-841a-4f31-a00b-bb0b021556a8 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | False | | availability_zone_hints | | | availability_zones | nova | | created_at | 2021-04-22T04:13:53Z | | description | | | distributed | True | | external_gateway_info | {"network_id": "5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0", "external_fixed_ips": [{"subnet_id": "9af50eb4-b0ad-4e9d-876e-cc3c9d35bb4d", "ip_address": "192.168.122.26"}], "enable_snat": true} | | flavor_id | | | ha | False | | id | 9ea43f9e-841a-4f31-a00b-bb0b021556a8 | | name | router1 | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | revision_number | 17 | | routes | | | status | ACTIVE | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-23T03:25:31Z | +-------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +++ then I created a floating ip and attached it to the port of the instance: +++ (overcloud) [stack@undercloud16 ~]$ neutron floatingip-create 5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Created a new floatingip: +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | created_at | 2021-04-23T02:16:25Z | | description | | | fixed_ip_address | | | floating_ip_address | 192.168.122.28 | | floating_network_id | 5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0 | | id | 96d5cfa3-1500-44be-9f5f-2c5edecec6b0 | | port_details | | | port_id | | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | qos_policy_id | | | revision_number | 0 | | router_id | | | status | DOWN | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-23T02:16:25Z | +---------------------+--------------------------------------+ (overcloud) [stack@undercloud16 ~]$ neutron floatingip-associate 96d5cfa3-1500-44be-9f5f-2c5edecec6b0 d667e73e-9ff3-4821-9858-8093cef4e142 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Associated floating IP 96d5cfa3-1500-44be-9f5f-2c5edecec6b0 (overcloud) [stack@undercloud16 ~]$ ping 192.168.122.28 PING 192.168.122.28 (192.168.122.28) 56(84) bytes of data. 64 bytes from 192.168.122.28: icmp_seq=1 ttl=62 time=7.28 ms 64 bytes from 192.168.122.28: icmp_seq=2 ttl=62 time=1.38 ms ^C --- 192.168.122.28 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 2ms rtt min/avg/max/mdev = 1.376/4.329/7.282/2.953 ms +++ this is reachable and there are no issues however when we create another floating ip and attach it to the vip port; then the vip is not reachable over the floating ip: +++ (overcloud) [stack@undercloud16 ~]$ neutron floatingip-create 5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0 neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Created a new floatingip: +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | created_at | 2021-04-23T02:19:22Z | | description | | | fixed_ip_address | | | floating_ip_address | 192.168.122.31 | | floating_network_id | 5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0 | | id | 43364dcb-94e7-4351-88fa-a360fcf1f049 | | port_details | | | port_id | | | project_id | d20a046f5c9140a78993c1e0dd3e6b58 | | qos_policy_id | | | revision_number | 0 | | router_id | | | status | DOWN | | tags | | | tenant_id | d20a046f5c9140a78993c1e0dd3e6b58 | | updated_at | 2021-04-23T02:19:22Z | +---------------------+--------------------------------------+ (overcloud) [stack@undercloud16 ~]$ neutron floatingip-associate 43364dcb-94e7-4351-88fa-a360fcf1f049 c8f99338-4b15-407c-bfe1-3de559b5b8ac neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. Associated floating IP 43364dcb-94e7-4351-88fa-a360fcf1f049 (overcloud) [stack@undercloud16 ~]$ ping 192.168.122.31 PING 192.168.122.31 (192.168.122.31) 56(84) bytes of data. ^C --- 192.168.122.31 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 67ms (overcloud) [stack@undercloud16 ~]$ +++ if we check the fip namespace on the compute node then we don't see any entry for the floating ip for the vip in the namespace: +++ [root@overcloud-novacompute-0 ~]# ip netns exec fip-5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0 ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: fpr-9ea43f9e-8@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether 8e:43:e2:de:36:cb brd ff:ff:ff:ff:ff:ff link-netns qrouter-9ea43f9e-841a-4f31-a00b-bb0b021556a8 inet 169.254.110.47/31 scope global fpr-9ea43f9e-8 valid_lft forever preferred_lft forever inet6 fe80::8c43:e2ff:fede:36cb/64 scope link valid_lft forever preferred_lft forever 26: fg-1405374c-e6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:46:87:c6 brd ff:ff:ff:ff:ff:ff inet 192.168.122.32/24 brd 192.168.122.255 scope global fg-1405374c-e6 valid_lft forever preferred_lft forever inet6 fe80::f816:3eff:fe46:87c6/64 scope link valid_lft forever preferred_lft forever [root@overcloud-novacompute-0 ~]# ip netns exec fip-5a1bb7a4-bbf5-4f79-9a4c-934fe526e0a0 ip route 169.254.110.46/31 dev fpr-9ea43f9e-8 proto kernel scope link src 169.254.110.47 192.168.122.0/24 dev fg-1405374c-e6 proto kernel scope link src 192.168.122.32 192.168.122.28 via 169.254.110.46 dev fpr-9ea43f9e-8 proto static +++ Version-Release number of selected component (if applicable): [root@overcloud-novacompute-0 ~]# podman ps | grep -i neutron 85dcd920eede undercloud16.ctlplane.site1.redhat.local:8787/rhosp-rhel8/openstack-neutron-openvswitch-agent:16.1.5 kolla_start 25 hours ago Up 20 hours ago neutron_ovs_agent b1ecdf929992 undercloud16.ctlplane.site1.redhat.local:8787/rhosp-rhel8/openstack-neutron-metadata-agent:16.1.5 kolla_start 25 hours ago Up 25 hours ago neutron_metadata_agent e58ed928ff61 undercloud16.ctlplane.site1.redhat.local:8787/rhosp-rhel8/openstack-neutron-l3-agent:16.1.5 kolla_start 25 hours ago Up 21 hours ago neutron_l3_agent How reproducible: Always Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.7 (Train) bug fix and enhancement advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:3762