Hide Forgot
Created attachment 1517666 [details] full console output + log traces of the non-pingable fip port Description of problem: On OSP 14 (and probably on OSP13 as well), launching instance with 2 NICs, with 2 floating ips, will have only the first FIP reachable, but not the second FIP. Version-Release number of selected component (if applicable): OSP14 -p 2018-12-20.4 How reproducible: Always Steps to Reproduce: 1. Install OSP 2. Launch a VM with 2 NICs 3. Assign FIP to each NIC 4. Ping each FIP You can use this script to do steps 2-4: https://code.engineering.redhat.com/gerrit/gitweb?p=Neutron-QE.git;a=blob;f=Scripts/create_multi_topology.sh ./create_multi_topology.sh -q -i cirros35 -t mni -e skip -n 2 -v 1 -c YES Actual results: [2018-12-31 09:09:19] Creating Router and 2 Networks - each one with both IPv4 and IPv6 Subnets: +-------------------------+--------------------------------------+ | Field | Value | +-------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | None | | availability_zones | None | | created_at | 2018-12-31T14:09:21Z | | description | | | distributed | None | | external_gateway_info | None | | flavor_id | None | | ha | None | | id | 1ab38f3f-5c45-4b30-a47a-cee93cc9cffc | | name | Router_eNet | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | revision_number | 0 | | routes | | | status | ACTIVE | | tags | | | updated_at | 2018-12-31T14:09:21Z | +-------------------------+--------------------------------------+ [2018-12-31 09:09:25] Creating Network net_ipv64_1: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2018-12-31T14:09:28Z | | description | | | dns_domain | | | id | 8078ff43-69f3-4633-8c65-7240d199963a | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | mtu | 1442 | | name | net_ipv64_1 | | port_security_enabled | True | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | provider:network_type | None | | provider:physical_network | None | | provider:segmentation_id | None | | qos_policy_id | None | | revision_number | 2 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | | | tags | | | updated_at | 2018-12-31T14:09:28Z | +---------------------------+--------------------------------------+ [2018-12-31 09:09:29] Creating ipv4 Subnet on net_ipv64_1 - subnet_ipv4_1: +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 10.0.1.2-10.0.1.254 | | cidr | 10.0.1.0/24 | | created_at | 2018-12-31T14:09:33Z | | description | | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 10.0.1.1 | | host_routes | | | id | bcb38bbf-86e2-4754-b51a-503967b1f251 | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | subnet_ipv4_1 | | network_id | 8078ff43-69f3-4633-8c65-7240d199963a | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2018-12-31T14:09:33Z | +-------------------+--------------------------------------+ [2018-12-31 09:09:35] Creating ipv6 Subnet on net_ipv64_1 - subnet_ipv6_1: +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 2001::2-2001::ffff:ffff:ffff:ffff | | cidr | 2001::/64 | | created_at | 2018-12-31T14:09:39Z | | description | | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 2001::1 | | host_routes | | | id | 5cb796bc-c136-4f3c-ba89-4bf4c19beb54 | | ip_version | 6 | | ipv6_address_mode | slaac | | ipv6_ra_mode | slaac | | name | subnet_ipv6_1 | | network_id | 8078ff43-69f3-4633-8c65-7240d199963a | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2018-12-31T14:09:39Z | +-------------------+--------------------------------------+ [2018-12-31 09:09:40] Adding subnet_ipv4_1 and subnet_ipv6_1 to the router. [2018-12-31 09:09:56] Creating Network net_ipv64_2: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | UP | | availability_zone_hints | | | availability_zones | | | created_at | 2018-12-31T14:09:59Z | | description | | | dns_domain | | | id | 3505cd39-e760-46fc-9a33-58f7efd4cc6c | | ipv4_address_scope | None | | ipv6_address_scope | None | | is_default | False | | is_vlan_transparent | None | | mtu | 1442 | | name | net_ipv64_2 | | port_security_enabled | True | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | provider:network_type | None | | provider:physical_network | None | | provider:segmentation_id | None | | qos_policy_id | None | | revision_number | 2 | | router:external | Internal | | segments | None | | shared | False | | status | ACTIVE | | subnets | | | tags | | | updated_at | 2018-12-31T14:09:59Z | +---------------------------+--------------------------------------+ [2018-12-31 09:10:00] Creating ipv4 Subnet on net_ipv64_2 - subnet_ipv4_2: +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 10.0.2.2-10.0.2.254 | | cidr | 10.0.2.0/24 | | created_at | 2018-12-31T14:10:04Z | | description | | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 10.0.2.1 | | host_routes | | | id | 23d79776-addc-436d-88e4-8458eb2ee98f | | ip_version | 4 | | ipv6_address_mode | None | | ipv6_ra_mode | None | | name | subnet_ipv4_2 | | network_id | 3505cd39-e760-46fc-9a33-58f7efd4cc6c | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2018-12-31T14:10:04Z | +-------------------+--------------------------------------+ [2018-12-31 09:10:06] Creating ipv6 Subnet on net_ipv64_2 - subnet_ipv6_2: +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | allocation_pools | 2002::2-2002::ffff:ffff:ffff:ffff | | cidr | 2002::/64 | | created_at | 2018-12-31T14:10:10Z | | description | | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 2002::1 | | host_routes | | | id | 24ec6c15-a7fc-47db-bffc-29228d458329 | | ip_version | 6 | | ipv6_address_mode | slaac | | ipv6_ra_mode | slaac | | name | subnet_ipv6_2 | | network_id | 3505cd39-e760-46fc-9a33-58f7efd4cc6c | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | revision_number | 0 | | segment_id | None | | service_types | | | subnetpool_id | None | | tags | | | updated_at | 2018-12-31T14:10:10Z | +-------------------+--------------------------------------+ [2018-12-31 09:10:12] Adding subnet_ipv4_2 and subnet_ipv6_2 to the router. [2018-12-31 09:10:28] Connecting the router to the external network "public" [2018-12-31 09:10:37] Creating security group rules for group "sec_group" +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:10:43Z | | description | | | direction | ingress | | ether_type | IPv4 | | id | 0d21d7bf-e8b3-46b8-a78c-682cdd889dca | | name | None | | port_range_max | 80 | | port_range_min | 80 | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | tcp | | remote_group_id | None | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:10:43Z | +-------------------+--------------------------------------+ +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:10:47Z | | description | | | direction | ingress | | ether_type | IPv4 | | id | da7dd7ea-7e1b-4cd4-a027-7baf585e5d22 | | name | None | | port_range_max | 22 | | port_range_min | 22 | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | tcp | | remote_group_id | None | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:10:47Z | +-------------------+--------------------------------------+ +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:10:50Z | | description | | | direction | ingress | | ether_type | IPv4 | | id | 83c7e770-9923-4c0d-8f2b-361461d76e89 | | name | None | | port_range_max | 443 | | port_range_min | 443 | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | tcp | | remote_group_id | None | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:10:50Z | +-------------------+--------------------------------------+ +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:10:54Z | | description | | | direction | ingress | | ether_type | IPv4 | | id | 09b9cebd-4d3e-4993-a7a2-703fc4dfa111 | | name | None | | port_range_max | None | | port_range_min | None | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | icmp | | remote_group_id | None | | remote_ip_prefix | 0.0.0.0/0 | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:10:54Z | +-------------------+--------------------------------------+ +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:10:57Z | | description | | | direction | ingress | | ether_type | IPv6 | | id | b89e0c8f-42ed-40e0-a527-f63ace21503a | | name | None | | port_range_max | None | | port_range_min | None | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | tcp | | remote_group_id | None | | remote_ip_prefix | None | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:10:57Z | +-------------------+--------------------------------------+ +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:11:01Z | | description | | | direction | ingress | | ether_type | IPv6 | | id | 9b27777d-dc06-45de-b90b-b5f09e78745b | | name | None | | port_range_max | None | | port_range_min | None | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | udp | | remote_group_id | None | | remote_ip_prefix | None | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:11:01Z | +-------------------+--------------------------------------+ +-------------------+--------------------------------------+ | Field | Value | +-------------------+--------------------------------------+ | created_at | 2018-12-31T14:11:04Z | | description | | | direction | ingress | | ether_type | IPv6 | | id | abb91775-fa54-4688-9b32-042065784fa3 | | name | None | | port_range_max | None | | port_range_min | None | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | protocol | icmp | | remote_group_id | None | | remote_ip_prefix | None | | revision_number | 0 | | security_group_id | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | updated_at | 2018-12-31T14:11:04Z | +-------------------+--------------------------------------+ +--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Remote Security Group | Security Group | +--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+ | 09b9cebd-4d3e-4993-a7a2-703fc4dfa111 | icmp | 0.0.0.0/0 | | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | 0d21d7bf-e8b3-46b8-a78c-682cdd889dca | tcp | 0.0.0.0/0 | 80:80 | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | 32adbc95-bf38-411f-88e0-6cb0d2f12b3f | None | None | | None | cbea8754-8fff-4358-b099-365161ad748d | | 55081afd-3c16-4a58-a4b5-86f5f42e0f8d | None | None | | None | cbea8754-8fff-4358-b099-365161ad748d | | 83c7e770-9923-4c0d-8f2b-361461d76e89 | tcp | 0.0.0.0/0 | 443:443 | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | 8abe0993-d058-4817-9151-63928462bc1d | None | None | | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | 9b27777d-dc06-45de-b90b-b5f09e78745b | udp | None | | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | 9d5f5431-2971-448c-99e8-2b85db64c1ae | None | None | | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | abb91775-fa54-4688-9b32-042065784fa3 | icmp | None | | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | b89e0c8f-42ed-40e0-a527-f63ace21503a | tcp | None | | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | cd54dacc-bd16-4378-bfcd-2ecfec716f2e | None | None | | cbea8754-8fff-4358-b099-365161ad748d | cbea8754-8fff-4358-b099-365161ad748d | | d4e8ed7d-d1b9-47b0-a78f-92afbc560606 | None | None | | cbea8754-8fff-4358-b099-365161ad748d | cbea8754-8fff-4358-b099-365161ad748d | | da7dd7ea-7e1b-4cd4-a027-7baf585e5d22 | tcp | 0.0.0.0/0 | 22:22 | None | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | +--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+ [2018-12-31 09:11:07] Creating openstack key pair to easily login into VMs: +-------------+-------------------------------------------------+ | Field | Value | +-------------+-------------------------------------------------+ | fingerprint | ba:f6:07:22:e7:42:66:8d:5b:83:83:0b:a5:6d:64:b7 | | name | tester-key | | user_id | 6c0e03c0e66c4288b0fa03592faec0fc | +-------------+-------------------------------------------------+ +------------+-------------------------------------------------+ | Name | Fingerprint | +------------+-------------------------------------------------+ | tester-key | ba:f6:07:22:e7:42:66:8d:5b:83:83:0b:a5:6d:64:b7 | +------------+-------------------------------------------------+ [2018-12-31 09:11:16] Creating 1 VM instanses. [2018-12-31 09:11:21] Creating and booting VM instance with 2 NICs: cirros35_vm1 +-----------------------------+------------------------------------------------------+ | Field | Value | +-----------------------------+------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | | | OS-EXT-STS:power_state | NOSTATE | | OS-EXT-STS:task_state | scheduling | | OS-EXT-STS:vm_state | building | | OS-SRV-USG:launched_at | None | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | | | adminPass | YcTkfpHG4iyy | | config_drive | | | created | 2018-12-31T14:11:35Z | | flavor | cirros_flavor (8c351db5-3e7c-443c-80c4-477c513b54e6) | | hostId | | | id | 2db24404-1fa2-4e76-8bfd-a8389bd4ccf4 | | image | cirros35 (101123d6-f0d8-4970-b1c2-92ee996410a1) | | key_name | tester-key | | name | cirros35_vm1 | | progress | 0 | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | properties | | | security_groups | name='eaf10726-f9f4-43ce-a40f-63b9e60c5735' | | status | BUILD | | updated | 2018-12-31T14:11:36Z | | user_id | 6c0e03c0e66c4288b0fa03592faec0fc | | volumes_attached | | +-----------------------------+------------------------------------------------------+ | progress | 0 | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | properties | | | security_groups | name='sec_group' | | | name='sec_group' | | status | ACTIVE | [2018-12-31 09:12:17] Configuring Networks for each of the 2 NICs: net_ipv64_2=10.0.2.3, net_ipv64_1= 10.0.1.10 [2018-12-31 09:12:25] Adding floating ip 10.0.0.210 to cirros35_vm1, and connecting to IP 10.0.1.10 (Port 99a072af-08f1-44f1-8e72-e45881a74db0). [2018-12-31 09:12:41] Setting a NAME to the new floating ip port: "cirros35_vm1_10.0.0.210" +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | None | | binding_profile | None | | binding_vif_details | None | | binding_vif_type | None | | binding_vnic_type | normal | | created_at | 2018-12-31T14:11:41Z | | data_plane_status | None | | description | | | device_id | 2db24404-1fa2-4e76-8bfd-a8389bd4ccf4 | | device_owner | compute:nova | | dns_assignment | fqdn='host-10-0-1-10.openstacklocal.', hostname='host-10-0-1-10', ip_address='10.0.1.10' | | | fqdn='host-2001--f816-3eff-fec3-c859.openstacklocal.', hostname='host-2001--f816-3eff-fec3-c859', ip_address='2001::f816:3eff:fec3:c859' | | dns_domain | None | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.1.10', subnet_id='bcb38bbf-86e2-4754-b51a-503967b1f251' | | | ip_address='2001::f816:3eff:fec3:c859', subnet_id='5cb796bc-c136-4f3c-ba89-4bf4c19beb54' | | id | 99a072af-08f1-44f1-8e72-e45881a74db0 | | mac_address | fa:16:3e:c3:c8:59 | | name | cirros35_vm1_10.0.0.210 | | network_id | 8078ff43-69f3-4633-8c65-7240d199963a | | port_security_enabled | True | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | qos_policy_id | None | | revision_number | 6 | | security_group_ids | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2018-12-31T14:12:45Z | +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+ [2018-12-31 09:12:48] Waiting for port status to be ACTIVE on cirros35_vm1, with internal IP address 10.0.1.10: | dns_domain | None | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.1.10', subnet_id='bcb38bbf-86e2-4754-b51a-503967b1f251' | | | ip_address='2001::f816:3eff:fec3:c859', subnet_id='5cb796bc-c136-4f3c-ba89-4bf4c19beb54' | | id | 99a072af-08f1-44f1-8e72-e45881a74db0 | | mac_address | fa:16:3e:c3:c8:59 | | name | cirros35_vm1_10.0.0.210 | | network_id | 8078ff43-69f3-4633-8c65-7240d199963a | | port_security_enabled | True | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | qos_policy_id | None | | revision_number | 6 | | security_group_ids | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | status | ACTIVE | [2018-12-31 09:12:51] Pinging the new FIP 10.0.0.210 from Undercloud (North-South test): PING 10.0.0.210 (10.0.0.210) 56(84) bytes of data. 64 bytes from 10.0.0.210: icmp_seq=2 ttl=63 time=2.13 ms 64 bytes from 10.0.0.210: icmp_seq=3 ttl=63 time=1.36 ms 64 bytes from 10.0.0.210: icmp_seq=4 ttl=63 time=0.930 ms 64 bytes from 10.0.0.210: icmp_seq=5 ttl=63 time=0.969 ms 64 bytes from 10.0.0.210: icmp_seq=6 ttl=63 time=0.887 ms --- 10.0.0.210 ping statistics --- 6 packets transmitted, 5 received, 16% packet loss, time 5003ms rtt min/avg/max/mdev = 0.887/1.256/2.135/0.472 ms % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to 10.0.0.210:80; Connection refused [2018-12-31 09:12:56] Generating ssh key to access cirros35_vm1 on 10.0.0.210: ssh-keygen -f ~/.ssh/known_hosts -R $fip Host 10.0.0.210 not found in /home/stack/.ssh/known_hosts [2018-12-31 09:12:56] Checking inside cirros35_vm1 uptime and pinging 8.8.8.8 (SNAT test): COUNT=0; until ssh -i $KEY_FILE -o StrictHostKeyChecking=no $ssh_user@$fip "ping 8.8.8.8 -c 1 && uptime" || [ $COUNT -eq 5 ]; do echo $(( COUNT++ )); done Warning: Permanently added '10.0.0.210' (RSA) to the list of known hosts. PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=116 time=50.331 ms --- 8.8.8.8 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max = 50.331/50.331/50.331 ms 14:12:56 up 0 min, 0 users, load average: 0.00, 0.00, 0.00 [2018-12-31 09:13:05] Adding floating ip 10.0.0.237 to cirros35_vm1, and connecting to IP 10.0.2.3 (Port 73b6ee55-39f5-41f8-af03-b37109d7a758). [2018-12-31 09:13:20] Setting a NAME to the new floating ip port: "cirros35_vm1_10.0.0.237" +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | None | | binding_profile | None | | binding_vif_details | None | | binding_vif_type | None | | binding_vnic_type | normal | | created_at | 2018-12-31T14:11:43Z | | data_plane_status | None | | description | | | device_id | 2db24404-1fa2-4e76-8bfd-a8389bd4ccf4 | | device_owner | compute:nova | | dns_assignment | fqdn='host-10-0-2-3.openstacklocal.', hostname='host-10-0-2-3', ip_address='10.0.2.3' | | | fqdn='host-2002--f816-3eff-fe0b-678b.openstacklocal.', hostname='host-2002--f816-3eff-fe0b-678b', ip_address='2002::f816:3eff:fe0b:678b' | | dns_domain | None | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.2.3', subnet_id='23d79776-addc-436d-88e4-8458eb2ee98f' | | | ip_address='2002::f816:3eff:fe0b:678b', subnet_id='24ec6c15-a7fc-47db-bffc-29228d458329' | | id | 73b6ee55-39f5-41f8-af03-b37109d7a758 | | mac_address | fa:16:3e:0b:67:8b | | name | cirros35_vm1_10.0.0.237 | | network_id | 3505cd39-e760-46fc-9a33-58f7efd4cc6c | | port_security_enabled | True | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | qos_policy_id | None | | revision_number | 6 | | security_group_ids | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | status | ACTIVE | | tags | | | trunk_details | None | | updated_at | 2018-12-31T14:13:24Z | +-----------------------+------------------------------------------------------------------------------------------------------------------------------------------+ [2018-12-31 09:13:27] Waiting for port status to be ACTIVE on cirros35_vm1, with internal IP address 10.0.2.3: | dns_domain | None | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='10.0.2.3', subnet_id='23d79776-addc-436d-88e4-8458eb2ee98f' | | | ip_address='2002::f816:3eff:fe0b:678b', subnet_id='24ec6c15-a7fc-47db-bffc-29228d458329' | | id | 73b6ee55-39f5-41f8-af03-b37109d7a758 | | mac_address | fa:16:3e:0b:67:8b | | name | cirros35_vm1_10.0.0.237 | | network_id | 3505cd39-e760-46fc-9a33-58f7efd4cc6c | | port_security_enabled | True | | project_id | a820cebba90f4bd296e6e846e9f03d17 | | qos_policy_id | None | | revision_number | 6 | | security_group_ids | eaf10726-f9f4-43ce-a40f-63b9e60c5735 | | status | ACTIVE | [2018-12-31 09:13:29] Pinging the new FIP 10.0.0.237 from Undercloud (North-South test): PING 10.0.0.237 (10.0.0.237) 56(84) bytes of data. --- 10.0.0.237 ping statistics --- 30 packets transmitted, 0 received, 100% packet loss, time 29003ms Expected results: Both NIC should get reachable FIPs Additional info: It was also reproduced on OSP13.
sos reports: http://rhos-release.virt.bos.redhat.com/log/bz1662680
This might be a problem with routing on the instance where packets are coming/going from the "wrong" interfaces. Slawek will try and reproduce it.
Hi, I tested it locally on devstack running with Neutron from master branch. All worked fine for me. Only problem which I had was that eth1 interface wasn't UP and configured by default. So I had to SSH to my VM and run dhclient on eth1 interface. See below: Interfaces on my vm: [14:01:13] vagrant@devstack-ubuntu-ovs ~ $ nova interface-list vm2_nics +------------+--------------------------------------+--------------------------------------+----------------------+-------------------+ | Port State | Port ID | Net ID | IP addresses | MAC Addr | +------------+--------------------------------------+--------------------------------------+----------------------+-------------------+ | ACTIVE | 4b4178e4-7b14-443c-83ac-ac61d66f1bcb | 99a4d1f6-ba2a-4084-8287-09ecabce546e | 10.0.2.110,2002::1ef | fa:16:3e:40:d5:ec | | ACTIVE | ea04a6ba-d595-4c6d-a512-d47bc28fe05d | 62b3974e-7e6a-43b5-9f3a-b045f22ea794 | 10.0.1.88,2001::92 | fa:16:3e:03:06:08 | +------------+--------------------------------------+--------------------------------------+----------------------+-------------------+ Floating IPs: [14:07:39] vagrant@devstack-ubuntu-ovs ~ $ neutron floatingip-list neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead. +--------------------------------------+----------------------------------+------------------+---------------------+--------------------------------------+ | id | tenant_id | fixed_ip_address | floating_ip_address | port_id | +--------------------------------------+----------------------------------+------------------+---------------------+--------------------------------------+ | 50722b42-3150-4bfc-9954-c72c5d619872 | 72df2ab5d6544e1984feddd08e24cac6 | 10.0.2.110 | 10.10.0.212 | 4b4178e4-7b14-443c-83ac-ac61d66f1bcb | | c47070f9-8312-4fb8-9a94-5749e3953409 | 72df2ab5d6544e1984feddd08e24cac6 | 10.0.1.88 | 10.10.0.192 | ea04a6ba-d595-4c6d-a512-d47bc28fe05d | +--------------------------------------+----------------------------------+------------------+---------------------+--------------------------------------+ Network config on cirros vm: $ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000 link/ether fa:16:3e:03:06:08 brd ff:ff:ff:ff:ff:ff inet 10.0.1.88/24 brd 10.0.1.255 scope global eth0 inet6 fe80::f816:3eff:fe03:608/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether fa:16:3e:40:d5:ec brd ff:ff:ff:ff:ff:ff inet6 fe80::f816:3eff:fe40:d5ec/64 scope link valid_lft forever preferred_lft forever $ sudo /sbin/cirros-dhcpc up eth1 udhcpc (v1.20.1) started Sending discover... Sending select for 10.0.2.110... Lease of 10.0.2.110 obtained, lease time 86400 route: SIOCADDRT: File exists WARN: failed: route add -net "0.0.0.0/0" gw "10.0.2.1" $ ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000 link/ether fa:16:3e:03:06:08 brd ff:ff:ff:ff:ff:ff inet 10.0.1.88/24 brd 10.0.1.255 scope global eth0 inet6 fe80::f816:3eff:fe03:608/64 scope link valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000 link/ether fa:16:3e:40:d5:ec brd ff:ff:ff:ff:ff:ff inet 10.0.2.110/24 brd 10.0.2.255 scope global eth1 inet6 fe80::f816:3eff:fe40:d5ec/64 scope link valid_lft forever preferred_lft forever $ And now ping to each of FIPs: [14:08:04] vagrant@devstack-ubuntu-ovs ~ $ ping 10.10.0.212 -c 1 PING 10.10.0.212 (10.10.0.212) 56(84) bytes of data. 64 bytes from 10.10.0.212: icmp_seq=1 ttl=63 time=1.59 ms --- 10.10.0.212 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.596/1.596/1.596/0.000 ms [14:08:08] vagrant@devstack-ubuntu-ovs ~ $ ping 10.10.0.192 -c 1 PING 10.10.0.192 (10.10.0.192) 56(84) bytes of data. 64 bytes from 10.10.0.192: icmp_seq=1 ttl=63 time=1.44 ms --- 10.10.0.192 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.445/1.445/1.445/0.000 ms I think we can close this as not a bug because it's depends on guest OS configuration how it will work.
Closing per comment 3 I had similar issues in the past (which I workaround with additional SSH commands to Cirros VMs where I setup eth1)