Bug 1970907

Summary: RHOSP16.1: The FIP of VIP is not reachable when allowed_address is set to full subnet range.
Product: Red Hat OpenStack Reporter: Shravan Kumar Tiwari <shtiwari>
Component: python-networking-ovnAssignee: Rodolfo Alonso <ralonsoh>
Status: CLOSED WONTFIX QA Contact: Eran Kuris <ekuris>
Severity: high Docs Contact:
Priority: unspecified    
Version: 16.1 (Train)CC: apevec, lhh, majopela, nusiddiq, ralonsoh, scohen
Target Milestone: ---Flags: shtiwari: needinfo+
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-06-22 11:24:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1986337    
Bug Blocks:    

Description Shravan Kumar Tiwari 2021-06-11 12:43:46 UTC 
Customer use case scenario:

Tenant of RHOSP infra wants to add the whole network as "allowed-address" and they use Terraform to spin up an environment. So right now they would have to lookup the information on the VIP port they are creating, in order to make it the allowed-address. If they could just add the entire network there, they would not have to perform this lookup first.

Description of problem:

As soon as customer sets the whole tenant network as allowed-address on the instance ports, as he can't predict what the VIP IP address will be, the VIP address is reachable inside the tenant network, but no longer via its Floating IP address.

Version-Release number of selected component (if applicable):
RHOSP16.1 (ML2/OVN)

How reproducible:


Steps to Reproduce:
* Network and subnet is created;
* (Named) ports for the instances are created;
* (Named) port for the virtual IP address is created;
* Network subnet of VIP-port is added as allowed-address on the instance ports;
* Floating IP addresses for the instance- and VIP-ports are allocated and assigned;
* Instances are created and configured with keepalived

Actual results:
ping from external to VIP FIP -> no response

Expected results:
The FIP of VIP should be reachable from outside

Additional info:

- Same setup works perfectly fine if we use single IP (VIP) in the allowed-address filed for the instance port (FIP of VIP is pingable from external side).
Comment 5 Rodolfo Alonso 2021-06-16 11:29:11 UTC 
Hello Shravan:

As you pointed out, this is related to [1]. The FIP VIP nat register is something like this:
_uuid               : e416fcb1-e25c-443a-a9e1-f3da7ccb5204
external_ids        : {"neutron:fip_external_mac"="fa:16:3e:30:ca:dc", "neutron:fip_id"="d8c5b251-f8b2-4eeb-b3f2-a300e504edc9", "neutron:fip_network_id"="265a39a3-e7c3-47bd-bd39-7536ec9d71a8", "neutron:fip_port_id"="7e2dec53-b04a-4306-aee9-55d95e952051", "neutron:revision_number"="1", "neutron:router_name"=neutron-a49e08e4-8d75-42f6-aa6a-7bad7f6b5f21}
external_ip         : "192.168.40.155"
external_mac        : []
logical_ip          : "10.0.10.10"
logical_port        : "7e2dec53-b04a-4306-aee9-55d95e952051"
options             : {}
type                : dnat_and_snat


Because the FIP is distributed, the "external_mac" is needed to determine where the VIP is located and what chassis is hosting the FIP.

In the case of a VIP, the FIP nat register does not have this "external_mac" address until an ARP packet has been detected by OVN. The VIP should be set in one of the VMs; when the VIP replies, OVN determines the chassis this VM is located, sets the VIP port as "UP" and Neutron writes the "external_mac".

When the VM port that will also host the VIP has as allowed address pair an IP address (/32), those are the OVS flows added in br-int:
 cookie=0x9d01a1e3, duration=43.679s, table=9, n_packets=0, n_bytes=0, idle_age=43, priority=90,ip,reg14=0x6,metadata=0x26,dl_src=fa:16:3e:50:d1:9f,nw_src=10.0.10.10 actions=resubmit(,10)
 cookie=0x149ee681, duration=43.679s, table=10, n_packets=1, n_bytes=42, idle_age=12, priority=90,arp,reg14=0x6,metadata=0x26,dl_src=fa:16:3e:50:d1:9f,arp_spa=10.0.10.10,arp_sha=fa:16:3e:50:d1:9f actions=resubmit(,11)
 cookie=0x92381c1e, duration=43.681s, table=21, n_packets=0, n_bytes=0, idle_age=43, priority=100,arp,reg14=0x6,metadata=0x26,arp_spa=10.0.10.10,arp_tpa=10.0.10.10,arp_op=1 actions=controller(userdata=00.00.00.11.00.00.00.00.00.00.00.04),resubmit(,22)
 cookie=0x92381c1e, duration=43.681s, table=21, n_packets=1, n_bytes=42, idle_age=12, priority=100,arp,reg14=0x6,metadata=0x26,arp_spa=10.0.10.10,arp_op=2 actions=controller(userdata=00.00.00.11.00.00.00.00.00.00.00.04),resubmit(,22)
 cookie=0x7ad870d, duration=43.683s, table=48, n_packets=0, n_bytes=0, idle_age=43, priority=90,ip,reg15=0x6,metadata=0x26,dl_dst=fa:16:3e:50:d1:9f,nw_dst=10.0.10.10 actions=resubmit(,49)


The flows in table 21, matching the MAC and the IP address, will send this packet to a controller. I need confirmation on this, but this is when OVS declares this port as "UP", Neutron receives this event and writes the "external_mac".

But when in allowed address pair a network is added (10.0.10.0/26 or 10.0.10.0/27, for example), those are the flows set in br-int:
 cookie=0xb222c35b, duration=0.728s, table=9, n_packets=0, n_bytes=0, idle_age=0, priority=90,ip,reg14=0x6,metadata=0x26,dl_src=fa:16:3e:50:d1:9f,nw_src=10.0.10.0/27 actions=resubmit(,10)
 cookie=0x45697253, duration=0.728s, table=10, n_packets=0, n_bytes=0, idle_age=0, priority=90,arp,reg14=0x6,metadata=0x26,dl_src=fa:16:3e:50:d1:9f,arp_spa=10.0.10.0/27,arp_sha=fa:16:3e:50:d1:9f actions=resubmit(,11)
 cookie=0x195f1787, duration=0.730s, table=48, n_packets=0, n_bytes=0, idle_age=0, priority=90,ip,reg15=0x6,metadata=0x26,dl_dst=fa:16:3e:50:d1:9f,nw_dst=10.0.10.0/27 actions=resubmit(,49)


At this point I would like to ask Numan if this is a bug or a feature; that means, if only IP/32 addresses can be defined as VIP or we can use a network range.

Regards.


[1]https://bugzilla.redhat.com/show_bug.cgi?id=1707241