Created attachment 1094864 [details] yaml file Description of problem: Bridges and ovs bonds are not getting created as expected with this "partially successful deployment". a) How do we configure provider networks in controller yaml file. In our case it needs to be attached to bond-data. We have a range of provider networks here: [172.19.12.0/24, 172.19.13.0/24, 172.19.14.0/24, 172.19.15.0/24], how can we specify all these in the controller.yaml file ? I assume as per the previous update this is the external networks, you were referring to. To make deployment work, I associated 172.19.12.0/24 as an external network and attached br-ex in the network-environment.yaml? How do I associate other provider networks also in yaml file. b) In our case, we are planning to configure a Gluster volume as NFS share across compute node for ephemeral volume through pcs. This is in 172.19.8.0/24 network. Is this StorageIpSubnet which you were referring to ? Can you help in putting it across the yaml file. So in our case it will be associate to "eth2" of all compute nodes. c) What would be the steps to create provider networks for the compute node in this case? Can you share that? Should this be done after the overcloud deployment manually ? or can be done by heat templates ? d) Should we create servicemaps for all the openstack services to match it with 172.19.9.0/24(InternalAPI/OpenStack Management services)? Version-Release number of selected component (if applicable): RHOS7 OSPD7 openstack-tripleo-heat-templates-0.8.6-71.el7ost.noarch openstack-tripleo-0.0.7-0.1.1664e566.el7ost.noarch How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info: https://docs.google.com/document/d/1-91FhQkar_6R2WdEU601OKBjG732PsIyoxLgwf0C06A/edit#
yaml files needs to be modified to include the below network layout: Network Layout: Controller Nodes(3 - Dell PowerEdge R430) ========================================== Provisioning(172.19.3.x - em1 + em2 - 2 X 1G ) --> bond-prov InternalAPI/Openstack Management (172.19.9.x/24 - p2p1 + p2p2 - 2 X 10G) --> bond-api External/Provider Network/PublicAPI(172.19.12.x/24 - 172.19.15.x/24 - em3 + em4 - 2 X 1G) --> bond-data On compute nodes (Starting with 3 Nodes - Scalable to 28 Compute nodes - UCSB-B200-M3 - Following are VNICs) ============================================================================================================= Provisioning - eth0 (1G) 172.19.3.x/24 Internal API/OpenStack Management - eth1 (1G) 172.19.9.x/24 Storage(Gluster Mount) - eth2 (10G) - 172.19.8.x/24 External/Provider Network/PublicAPI - eth3 + eth4 (2 X 10G) --> bond-data - (172.19.12.x/24 - 172.19.15.x/24)
Hi Abhilash, right now we typically use the first nic for the controlplane/provisioning network and bond the 2nd/3rd interface with the various networks trunked to that, like the controller config for the bond-with-vlans setup[1]. In that case we use nics2/3 in a bond with the external, internal/api, storage/mgmt and tenant networks delivered as vlans trunked to the bond. I can't recall if there was a reason we couldn't use nic1 in a bond (e.g. after provisioning), so I need to confirm/check that unless someone else here knows. Otherwise, we would in need to change [1] to define more ovs-bonds - you have 3... and the appropriate vlans trunked to the bonds as in the outline in comment 3. This isn't something our current templates do and I don't know off-hand if there are issues with trunking vlans across multiple nic bonds on the same ovs br-ex. I am just responding from an irc query here, I will try and find out about the nic1 issue at least and revisit later, hope that is helpful as a first pointer, thanks, marios [1] https://github.com/openstack/tripleo-heat-templates/blob/master/network/config/bond-with-vlans/controller.yaml
I am attaching a network-environment.yaml, controller.yaml, and compute.yaml templates that should work for the requested network topology. Since the compute nodes are UCS hosts, the vnics are flexible. I configured bonds for each network, but in other deployments we have used a single vNIC per network, as shown in this template: https://review.openstack.org/#/c/204494/8/network/config/multiple-nics/compute.yaml The controller.yaml that is attached has the real NIC names, but the compute.yaml I used nic abstractions (nic1, nic2, etc.) that should map to the first through Nth vNIC.
Created attachment 1095583 [details] network-environment.yaml
Created attachment 1095584 [details] controller.yaml
Created attachment 1095585 [details] compute.yaml
Note, the above network-environment.yaml does not include the proper VLAN IDs for the various networks. Please add the following to network-environment.yaml once these are known: parameter_defaults: InternalApiNetworkVlanID: ??? StorageNetworkVlanID: ??? ExternalNetworkVlanID: ??? The provider networks are created as a post-deployment command. Both the controller and the compute will be connected to the provider networks via the br-ex bridge. Also, the allocation pools may need to be modified. The ExternalAllocationPool only needs to be large enough for one IP for each controller + one IP for each compute + a handful of IPs for routers and VIPs. The rest of the network can be used for VM IPs, so those will be used in the subnet-create command. Here is an example of the commands used to create a provider network on VLAN 201 (assuming the physical router gateway is .1): neutron net-create --provider:physical_network datacentre \ --provider:network_type vlan --provider:segmentation_id 201 \ --shared ext-net1 neutron subnet-create --name ext-subnet1 \ --enable_dhcp=False \ --allocation-pool start=172.19.12.101,end=172.19.12.250 \ --gateway 172.19.12.1 \ ext-net1 172.19.12.0/24 Then, you need to create a router for the subnet (here we choose an IP by creating a port and then attach it to the router) neutron router-create ext-router1 neutron port-create --fixed-ip subnet_id=ext-subnet1,ip_address=172.19.12.2 (grab the ID from the newly created port for the next command) neutron router-interface-add ext-router1 port=<UUID>
Also note that the network-environment.yaml file should include pointers to the NIC config templates, but I didn't include these because I didn't know the correct path. The resource_registry section of the network-environment.yaml should look something like this: resource_registry: OS::TripleO::Compute::Net::SoftwareConfig: /home/stack/templates/nic-configs/com pute.yaml OS::TripleO::Controller::Net::SoftwareConfig: /home/stack/templates/nic-configs/ controller.yaml
1. Tried deploying using the heat templates provided, could see bridges/bonds are getting created and IP addresses are getting associated. 2. The heat templates used are attached. 3. The deployment command is stuck for very long(more than a hour) - openstack overcloud deploy --templates ~/templates/overcloud -e /home/stack/templates/overcloud/environments/network-isolation.yaml -e /home/stack/templates/overcloud/network-environment.yaml --control-flavor control --compute-flavor compute --ntp-server 172.16.13.97 --neutron-network-type vxlan --neutron-tunnel-types vxlan Observations: 1. ssh access to controller via heat-admin is delayed but accessibility is there. For compute node ssh access via heat-admin is normal. 2. The internal API IP, External IP, Storage IP configured under controller doesn't reach via ping through director or compute nodes. 3. Provisioning IP of Controller node is not reachable via ping from customer's LAN. But Provisioning IP of compute node is reachable via ping from customer's LAN Customer's environment is documented at https://docs.google.com/document/d/1-91FhQkar_6R2WdEU601OKBjG732PsIyoxLgwf0C06A/edit Controller Nodes(3 - Dell PowerEdge R430) ========================================== Provisioning(172.19.3.x - em1 + em2 - 2 X 1G ) --> bond-prov InternalAPI/Openstack Management (172.19.9.x/24 - p2p1 + p2p2 - 2 X 10G) --> bond-api External/Provider Network/PublicAPI(172.19.12.x/24 - 172.19.15.x/24 - em3 + em4 - 2 X 1G) --> bond-data On compute nodes (Starting with 3 Nodes - Scalable to 28 Compute nodes - UCSB-B200-M3 - Following are VNICs) ============================================================================================================= Provisioning - eth0 (1G) 172.19.3.x/24 Internal API/OpenStack Management - eth1 (1G) 172.19.9.x/24 Storage(Gluster Mount) - eth2 (10G) - 172.19.8.x/24 External/Provider Network/PublicAPI - eth3 + eth4 (2 X 10G) --> bond-data - (172.19.12.x/24 - 172.19.15.x/24) Additional info to above tests: -Since Cisco UCS(Compute nodes) are well known for bonding limitations, tried removing that ovs_bond in compute.yaml and tried with 1 VNIC alone -To isolate out networking issues and in course of preparing test bed for customer to test their apps, I successfully deployed 1 Controller and 1 Compute with Packstack on Friday which went all good with proposed network layout and bonds and couldn't see any network issues there. -Now customer is insisting to go ahead with manual deployment of 3 Controllers and rest of compute nodes which would scale up to 28. I have started with deployment of 3 Controller and 2 compute with packstack as of now, but as advised from GSS this wouldn't be supported way forward. -This prototype we build, customer will test out and move it into production once they feel results are good and stable. Customer already have RHELOSP 5 manual deployment which we done last year and their workload runs good in that.
yaml files: http://10.65.231.4/sysreports/803942/01524102/30-yaml_2015-11-19.tar.gz sosreport from the overcloud compute node: http://10.65.231.4/sysreports/803942/01524102/10-sosreport-overcloud-compute-0.localdomain-20151118133509.tar.xz sosreport from the director node: http://10.65.231.4/sysreports/803942/01524102/20-sosreport-manager.cleartrip.com-20151119000130.tar.xz
(In reply to Abhilash S V from comment #11) > 2. The internal API IP, External IP, Storage IP configured under controller > doesn't reach via ping through director or compute nodes. This will be why the deploy is hanging. If the controller node can't be reached on those addresses then the deployment can't complete. > -To isolate out networking issues and in course of preparing test bed for > customer to test their apps, I successfully deployed 1 Controller and 1 > Compute with Packstack on Friday which went all good with proposed network > layout and bonds and couldn't see any network issues there. Note that packstack doesn't do network isolation like director does, so this doesn't really prove anything about the environment. The first thing to debug is why the controller can't be contacted on some of the isolated networks. You'll want to check the os-collect-config journal on a controller node and look for the os-net-config output to make sure the network setup ran correctly. Then take a look at the actual network configuration that happened - "ip a" and "ovs-vsctl show" in particular. Make sure that the configurations there match the actual hardware environment.
Looking through the heat-engine log on the undercloud, the first error is: 2015-11-18 14:00:52.625 19841 TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float: 2015-11-18 14:02:05.192 19844 TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float: 2015-11-18 14:02:47.940 19838 TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: resources[0].Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float: 2015-11-18 14:02:49.710 19843 TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: resources.Controller.resources[0].Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float: 2015-11-18 14:04:09.950 19837 TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: resources[0].Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float: 2015-11-18 14:04:11.611 19843 TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: resources.Compute.resources[0].Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float: This is potentially the root cause - try supplying a TenantNetworkVlanID; it looks like an empty string is being passed at the moment. (The part about converting to float is probably a red herring - Heat tries to convert a 'Number' parameter to an int, and failing that a float.) From that point on the undercloud is full of errors of the form: 2015-11-18 20:36:33.865 19837 ERROR heat.engine.resource [-] Error marking resource as failed 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource Traceback (most recent call last): 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 525, in _action_recorder 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource self.state_set(action, self.FAILED, '%s aborted' % action) 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1094, in state_set 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource self._store_or_update(action, status, reason) 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 1028, in _store_or_update 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource 'stack_id': self.stack.id, 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource File "/usr/lib/python2.7/site-packages/heat/engine/resource.py", line 225, in stack 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource assert stack is not None, "Need a reference to the Stack object" 2015-11-18 20:36:33.865 19837 TRACE heat.engine.resource AssertionError: Need a reference to the Stack object I've raised the upstream bug https://bugs.launchpad.net/heat/+bug/1519025 to investigate this issue. This could be what is causing the Heat process to appear IN_PROGRESS forever, but it is *not* the root cause - the error occurs while trying to cancel an operation, so something in the deployment is failing first.
Note that the first of the AssertionErrors is at 2015-11-18 14:57:21.980 and appears to be associated with a stack delete, so this is probably not the cause of the deploy hanging.
Customer doesn't have tenant networks in this case, so I removed the tenantvlanid from heat template once I got the following error. ==================== "TRACE heat.engine.resource ResourceFailure: resources.NetworkConfig: Parameter 'TenantNetworkVlanID' is invalid: could not convert string to float:" ==================== Customer has only provider networks where their vms will be connecting. So no separate vlans for tenants in this case. So I'm not sure whether you can call above error root cause. I have got an acknowledgement from customer to redeploy the environment with Director scrapping the packstack installation. I will collect the logs as mentioned in the BZ from the customer.
I see a number of problems with the tempates as attached to this ticket. In particular the deployment is being called with a tenant network, but since there is no tenant network defined we are getting errors: 1) No DNS servers defined in network-environment.yaml 2) No VLAN IDs or subnet info for the Storage Network defined in network-environment.yaml. 3) Missing required parameters from network-environment.yaml: parameter_defaults: ControlPlaneSubnetCidr: "24" EC2MetadataIp: <Undercloud_IP> ControlPlaneDefaultRoute: <Provisioning_Gateway> 4) Missing ServiceNetMap for determining the networks to place services on (most services will be on internal_api with the exception of those on those on storage) 5) Missing BondInterfaceOvsOptions parameter from network-environment.yaml 6) If custom networks are being used (no Tenant network, no Storage Management network), then the networks that are used need to be placed in network-environment.yaml, and network-isolation.yaml should not be included in the deployment parameters. 7) The compute.yaml and controller.yaml were modified to remove parameters from the Parameters: section. In particular, I see that references to the StorageVlanId and Externalq parameters have been removed. These are required. So the StorageNetworkVlanId needs to be in the compute.yaml template, and both the StorageNetworkVlanId and ExternalNetworkVlanId need to be included in both templates. What happened to the sample network-environment.yaml and NIC config templates that I provided? Were they not used as a basis for the deployment?
(In reply to Dan Sneddon from comment #17) Apologies, but the comments in #17 were based on the version of the templates attached to the ticket, not the ones from comment #12. The ones in comment #12 are almost correct. I see only the following issue with the templates from comment #12: 1) If custom networks are being used (no Tenant network, no Storage Management network), then the networks that are used need to be placed in network-environment.yaml, and network-isolation.yaml should not be included in the deployment parameters. So the solution may be to add the following to the resource_registry: section of network-environment.yaml (and do not include network-isolation.yaml in the deployment parameters): OS::TripleO::Network::External: /home/stack/templates/tripleo-heat-templates/network/external.yaml OS::TripleO::Network::InternalApi: /home/stack/templates/tripleo-heat-templates/network/internal_api.yaml OS::TripleO::Network::Storage: /home/stack/templates/tripleo-heat-templates/network/storage.yaml # Port assignments for the controller role OS::TripleO::Controller::Ports::ExternalPort: /home/stack/templates/tripleo-heat-templates/network/ports/external.yaml OS::TripleO::Controller::Ports::InternalApiPort: /home/stack/templates/tripleo-heat-templates/network/ports/internal_api.yaml OS::TripleO::Controller::Ports::StoragePort: /home/stack/templates/tripleo-heat-templates/network/ports/storage.yaml OS::TripleO::Controller::Ports::RedisVipPort: /usr/share/openstack-tripleo-heat-templates/network/ports/vip.yaml # Port assignments for the compute role OS::TripleO::Compute::Ports::InternalApiPort: /home/stack/templates/tripleo-heat-templates/network/ports/internal_api.yaml OS::TripleO::Compute::Ports::StoragePort: /home/stack/templates/tripleo-heat-templates/network/ports/storage.yaml # Port assignments for the ceph storage role OS::TripleO::CephStorage::Ports::StoragePort: /home/stack/templates/tripleo-heat-templates/network/ports/storage.yaml
I have modified the templates as per your last changes, but as discussed this has already been tried. The new templates are attached : yaml_2015-11-23.tar.gz Before going ahead with deployment, I'm in process of introspection of nodes. I'm going to deploy 3 controller and 2 compute nodes as below: openstack overcloud deploy --templates ~/templates/overcloud -e /home/stack/templates/overcloud/network-environment.yaml --control-scale 3 --compute-scale 2 --control-flavor control --compute-flavor compute --ntp-server 172.16.13.97 --neutron-network-type vxlan --neutron-tunnel-types vxlan
Created attachment 1097824 [details] yaml_2015_11_23
As per the error logs, I see that we are observing the following from os-collect-config in compute nodes: Nov 23 17:43:11 overcloud-compute-0.localdomain os-collect-config[8786]: Trying to ping 172.19.8.10 for local network 172.19.8.0/24...FAILURE Nov 23 17:43:11 overcloud-compute-0.localdomain os-collect-config[8786]: [2015-11-23 17:43:11,593] (heat-config) [DEBUG] 172.19.8.10 is not pingable. Local Network: 172.19.8.0/24 Nov 23 17:43:11 overcloud-compute-0.localdomain os-collect-config[8786]: [2015-11-23 17:43:11,593] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/bd147c1a-dbcf-47b9-acf0-84e3eea97f9d. [1] Customer is working to fix the routing issue. Will do redeployment once the issue is fixed from customer end.
Deployment has went through fine when the routing issues between compute and controller nodes are fixed. However there is still a concern at accessibility of controller nodes via local LAN at customer's environment [root@overcloud-controller-0 ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 172.19.12.1 0.0.0.0 UG 0 0 0 vlan132 169.254.169.254 172.19.3.21 255.255.255.255 UGH 0 0 0 br-prov 172.19.3.0 0.0.0.0 255.255.255.0 U 0 0 0 br-prov 172.19.8.0 0.0.0.0 255.255.255.0 U 0 0 0 vlan128 172.19.9.0 0.0.0.0 255.255.255.0 U 0 0 0 vlan129 172.19.12.0 0.0.0.0 255.255.255.0 U 0 0 0 vlan132 [heat-admin@overcloud-compute-0 ~]$ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 172.19.3.1 0.0.0.0 UG 0 0 0 br-prov 169.254.169.254 172.19.3.21 255.255.255.255 UGH 0 0 0 br-prov 172.19.3.0 0.0.0.0 255.255.255.0 U 0 0 0 br-prov 172.19.8.0 0.0.0.0 255.255.255.0 U 0 0 0 vlan128 172.19.9.0 0.0.0.0 255.255.255.0 U 0 0 0 vlan129 Ping test from local lan --------------------------------- test:~ asv$ ping 172.19.3.51 ————> Compute IP configured on br-prov PING 172.19.3.51 (172.19.3.51): 56 data bytes 64 bytes from 172.19.3.51: icmp_seq=0 ttl=60 time=150.715 ms 64 bytes from 172.19.3.51: icmp_seq=1 ttl=60 time=132.258 ms ^C --- 172.19.3.51 ping statistics --- 3 packets transmitted, 2 packets received, 33.3% packet loss round-trip min/avg/max/stddev = 132.258/141.487/150.715/9.228 ms test:~ asv$ ping 172.19.3.50 ————————> Controller IP configured on br-prov PING 172.19.3.50 (172.19.3.50): 56 data bytes Request timeout for icmp_seq 0 ^C --- 172.19.3.50 ping statistics --- 2 packets transmitted, 0 packets received, 100.0% packet loss All networks have their own gateway 172.19.3.0/24 — > 172.19.3.1 (provisioning network) 172.19.8.0/24 — > 172.19.8.1 (storage network) 172.19.9.0/24 —> 172.19.9.1 (API network) 172.19.12.0/24 —> 172.19.12.1 (external + provider network) 172.19.13.0/24 — > 172.19.13.1 (provider network) 172.19.14.0/24 — > 172.19.14.1 (provider network) 172.19.15.0/24 — > 172.19.15.1 (provider network) In customer’s environment all networks are VLAN tagged. So traffic coming to server is NIC based, which means (em1 + em2) configured for provisioning network is expected to handle 3.x traffic, which means we would need to do source routing so that it goes out of same nic, so that switch handles it without dropping which happens incase if its going through default gw of 172.19.12.1 (How can I route network traffic such that the packets go out via the same interface they came in? https://access.redhat.com/solutions/19596) 1. Can we do this via heat templates, if yes, how can this be achieved. 2. If no, should we configure the source routing in the controller/compute nodes manually, how will an update to the images affect the configuration in the future?
(In reply to Abhilash S V from comment #22) > Deployment has went through fine when the routing issues between compute and > controller nodes are fixed. However there is still a concern at > accessibility of controller nodes via local LAN at customer's environment > > [root@overcloud-controller-0 ~]# route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 0.0.0.0 172.19.12.1 0.0.0.0 UG 0 0 0 > vlan132 > 169.254.169.254 172.19.3.21 255.255.255.255 UGH 0 0 0 > br-prov > 172.19.3.0 0.0.0.0 255.255.255.0 U 0 0 0 > br-prov > 172.19.8.0 0.0.0.0 255.255.255.0 U 0 0 0 > vlan128 > 172.19.9.0 0.0.0.0 255.255.255.0 U 0 0 0 > vlan129 > 172.19.12.0 0.0.0.0 255.255.255.0 U 0 0 0 > vlan132 > > > [heat-admin@overcloud-compute-0 ~]$ route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric Ref Use Iface > 0.0.0.0 172.19.3.1 0.0.0.0 UG 0 0 0 > br-prov > 169.254.169.254 172.19.3.21 255.255.255.255 UGH 0 0 0 > br-prov > 172.19.3.0 0.0.0.0 255.255.255.0 U 0 0 0 > br-prov > 172.19.8.0 0.0.0.0 255.255.255.0 U 0 0 0 > vlan128 > 172.19.9.0 0.0.0.0 255.255.255.0 U 0 0 0 > vlan129 > > > Ping test from local lan > --------------------------------- > test:~ asv$ ping 172.19.3.51 ————> Compute IP configured on br-prov > PING 172.19.3.51 (172.19.3.51): 56 data bytes > 64 bytes from 172.19.3.51: icmp_seq=0 ttl=60 time=150.715 ms > 64 bytes from 172.19.3.51: icmp_seq=1 ttl=60 time=132.258 ms > ^C > --- 172.19.3.51 ping statistics --- > 3 packets transmitted, 2 packets received, 33.3% packet loss > round-trip min/avg/max/stddev = 132.258/141.487/150.715/9.228 ms > > > test:~ asv$ ping 172.19.3.50 ————————> Controller IP configured on > br-prov > PING 172.19.3.50 (172.19.3.50): 56 data bytes > Request timeout for icmp_seq 0 > ^C > --- 172.19.3.50 ping statistics --- > 2 packets transmitted, 0 packets received, 100.0% packet loss > > > All networks have their own gateway > > 172.19.3.0/24 — > 172.19.3.1 (provisioning network) > 172.19.8.0/24 — > 172.19.8.1 (storage network) > 172.19.9.0/24 —> 172.19.9.1 (API network) > > 172.19.12.0/24 —> 172.19.12.1 (external + provider network) > 172.19.13.0/24 — > 172.19.13.1 (provider network) > 172.19.14.0/24 — > 172.19.14.1 (provider network) > 172.19.15.0/24 — > 172.19.15.1 (provider network) > > > In customer’s environment all networks are VLAN tagged. So traffic coming to > server is NIC based, which means (em1 + em2) configured for provisioning > network is expected to handle 3.x traffic, which means we would need to do > source routing so that it goes out of same nic, so that switch handles it > without dropping which happens incase if its going through default gw of > 172.19.12.1 (How can I route network traffic such that the packets go out > via the same interface they came in? > https://access.redhat.com/solutions/19596) > > 1. Can we do this via heat templates, if yes, how can this be achieved. > 2. If no, should we configure the source routing in the controller/compute > nodes manually, how will an update to the images affect the configuration in > the future? This behavior is because the Controllers do not have the provisioning network as their default gateway. By default, RHEL enables reverse path filtering, which will filter out packets that came in from a different interface where the packets would be going out using natural routing rules. There is a way to turn off this behavior, which will result in the node responding out the same interface where the traffic was received. The customer may disable rp_filter, but should be aware of the security implications (although it doesn't sound like they are concerned about this access, and in fact desire it). Follow this guide to turn off rp_filter: https://access.redhat.com/solutions/53031 Note that you will need to do this whenever a controller is rebuilt.
This bug did not make the OSP 8.0 release. It is being deferred to OSP 10.