We are using DHCP on the provisioning network, with the default gateway going through the foreman server. If someone is using DHCP on the network they use for external connectivity, then (a) this DHCP server will also be providing default gateway information and (b) they probably need to use that gateway in order for external connectivity to function. Currently, whichever interface updates most recently will set the default gateway, leading to inconsisent and broken behavior.
Changed my mind, I think this is best addressed in the installer or in quickstack.
https://github.com/theforeman/foreman-installer-staypuft/pull/59
The controller needs to use the default gateway from the external network. Using the foreman server as the default gateway would cause any connections from outside that external network to fail, because the response would route through the foreman server and hit the outbound NAT rule.
I am proposing this as a fix: https://github.com/larsks/foreman-installer-staypuft/commit/29d964000124075ee4b66832718f98f957f3aa45 ...but I have not tested it yet. I'm waiting on a beaker host.
This is now an actual PR: https://github.com/theforeman/foreman-installer-staypuft/pull/60
With this change in place, the installer seems to set up the correct routing on the controller. I am able to access horizon and other api services from outside the local network.
This fix is not included in the latest puddle yet, so we cannot switch the bug from ON_QA to VERIFIED , I'm moving it back to back ASSIGNED .
Why not use policy routing as described for example here: http://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/ to ensure traffic to and from the provisioning interface uses the provisioning interface, and likewise the external interface. The default route then no longer matters, whether it gets set to the provisioning interface, or the external interface. Removing the default route on the provisioning interface (by adding DEFROUTE=no to ifcfg-<PROVISIONING INTERFACE>) forces a DHCP or static interface configuration on the external interface. I have deployed using no IP address on the external interface, and have left default route on the provisioning interface, and my API endpoints all work, as well as external connectivity over the external interface (basically layer 2 access is given by way of br-ex which contains the external interface as a port). Thoughts?
Keep in mind that all of these shenanigans are only because we don't have network management in staypuft. Because of that, we *are* requiring DHCP on all networks. This will change in the near future. If you are able to access your API endpoints using a target address that is not on your provisioning network -- from somewhere other than a device directly attached to your provisioning network -- then you may just have very lucky routing. I would expect that to fail in many environments (because the default gateway on the provisioning network would drop packets with L3 origins that do not match the local network).
Verified: with rhel-osp-installer-0.1.10-2.el6ost.noarch openstack-foreman-installer-2.0.21-1.el6ost.noarch ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el6ost.noarch openstack-puppet-modules-2014.1-20.2.el6ost.noarch There's only one DGW and it's via the externally connected NIC.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHBA-2014-1090.html