Bug 1124598

Summary:	Default gateway conflicts between provisioning network and external network with DHCP
Product:	Red Hat OpenStack	Reporter:	Lars Kellogg-Stedman <lars>
Component:	rhel-osp-installer	Assignee:	John Eckersberg <jeckersb>
Status:	CLOSED ERRATA	QA Contact:	Omri Hochman <ohochman>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	Foreman (RHEL 6)	CC:	acathrow, hbrock, kambiz, mburns, mhulan, rhos-maint, sasha, sclewis, yeylon
Target Milestone:	ga
Target Release:	Installer
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	rhel-osp-installer-0.1.6-6.el6ost	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-08-21 18:06:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Lars Kellogg-Stedman 2014-07-29 21:33:32 UTC

We are using DHCP on the provisioning network, with the default gateway going through the foreman server.

If someone is using DHCP on the network they use for external connectivity, then (a) this DHCP server will also be providing default gateway information and (b) they probably need to use that gateway in order for external connectivity to function.

Currently, whichever interface updates most recently will set the default gateway, leading to inconsisent and broken behavior.

Comment 1 Hugh Brock 2014-07-29 21:44:18 UTC

Changed my mind, I think this is best addressed in the installer or in quickstack.

Comment 2 John Eckersberg 2014-07-30 22:13:12 UTC

https://github.com/theforeman/foreman-installer-staypuft/pull/59

Comment 4 Lars Kellogg-Stedman 2014-07-31 13:52:28 UTC

The controller needs to use the default gateway from the external network.

Using the foreman server as the default gateway would cause any connections from outside that external network to fail, because the response would route through the foreman server and hit the outbound NAT rule.

Comment 6 Lars Kellogg-Stedman 2014-07-31 15:23:30 UTC

I am proposing this as a fix:

https://github.com/larsks/foreman-installer-staypuft/commit/29d964000124075ee4b66832718f98f957f3aa45

...but I have not tested it yet.  I'm waiting on a beaker host.

Comment 7 Lars Kellogg-Stedman 2014-07-31 20:05:29 UTC

This is now an actual PR:

https://github.com/theforeman/foreman-installer-staypuft/pull/60

Comment 8 Lars Kellogg-Stedman 2014-08-01 01:45:43 UTC

With this change in place, the installer seems to set up the correct routing on the controller.  I am able to access horizon and other api services from outside the local network.

Comment 11 Omri Hochman 2014-08-07 10:20:19 UTC

This fix is not included in the latest puddle yet,  so we cannot switch the bug from ON_QA to VERIFIED ,  

I'm moving it back to back ASSIGNED .

Comment 13 Kambiz Aghaiepour 2014-08-13 12:18:21 UTC

Why not use policy routing as described for example here:

http://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/

to ensure traffic to and from the provisioning interface uses the provisioning interface, and likewise the external interface.  The default route then no longer matters, whether it gets set to the provisioning interface, or the external interface.  Removing the default route on the provisioning interface (by adding DEFROUTE=no to ifcfg-<PROVISIONING INTERFACE>) forces a DHCP or static interface configuration on the external interface.  I have deployed using no IP address on the external interface, and have left default route on the provisioning interface, and my API endpoints all work, as well as external connectivity over the external interface (basically layer 2 access is given by way of br-ex which contains the external interface as a port).

Thoughts?

Comment 14 Lars Kellogg-Stedman 2014-08-13 14:45:11 UTC

Keep in mind that all of these shenanigans are only because we don't have network management in staypuft.  Because of that, we *are* requiring DHCP on all networks.

This will change in the near future.

If you are able to access your API endpoints using a target address that is not on your provisioning network -- from somewhere other than a device directly attached to your provisioning network -- then you may just have very lucky routing.  I would expect that to fail in many environments (because the default gateway on the provisioning network would drop packets with L3 origins that do not match the local network).

Comment 15 Alexander Chuzhoy 2014-08-18 18:45:11 UTC

Verified: with rhel-osp-installer-0.1.10-2.el6ost.noarch

openstack-foreman-installer-2.0.21-1.el6ost.noarch
ruby193-rubygem-foreman_openstack_simplify-0.0.6-8.el6ost.noarch
openstack-puppet-modules-2014.1-20.2.el6ost.noarch

There's only one DGW and it's via the externally connected NIC.

Comment 16 errata-xmlrpc 2014-08-21 18:06:57 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1090.html