Bug 1301879 - [RFE] - Support setting pre-existing static routes for complex networking
[RFE] - Support setting pre-existing static routes for complex networking
Status: CLOSED DUPLICATE of bug 1311590
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-host-deploy (Show other bugs)
3.6.0
Unspecified Unspecified
low Severity high
: ovirt-4.0.0-alpha
: ---
Assigned To: Dan Kenigsberg
Meni Yakove
: FutureFeature, Reopened
Depends On: 1338732 RHEV_36_HTB RHV_4.1_Beta_Tracker
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-26 04:23 EST by Ade Bradshaw
Modified: 2017-02-06 18:11 EST (History)
16 users (show)

See Also:
Fixed In Version:
Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-02 04:17:48 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Network
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Ade Bradshaw 2016-01-26 04:23:54 EST
Description of problem:
When setting up a new hypervisor, existing static routes file is ignored during the creation of the ovirt bridge, resulting in loss of network connectivity


Version-Release number of selected component (if applicable):
3.6 beta (25 Jan)

How reproducible:
100%


Steps to Reproduce:
1. Create a set of static routes in a file in /etc/sysconfig/network-scripts
2. Add server to RHEV
3. Check if static routes are still there

Actual results:
New static routes file for ovirt bridge is created but doesnt contain the static routes from the original file

Expected results:
The existing routes would be included in the new file

Additional info:
Comment 1 Dan Kenigsberg 2016-02-03 04:22:09 EST
I am afraid we do not support manual configuration of networking on top of the ones natively implemented by oVirt.

However, you have two workarounds:

1. recreate your static routes in a hook placed under /usr/libexec/vdsm/hooks/after_network_setup.

2. create the ovirtmgmt network yourself prior to adding the host. this requires creating the ovirtmgmt bridge AND defining a libvirt network named vdsm-ovirtmgmt

<network>
  <name>vdsm-ovirtmgmt</name>
  <uuid>34127d8a-0f80-4888-839a-bbfcf12339dc</uuid>
  <forward mode='bridge'/>
  <bridge name='ovirtmgmt'/>
</network>

you should make sure that an automatic reconfiguration of ovirtmgmt never happens (it would overwrite yours)
Comment 2 Martin Tessun 2016-02-04 04:20:42 EST
Hi Dan,

(In reply to Dan Kenigsberg from comment #1)
> I am afraid we do not support manual configuration of networking on top of
> the ones natively implemented by oVirt.

that is not the case here. The routes are existing *before* the host is added to the hypervisor.
As the customer only has static routes and no default gateway in his Admin LAN, the result is that while adding the host as a hypervisor its connectivity is lost.
So we should take care about this during "encapsulating" the ovirtmgmt/rhevm device to a bridge, as the existing device might contain static routes.

> 
> However, you have two workarounds:
> 
> 1. recreate your static routes in a hook placed under
> /usr/libexec/vdsm/hooks/after_network_setup.

Unfortunately that doesn't work during adding the host to the HV, as these files are not necessarily there (packages are not yet installed and get installed during adding the host).

> 
> 2. create the ovirtmgmt network yourself prior to adding the host. this
> requires creating the ovirtmgmt bridge AND defining a libvirt network named
> vdsm-ovirtmgmt

To be honest that is no option for any customer and would also be a nightmare to support, if not done correctly.

> 
> <network>
>   <name>vdsm-ovirtmgmt</name>
>   <uuid>34127d8a-0f80-4888-839a-bbfcf12339dc</uuid>
>   <forward mode='bridge'/>
>   <bridge name='ovirtmgmt'/>
> </network>
> 
> you should make sure that an automatic reconfiguration of ovirtmgmt never
> happens (it would overwrite yours)

This as well is a nigtmare for support. So this as well is no option at all.
Reopening this bug, as this case should be handled during setup (be it that an Error is given back, telling that static routes are not supported).

Cheers,
Martin
Comment 3 Dan Kenigsberg 2016-02-04 11:21:50 EST
Martin, there are a lot of network preconfiguration that ovirt cannot work with. We cannot detect and warn about them all.

Actually, the list of stuff that we do support is very limited. It includes untouched, freshly-installed RHEL host, most preferably with NetworkManager turned off.

Maybe you can share your specific static route (your ifcfg* rule* and route* files); there could be a chance that we can find a workaround for your special case.

However, ovirt controls static routes by itself, in a very strict fashion (see http://www.ovirt.org/Features/Multiple_Gateways ). Anything else is usupported and can only be hacked with hooks.
Comment 4 Martin Tessun 2016-02-04 11:35:37 EST
Hi Dan,

I fully understand this. But in that case, I would expect either the installer to fail with some messages (static routes detected which are not expected or whatever message) and not "encapsulate" the interface with the static routes, dropping the routes silently, thus leaving the system unreachable.

As this behaviour is also not mentioned in the Installation Guide, I think that this could be added at least, if a controlling mechanism in setting up the host is not easy to implement.

So the only thing that really is "not nice" here is that the routes get silently dropped, and thus network connectivity gets also lost.

Having static routes configurable in RHEV would be an RFE then, but I would expect that the setup does not break connectivity and rather aborts before doing those actions.

Cheers,
Martin
Comment 5 Yaniv Kaul 2016-02-04 12:41:13 EST
(In reply to Martin Tessun from comment #4)
> Hi Dan,
> 
> I fully understand this. But in that case, I would expect either the
> installer to fail with some messages (static routes detected which are not
> expected or whatever message) and not "encapsulate" the interface with the
> static routes, dropping the routes silently, thus leaving the system
> unreachable.

We can't warn on everything we don't support, though in this specific case I agree it's very problematic. Should we abort host installation in this case?

> 
> As this behaviour is also not mentioned in the Installation Guide, I think
> that this could be added at least, if a controlling mechanism in setting up
> the host is not easy to implement.

That's a different bug.

> 
> So the only thing that really is "not nice" here is that the routes get
> silently dropped, and thus network connectivity gets also lost.
> 
> Having static routes configurable in RHEV would be an RFE then, but I would
> expect that the setup does not break connectivity and rather aborts before
> doing those actions.
> 
> Cheers,
> Martin


Why is this bug on network bug assigned to Sandro?
Comment 6 Sandro Bonazzola 2016-02-05 02:16:38 EST
(In reply to Yaniv Kaul from comment #5)
 
> Why is this bug on network bug assigned to Sandro?

It has been opened on ovirt-setup-lib, so I'm default assignee there.
It is wrong component for this bug, moving to ovirt-host-deploy.
Being this handled by Dan, assigning the bug to him.
Comment 7 Dan Kenigsberg 2016-02-07 07:58:17 EST
can share your specific static route (your ifcfg* rule* and route* files)
Comment 9 Martin Tessun 2016-02-08 08:10:21 EST
(In reply to Yaniv Kaul from comment #5)
> (In reply to Martin Tessun from comment #4)
> > Hi Dan,
> > 
> > I fully understand this. But in that case, I would expect either the
> > installer to fail with some messages (static routes detected which are not
> > expected or whatever message) and not "encapsulate" the interface with the
> > static routes, dropping the routes silently, thus leaving the system
> > unreachable.
> 
> We can't warn on everything we don't support, though in this specific case I
> agree it's very problematic. Should we abort host installation in this case?
> 

Either that, or putting some warning in that static routes will get lost during installation as these have been detected, so that the customer could bail out, if needed and fix the network config first.

> > 
> > As this behaviour is also not mentioned in the Installation Guide, I think
> > that this could be added at least, if a controlling mechanism in setting up
> > the host is not easy to implement.
> 
> That's a different bug.

Sure. I will happily open this one, as soon as we agree on the path to take here.

Cheers,
Martin
Comment 10 Yaniv Lavi 2016-02-10 09:10:59 EST
What is the use case? what are you trying to achieve with the static routes?
Comment 11 Martin Tessun 2016-02-10 11:34:49 EST
Hi Yaniv,

it is just the customers setup. Having only the networks needed routed to their "Admin LAN". As the Hypervisors Management Network is attached to the Admin LAN, the routes are setup there.

There are lots of customer setups out there that have even much more complex routings within their environment.

So the Usecase is: Customer Network Setup and Policies.

Cheers,
Martin
Comment 12 Dan Kenigsberg 2016-02-11 02:05:36 EST
I think that Yaniv is asking more about the network topology at the customer(s). For example, we've implemented bug 618636 after we had multiple customer requests with similar routing problems.
Comment 13 Ade Bradshaw 2016-02-11 02:22:54 EST
Hi all

The issue here is that the customer does not use a default gateway at all. Instead, static routes are added to the eno49 interface (the only interface configured).  I have seen this setup (no default gateway) a few times before, its not unusual.

Without the static routes, traffic from eno49 will not route back the the admin network. So when the routes files is effectively removed during the setup, we lose connectivity

Could we not say, "we are using eno49 for ovirt, therefore add the routes file for eno49 to the ovirt routes file". It seems to me as though that logic would be quite simple to do 

Thanks for your help, Adrian
Comment 15 Dan Kenigsberg 2016-03-02 04:15:01 EST
We can solve this RFE by making extra static route a first-class citizen in the Vdsm and Engine APIs, as requested by bug 1311590
Comment 16 Yaniv Lavi 2016-03-02 04:17:48 EST

*** This bug has been marked as a duplicate of bug 1311590 ***

Note You need to log in before you can comment on or make changes to this bug.