Bug 1506240

Not a RFE.

What is your use case?
Why didn't you want to do this?

Hi Yaniv,
I and myakove talked with ykaul about it and he said that we raise an error by design to reduce the option to create problems by the end user and to make our hosted engine more stable.
but we asked to be able to do it when we know what we are doing, to allow us to do it and not block it in the deploy phase.

You didn't reply on the question.
What is your use case? Please provide this.

In our case we had a problem to deploy hosted engine VM with the allocated IP:

  fqdn: hosted-engine-02.scl.lab.tlv.redhat.com
  MAC: 00:1a:4a:16:27:30
  IP: 10.35.163.168

because the host that guest this hosted engine VM was:
  puma24.scl.lab.tlv.redhat.com(10.35.160.57)

which is not on the same subnet(24)

Then we got the following error:
[ ERROR ] The Engine VM (10.35.163.168/24) and this host (10.35.160.57/24)
will not be on the same IP subnet. Static routing configuration is not
supported by automatic VM configuration.

Please check our documentation, both ha-hosts required to lay within the same subnet in order to perform as ha-cluster. Migrating SHE-VM from one subnet to another is not supported for both, DHCP or Static configurations. 
Please provide your input.

(In reply to Kobi Hakimi from comment #5)
> In our case we had a problem to deploy hosted engine VM with the allocated
> IP:
> 
>   fqdn: hosted-engine-02.scl.lab.tlv.redhat.com
>   MAC: 00:1a:4a:16:27:30
>   IP: 10.35.163.168
> 
> because the host that guest this hosted engine VM was:
>   puma24.scl.lab.tlv.redhat.com(10.35.160.57)
> 
> which is not on the same subnet(24)
> 
> Then we got the following error:
> [ ERROR ] The Engine VM (10.35.163.168/24) and this host (10.35.160.57/24)
> will not be on the same IP subnet. Static routing configuration is not
> supported by automatic VM configuration.

This is not a answer to why you did this. Only the description of what you did.
Why did you want to deploy like this? Why not allocate 10.35.160.x ip to the HE guest?

(In reply to Yaniv Lavi from comment #7)
> (In reply to Kobi Hakimi from comment #5)
> > In our case we had a problem to deploy hosted engine VM with the allocated
> > IP:
> > 
> >   fqdn: hosted-engine-02.scl.lab.tlv.redhat.com
> >   MAC: 00:1a:4a:16:27:30
> >   IP: 10.35.163.168
> > 
> > because the host that guest this hosted engine VM was:
> >   puma24.scl.lab.tlv.redhat.com(10.35.160.57)
> > 
> > which is not on the same subnet(24)
> > 
> > Then we got the following error:
> > [ ERROR ] The Engine VM (10.35.163.168/24) and this host (10.35.160.57/24)
> > will not be on the same IP subnet. Static routing configuration is not
> > supported by automatic VM configuration.
> 
> This is not a answer to why you did this. Only the description of what you
> did.
> Why did you want to deploy like this? Why not allocate 10.35.160.x ip to the
> HE guest?


The answer to "why" is in the description...
because I already had ALLOCATED IP.
why reallocate IP? why not use the existing one?

(In reply to Kobi Hakimi from comment #8)
> (In reply to Yaniv Lavi from comment #7)
> > (In reply to Kobi Hakimi from comment #5)
> > > In our case we had a problem to deploy hosted engine VM with the allocated
> > > IP:
> > > 
> > >   fqdn: hosted-engine-02.scl.lab.tlv.redhat.com
> > >   MAC: 00:1a:4a:16:27:30
> > >   IP: 10.35.163.168
> > > 
> > > because the host that guest this hosted engine VM was:
> > >   puma24.scl.lab.tlv.redhat.com(10.35.160.57)
> > > 
> > > which is not on the same subnet(24)
> > > 
> > > Then we got the following error:
> > > [ ERROR ] The Engine VM (10.35.163.168/24) and this host (10.35.160.57/24)
> > > will not be on the same IP subnet. Static routing configuration is not
> > > supported by automatic VM configuration.
> > 
> > This is not a answer to why you did this. Only the description of what you
> > did.
> > Why did you want to deploy like this? Why not allocate 10.35.160.x ip to the
> > HE guest?
> 
> 
> The answer to "why" is in the description...
> because I already had ALLOCATED IP.
> why reallocate IP? why not use the existing one?

I still can't get on how this scenario is legit?
How any additional ha-hosts that lays within 10.35.160.x are expected to get to 10.35.163.168/24? 
You'll have to create static routes on all hosts to be able to get to HE-VM.
What's youre use case?

Normal network design doesn't expected to run more than one subnet within single VLAN, it will technically work, but this will make the whole concept of VLAN separation worthless. VLANs meant to break broadcast domains and for better traffic management and security isolation.

Yaniv, any input from your side?

(In reply to Nikolai Sednev from comment #9)
> (In reply to Kobi Hakimi from comment #8)
> > (In reply to Yaniv Lavi from comment #7)
> > > (In reply to Kobi Hakimi from comment #5)
> > > > In our case we had a problem to deploy hosted engine VM with the allocated
> > > > IP:
> > > > 
> > > >   fqdn: hosted-engine-02.scl.lab.tlv.redhat.com
> > > >   MAC: 00:1a:4a:16:27:30
> > > >   IP: 10.35.163.168
> > > > 
> > > > because the host that guest this hosted engine VM was:
> > > >   puma24.scl.lab.tlv.redhat.com(10.35.160.57)
> > > > 
> > > > which is not on the same subnet(24)
> > > > 
> > > > Then we got the following error:
> > > > [ ERROR ] The Engine VM (10.35.163.168/24) and this host (10.35.160.57/24)
> > > > will not be on the same IP subnet. Static routing configuration is not
> > > > supported by automatic VM configuration.
> > > 
> > > This is not a answer to why you did this. Only the description of what you
> > > did.
> > > Why did you want to deploy like this? Why not allocate 10.35.160.x ip to the
> > > HE guest?
> > 
> > 
> > The answer to "why" is in the description...
> > because I already had ALLOCATED IP.
> > why reallocate IP? why not use the existing one?
> 
> I still can't get on how this scenario is legit?
> How any additional ha-hosts that lays within 10.35.160.x are expected to get
> to 10.35.163.168/24? 
> You'll have to create static routes on all hosts to be able to get to HE-VM.
> What's youre use case?
> 
> Normal network design doesn't expected to run more than one subnet within
> single VLAN, it will technically work, but this will make the whole concept
> of VLAN separation worthless. VLANs meant to break broadcast domains and for
> better traffic management and security isolation.
> 
> Yaniv, any input from your side?

I wrote the reason in the expected result:
"""
Expected results:
Maybe we can convert the error into a warning or let the user continue.
In our case the same IP work as expected when we do it with DHCP.
"""

I want to let the user/admin the option to continue in case he knows what he doing and wants the HE VM not in the same VLAN.

Returning back the need info to Yaniv.

This configuration is not legit and will lead in to plenty issues:
1.In order to have such corner case covered, you'll have to have persistent static routes on all ha-hosts and on the engine, so they'll be able to communicated to each other.
2.The network design is about to have one subnet inside one VLAN, this is the way that any normal network topology is designed, in this scenario we have 2 different network subnets within the same VLAN, which is against any normal network topology design and will cause performance issues and suffer from security flows and QOS problems.
3.Static routes won't be permanent as VDSM will be managing them and so after restarts on hosts, they'll loose network connectivity to the engine as their routes will be wiped out.
4.Engine's migrations will be processed through the router, this will add more latency, will add additional point of failure, which can be inaccessible due to network issues or outages and will also push customers to enhance their networks to have first hop redundancy, making their network more expensive. 

I've opened https://bugzilla.redhat.com/show_bug.cgi?id=1616158 to cover the issue for DHCP flow.

IMHO this bug should be closed as not a bug, as I don't see how it can contribute to normal network and system performance and configuration of HE.

I see this as edge case and the HE VM ip can be re-allocated easily.
We didn't get any strong case to add such support.
Therefore closing.