The current path we are working on is first enabling it using hooks.

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Need to discuss scale testing with scale team with VM running on network and testing communication within the networks. Need to check parity with existing implementation. 
Need to consider upgrade path for in cluster network switch to OVS. Dan, do we have thoughts\design for this yet?

What is the actual reason behind this move? Which features does the Linux networking stack not provide without ovs?

(In reply to Hannes Frederic Sowa from comment #4)
> What is the actual reason behind this move? Which features does the Linux
> networking stack not provide without ovs?

- SDN features (future - OVN)
- SDN features (future - integration with 3rd party solutions)
- Scalability - the time to set up large (100's?) of networks is supposed to be significantly less in OVS. We have bugs around 60 or so networks.
- Broadcast domain?
- API (Python and CLI) - not sure about bridge.
- Statistics - not sure about bridge.
- DPDK integration - not sure about bridge.
- Other advanced features? Routing and friends? Easier with a single interface / component (Bond come to mind as well).

(In reply to Yaniv Kaul from comment #5)
> (In reply to Hannes Frederic Sowa from comment #4)
> > What is the actual reason behind this move? Which features does the Linux
> > networking stack not provide without ovs?
> 
> - SDN features (future - OVN)

I can understand. So RHEV should compete with OpenStack?

Please understand that we nowadays seldom ship an OpenStack installation without an external partner providing the network (nuage, plumgrid, etc.). We are currently missing critical features in OpenStack still (DVR routing). As a kernel developer the current OpenStack use basically forces a lot of packets on slow paths.

> - SDN features (future - integration with 3rd party solutions)

Same arguments as above. This probably will have to be decided by Product Management, but I don't really see the reason.

> - Scalability - the time to set up large (100's?) of networks is supposed to
> be significantly less in OVS. We have bugs around 60 or so networks.

It would be great if we get those bugs, as we are responsible for kernel networking.

> - Broadcast domain?

Huge problem for overlay networks, as the packet needs to be cloned in the kernel and send out to *every* hypervisor.

> - API (Python and CLI) - not sure about bridge.

Yes, the API is much more complicated.

> - Statistics - not sure about bridge.

We certainly can solve that.

> - DPDK integration - not sure about bridge.

I thought this is something we only want to do for Containers. Why the change here?

> - Other advanced features? Routing and friends? Easier with a single
> interface / component (Bond come to mind as well).

Nothing comes to my mind either.

(In reply to Hannes Frederic Sowa from comment #6)
> > - DPDK integration - not sure about bridge.
> 
> I thought this is something we only want to do for Containers. Why the
> change here?

Mine statement is wrong, sorry.

Sorry, we do use it for Virtualization, but everyone always thought as an intermediate solution until the Kernel is up to the task with that, which requires much more effort. DPDK should not be an end-goal.

(In reply to Hannes Frederic Sowa from comment #6)
> (In reply to Yaniv Kaul from comment #5)
> > (In reply to Hannes Frederic Sowa from comment #4)
> > > What is the actual reason behind this move? Which features does the Linux
> > > networking stack not provide without ovs?
> > 
> > - SDN features (future - OVN)
> 
> I can understand. So RHEV should compete with OpenStack?

No, but we also want a bit of SDN. Even in traditional data center virtualization solution you want SDN, for cross data center communication, for example, for private networks, etc.

> 
> Please understand that we nowadays seldom ship an OpenStack installation
> without an external partner providing the network (nuage, plumgrid, etc.).
> We are currently missing critical features in OpenStack still (DVR routing).
> As a kernel developer the current OpenStack use basically forces a lot of
> packets on slow paths.
> 
> > - SDN features (future - integration with 3rd party solutions)
> 
> Same arguments as above. This probably will have to be decided by Product
> Management, but I don't really see the reason.

Same reason as OpenStack + some customers already have that 3rd party (Nuage and friends) and want to use it with RHEV as well.

> 
> > - Scalability - the time to set up large (100's?) of networks is supposed to
> > be significantly less in OVS. We have bugs around 60 or so networks.
> 
> It would be great if we get those bugs, as we are responsible for kernel
> networking.

I'll look for it.

> 
> > - Broadcast domain?
> 
> Huge problem for overlay networks, as the packet needs to be cloned in the
> kernel and send out to *every* hypervisor.
> 
> > - API (Python and CLI) - not sure about bridge.
> 
> Yes, the API is much more complicated.
> 
> > - Statistics - not sure about bridge.
> 
> We certainly can solve that.
> 
> > - DPDK integration - not sure about bridge.
> 
> I thought this is something we only want to do for Containers. Why the
> change here?

Why? If it provides better performance / lower latency, why not use it with virtualization as well (vhost with DPDK, or something)?

> 
> > - Other advanced features? Routing and friends? Easier with a single
> > interface / component (Bond come to mind as well).
> 
> Nothing comes to my mind either.

Integration with other networking gear, via OpenFlow? Standard monitoring via sFlow?

(In reply to Yaniv Kaul from comment #8)
> (In reply to Hannes Frederic Sowa from comment #6)
> > (In reply to Yaniv Kaul from comment #5)
> > > (In reply to Hannes Frederic Sowa from comment #4)
> > > > What is the actual reason behind this move? Which features does the Linux
> > > > networking stack not provide without ovs?
> > > 
> > > - SDN features (future - OVN)
> > 
> > I can understand. So RHEV should compete with OpenStack?
> 
> No, but we also want a bit of SDN. Even in traditional data center
> virtualization solution you want SDN, for cross data center communication,
> for example, for private networks, etc.

I do think those are pretty easily realizable without SDN, but I don't want to make this decision. If feature requests came in for that.

Private networks should already be supported by vlans etc. I guess the only reason why you want to have overlay is, that you want to reuse the same ip address space again, no?

> > Please understand that we nowadays seldom ship an OpenStack installation
> > without an external partner providing the network (nuage, plumgrid, etc.).
> > We are currently missing critical features in OpenStack still (DVR routing).
> > As a kernel developer the current OpenStack use basically forces a lot of
> > packets on slow paths.
> > 
> > > - SDN features (future - integration with 3rd party solutions)
> > 
> > Same arguments as above. This probably will have to be decided by Product
> > Management, but I don't really see the reason.
> 
> Same reason as OpenStack + some customers already have that 3rd party (Nuage
> and friends) and want to use it with RHEV as well.

Okay, I can't argue here but would like to see a datacenter architecture were that should work.

> > > - Scalability - the time to set up large (100's?) of networks is supposed to
> > > be significantly less in OVS. We have bugs around 60 or so networks.
> > 
> > It would be great if we get those bugs, as we are responsible for kernel
> > networking.
> 
> I'll look for it.

Thank you, that would be really helpful!

> > I thought this is something we only want to do for Containers. Why the
> > change here?
> 
> Why? If it provides better performance / lower latency, why not use it with
> virtualization as well (vhost with DPDK, or something)?

dpdk comes at a huge cost, keeping cores busy looping to get new packets. This isn't very efficient at all.

The other side is licensing, we at least try to prefer GPLv2 in-kernel solutions over dpdk ones. That is e.g. why we do have the dp-accel project to bring as much speed-ups to the kernel as possible.

> > > - Other advanced features? Routing and friends? Easier with a single
> > > interface / component (Bond come to mind as well).
> > 
> > Nothing comes to my mind either.
> 
> Integration with other networking gear, via OpenFlow? Standard monitoring
> via sFlow?

sflow can already be integrated. I am not sure if integration with OpenFlow is something RHEV should provide.

We are not competing vs OSP. OSP has it's main use case for mode 2 application and oVirt centers around traditional mode 1 workloads. Fast and scale-able networking provision is not needed only for OSP use, but can improve usability for mode 1 as well.

We do want to work on a integration with next generation technology that has a good eco system and will allow us to increase scale limitation. It not a matter of a single feature. We see this in OSV and looking to future in OVN. 

I do want to stress we are discussing VM networks and not host networks and that we have private code the handle ifcfg that we would prefer to not maintain for VM networks. Easily done is not easily maintainable and we don't want to do our own thing, if there is a good alternative. We can make the existing code more scale-able, but still leave it to us to maintain it and we want to defer effort to future investments with more feature velocity.

I'm quite sure that Hannes has concerns, we might want to sit down with him to hear them and get his (or the platform network team's input) on this RFE.

Yes, I am happy to assist. Let us and our team know how and where to help.

In the URL, I don't see a single mention of OpenFlow controller. I hope you do realize that when using Open vSwitch, you need to implement (and act as) OpenFlow controller? Open vSwitch is a vSwitch implementation and it assumes it connects to a controller. It cannot operate reliably without one.

This is a big pain in Neutron where they are using very hacky way to configure Open vSwitch without a controller (maybe they implement OpenFlow controller nowadays, though, I don't keep track). It falls down on events like ovs-vswitchd daemon restart and similar.

This document describes implementation of basic OVS functionality. After this step it should behave the same way as our current native linux networking implementation (but faster because of we are not using initscripts). OVS boundaries are in this case limited for a host. Is a controller needed in this case?

This transformation should prepare ground for OVN and the real OVS utilization.

If it's only about avoiding init scripts, the NetworkManager's API should have also been something to investigate.

(In reply to Fabian Deutsch from comment #15)
> If it's only about avoiding init scripts, the NetworkManager's API should
> have also been something to investigate.

See comment #10. It's not just scale, it opens the door to SDN.

Yaniv,

I was starting to set up a mail to ovirt-devel, but felt it kind of hard to write, because I don't know anything about the project and its governance.

I try to bring up some points here, also about the discussion about the road map and we can discuss technical things upstream.

How is oVirt governed? Does RH have a leading role on this project (I assume the answer is yes) and how much does community talk into that.

I ask because if the RoadMap says SDN, probably OVS is your only choice. If this will get you compatibility with OSP, I doubt that just because of complexity reasons. But then I do not have any more arguments. Probably the requests came from customers then to integrate RHEV into OSP or vice versa? I find this kind of confusion, as the reason to use OSP is to have multi-tenancy and the only advantage over a VLAN separation etc. is that you can much more easily reuse the same network ip addresses within the same management domain.

If SDN is just a side-effect of this, I would really think about what RHEV is and which customers to attract with that. OVS comes with a lot of additional complexity which still causes a lot of trouble. I do have the feeling that no OpenStack cloud does get completely set up by RH but mostly other partners do the network and replace the network configuration with their own plugins. Overlays do have a lot of additional complexity, lot's of things need to be implemented with netfilter, namespaces etc. anyway which increases complexity a lot.

I unfortunately don't understand the arguments with type 1 or type 2 workloads, I don't see how this plays a role. The number of VMs on a system is basically limited by the amount of RAM we currently can put into the boxes. And the traditional Linux networking model is most of the time easily capable to support this. Otherwise please open bugs, we can certainly help there.

I don't know how RHEV deals with storage, but this indeed looks to me as RHEV would like to go more and more into direction of OSP.

(In reply to Petr Horáček from comment #14)
> This document describes implementation of basic OVS functionality. After
> this step it should behave the same way as our current native linux
> networking implementation (but faster because of we are not using
> initscripts). OVS boundaries are in this case limited for a host. Is a
> controller needed in this case?

If ovs is used just as a MAC learning switch then no controller is needed. But in such case it provides absolutely no advantage over the Linux bridge. If the only reason is automatic bridge setup, then please talk with NetworkManager developers, I'm sure they can help.

In any other ovs use case than a MAC learning switch, a controller is needed.

(In reply to Jiri Benc from comment #18)
> (In reply to Petr Horáček from comment #14)
> > This document describes implementation of basic OVS functionality. After
> > this step it should behave the same way as our current native linux
> > networking implementation (but faster because of we are not using
> > initscripts). OVS boundaries are in this case limited for a host. Is a
> > controller needed in this case?
> 
> If ovs is used just as a MAC learning switch then no controller is needed.
> But in such case it provides absolutely no advantage over the Linux bridge.
> If the only reason is automatic bridge setup, then please talk with
> NetworkManager developers, I'm sure they can help.
> 
> In any other ovs use case than a MAC learning switch, a controller is needed.

Please read comment #10.

Hanness - please see bug 1193083 - one of the reasons we wish to switch from current networking stack to OVS.

Moving from 4.0 alpha to 4.0 beta since 4.0 alpha has been already released and bug is not ON_QA.

Moving out as this is a ongoing dev\QE process for tech preview.

Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Note: the link in URL field doesn't work.

How come that this RFE is ON_QA? when it depends on bugs that on NEW/POST and so on..? 
I don't think this should be ON_QA.

Should this move to 4.2 based on Petr's work?

(In reply to Yaniv Lavi from comment #32)
> Should this move to 4.2 based on Petr's work?

I don't think so, as this bug has been deeply abused for OvS.

To support OVS switch type we need to document what is not supported (yet) on it.
Partial list is:

- reading LLDP (RHV-2679)
- setting DNS
- source routing (bug 1383035)
- iSCSI bond (bug 1441245)
- port mirroring (bug 1362492)
- host QoS (bug 1380271)
- setting bridge options (bug 1380273)

We are not even close to be finished with this bug.

We didn't get to this bug for more than 2 years, and it's not being considered for the upcoming 4.4. It's unlikely that it will ever be addressed so I'm suggesting to close it.
If you feel this needs to be addressed and want to work on it please remove cond nack and target accordingly.

Closing as deferred, because we plan to keep OVS cluster type as tech preview.