Bug 1608531

Jeya can you share a bit more detail on the ideal of how we should make this available. My thinking would be to have this configured in nova.conf like: 

  libvirt/vhostuser_reconnect=5

But that means that will be configured per host. Is there any requirement to have this configured by guests? I imagine no.

Also as you have indicated that should first be available upstream bug 1583976 and that looks to be a feature we will have to wait to have it accepted for the next release.

As the 'reconnect' argument is passed from nova-compute to the qemu, Having the configuration in nova.conf in the compute as suggested per host should be fine.

When i opened the first bug 1583976, it got assigned to OSP14 as it was considered as a new dev effort. But as our customers are using OSP13, we need this in OSP13 first.

this is not a bug. this is a misconfiguration.
qemus vhost user reconnect is a feature that was specifically develop to allow reestablishing the vhost user connect when the switching backen was reasterd.

it was designed by both the qemu and dpdk comunities to only work when qemu is the server and the vswitch/userpsce backend is the client. it is not supported by qemu when qemu is the client. introducing a nova config option to alter the creation of the vhost user interface would violate several precedents that have been set about not having nova.conf option that alter networking behaviour.

when i developed reconnect support for ovs dpdk in neutron (
https://git.openstack.org/cgit/openstack/neutron/commit/?id=ca60a91cbd3668c469313a3bd6c0c99aeff30a74) it was discussed and agreed that 
it would be the responsibility of the neutron ml2 driver to set the vhost user mode and pass and relevant options such as the vhost-user mode or socket to nova as part of the vif port bidnign details.

specifically when neutron detects that the host vrouter instance support reconnect it should set the correct vif_details here 
https://github.com/openstack/networking-opencontrail/blob/62b57683cae2210023a210f1df2be51dd50822e9/networking_opencontrail/drivers/contrail_driver_base.py#L342
as we do ovn,odl and the reference ovs ml2 drivers.

adding a nova config option would make this feature unupstream-able and would also likely conflict with the proposed work to enable vrouter hardware offload support in stein https://review.openstack.org/#/c/567148/. 

it is imporant from a customer point of view to convey that this is not expected to work and  a support for this functionality  does no exist for contrail vrouter with openstack today. any work to address it would be new development for OSP 15 that would have to be backport to osp 14 and 13.

Sean I think the work done in Neutron was just about to add support for vhu client mode when available in OVS capabilities.

I think the request here is about to add a timeout when vhu server is used (QEMU in client) so the socket is not closed. It's something QEMU is providing and libvirt exposing. I imagine we could add it for Nova and provide a backport so all "vswitch" that are using vhu server can take benefit of it. Also I don't think that is going to create any conflict with the in progress work to support hardware offloading with vrouter.

Is that reasonable for you?

firstly from a termanloagy point of view the vhost user mode is generally referenced from the perspcetive of qemu not the vsiwthc. as such vhu server without general contexts implies qemu is the server and dpdk is the client.


regarding other vwitch, vpp and snabb swithc i belvie support both client and server mode and also default to using qemu as the server. lagopus switch does not have support for openstack and i belive both the erricon and windriver dpdk vswitch support qemu as the vhost-user server. as such i dont think any other vswitch/vrouter would use this fuctionality. 


regarding the neuron  change no it was not jsut about enable this feature for ovs. when we implemented support for vhost-user in nova daniel berrange insited that the vhost-user socket mode and other parmater must not be stored in the nova confing and should be past form neutron to nova as part as port binding. as other feature has been added such as reconnect were added that stance was maintiend by several people on the nova core team. that specific change was for ovs but vpp already used server mode and used reconnect without openstack code change when the feature was added to dpdk and qemu.


as i said on the other thread i think having the ablity to generate the xml in nova is reasonable but not addting a nova config options to configure that behavior. we do not have the ability in osp 13 to regenerate the xml on a per node bases as part of livemigration so this config option should not be set on a per host basise. if the xml generation is added to nova then the neutron ml2 driver for contrail should alos be modifed to use this feature and we should extend the neutron port bindings extention to define the relevent port binding details key.

> firstly from a termanloagy point of view the vhost user mode is generally referenced from the perspcetive of qemu not the vsiwthc. as such vhu server without general contexts implies qemu is the server and dpdk is the client.

No sure of that since the first implementation was vswitch as server and QEMU as client.

> as i said on the other thread i think having the ablity to generate the xml in nova is reasonable but not addting a nova config options to configure that behavior. we do not have the ability in osp 13 to regenerate the xml on a per node bases as part of livemigration so this config option should not be set on a per host basise. if the xml generation is added to nova then the neutron ml2 driver for contrail should alos be modifed to use this feature and we should extend the neutron port bindings extention to define the relevent port binding details key.

If that is the preference I don't have any issue here just that does not look right to tweak the virtualization layer from Neutron. A per host configuration is basically what we do with tx/rx queue sizes for virtio devices but it is not ideal I'm agreed.

I don't think it's a concern for live-migration since we are expecting all the hosts to support the same capabilities. In a backportable point of view, have it as a configuration is probably the simplest way.

(In reply to Sahid Ferdjaoui from comment #6)
> > firstly from a termanloagy point of view the vhost user mode is generally referenced from the perspcetive of qemu not the vsiwthc. as such vhu server without general contexts implies qemu is the server and dpdk is the client.
> 
> No sure of that since the first implementation was vswitch as server and
> QEMU as client.

the fist implmentation was snabb switch not dpdk and snabb switch orginally did not support running as the server until https://github.com/snabbco/snabb/commit/17df9a4fae92082704950dfe35b9d39b8139496a#diff-b7e4af4ff42f4c76f3c78066bb2d5de3

dpdk's first implementation of vhsot-user was as the server and added client support after because they were porting there prexising vhost-cuse implementation which was based on an earlier poc that predates the creating of vhost-user in qemu called dpdk userspce vhost which was part of ovdk. dpdk started as the server because tehy were trying to emulate teh kernel vhost protocol and were forced to to avoid changing qemu which was part of there original design constraints.


> 
> > as i said on the other thread i think having the ablity to generate the xml in nova is reasonable but not addting a nova config options to configure that behavior. we do not have the ability in osp 13 to regenerate the xml on a per node bases as part of livemigration so this config option should not be set on a per host basise. if the xml generation is added to nova then the neutron ml2 driver for contrail should alos be modifed to use this feature and we should extend the neutron port bindings extention to define the relevent port binding details key.
> 
> If that is the preference I don't have any issue here just that does not
> look right to tweak the virtualization layer from Neutron. A per host
> configuration is basically what we do with tx/rx queue sizes for virtio
> devices but it is not ideal I'm agreed.
the tx/rx quese were originally intended to be set in the image so they could not be changed for the lifetime of the instance without a rebuild. they got move to the config eventually as part of the reivew which was a mistake in my view but its not some i think we should try to emulate.

> 
> I don't think it's a concern for live-migration since we are expecting all
> the hosts to support the same capabilities. 
 my consern was on upgrade but provide we did not change the config value as part of the upgrade it should be ok.
> In a backportable point of view,
> have it as a configuration is probably the simplest way.

As discussed in bug triage meeting i am closing this as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1583976.

as we have not determined the correct solution upstream for osp 14 we cannot asses the applicability for backport. i will add a note to https://bugzilla.redhat.com/show_bug.cgi?id=1583976 to indicate that it has been expressly requested for backport to osp-13 and we will use that as a single point to track the completion of this feature until it is completed in the current dev releases.

*** This bug has been marked as a duplicate of bug 1583976 ***