Bug 1832708 - [ovs-dpdk] [vhost-user] too many destroy_connection calls on restarting ovs-vswitchd without tso
Summary: [ovs-dpdk] [vhost-user] too many destroy_connection calls on restarting ovs-v...
Keywords:
Status: CLOSED DEFERRED
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openvswitch
Version: 16.0 (Train)
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: ---
Assignee: Gowrishankar Muthukrishnan
QA Contact: Eran Kuris
URL:
Whiteboard:
Depends On:
Blocks: 1845488
TreeView+ depends on / blocked
 
Reported: 2020-05-07 07:07 UTC by Gowrishankar Muthukrishnan
Modified: 2020-06-09 11:47 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1845488 (view as bug list)
Environment:
Last Closed: 2020-06-09 11:47:21 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
ovs-vswitchd.log during restart (385.40 KB, text/plain)
2020-05-07 07:22 UTC, Gowrishankar Muthukrishnan
no flags Details

Description Gowrishankar Muthukrishnan 2020-05-07 07:07:12 UTC
Description of problem:
After disabling userspace_tso in OVS-DPDK datapath (meaning it was enabled before) and restarting openvswitch service leads too many attempts on destroying connection (to the qemu-kvm).

Below message is repeating in ovs-vswitchd.log:
2020-05-07T06:16:01.648Z|02465|netdev_dpdk|INFO|vHost Device '/var/lib/vhost_sockets/vhuc10acbd9-89' connection has been destroyed
2020-05-07T06:16:01.648Z|02466|dpdk|INFO|Dropped 4755 log messages in last 0 seconds (most recently, 0 seconds ago) due to excessive rate

Version-Release number of selected component (if applicable):
openvswitch2.13-2.13.0-18flim.el8.x86_64

How reproducible:
Every time following steps below.

Steps to Reproduce:
1. If TSO is not enabled, do so in ovs-dpdk.
   sudo ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=true

2. Launch a VM with vhost-user socket in server mode.
   Eg. <source type='unix' path='/var/lib/vhost_sockets/vhue8ff400b-30' mode='server'/> in libvirt xml.

   If in OSP, create instance.

3. Ensure guest kernel driver is able to detect TSO.
   sudo ethtool -k eth2 | egrep '(scatter|tcp|gso|csum|check|segment)'
   Check "on" for tx-checksumming, scatter-gather, tcp-segmentation-offload

4. Disable TSO in ovs-dpdk.
   sudo ovs-vsctl set Open_vSwitch . other_config:userspace-tso-enable=false

5. Restart openvswitch.
   sudo systemctl restart openvswitch.service

6. Check /var/log/messages/ovs-vswitchd.service for repeating messages as reported in problem statement.

Actual results:
Repeating calls to destroy_connection.

Expected results:
No too many attempts on destroying vhost-user connection.

Additional info:

Comment 1 Gowrishankar Muthukrishnan 2020-05-07 07:22:17 UTC
Created attachment 1686073 [details]
ovs-vswitchd.log during restart

Comment 2 Gowrishankar Muthukrishnan 2020-05-07 07:24:01 UTC
Socket files used for VM vNICs:
[heat-admin@overcloud-computeovsdpdksriov-0 ~]$ sudo virsh dumpxml instance-00000053|grep vhu
      <source type='unix' path='/var/lib/vhost_sockets/vhue8ff400b-30' mode='server'/>
      <source type='unix' path='/var/lib/vhost_sockets/vhu85273654-c7' mode='server'/>
      <source type='unix' path='/var/lib/vhost_sockets/vhuc10acbd9-89' mode='server'/>


Note You need to log in before you can comment on or make changes to this bug.