Description of problem: Due to performance test please accept the following corrections to DPDK configuration documentation as a work around to avoid performance inconsistency. https://access.redhat.com/documentation/en/red-hat-openstack-platform/8/configure-dpdk-for-openstack-networking/configure-dpdk-for-openstack-networking 1. To avoid packet drops to the kernel, what decreasing the performance, please add OVS bridge state check. with the following command: # ovs-ofctl show <bridge name> The (LOCAL) bridges should be in DOWN state, the vhostuser ports should be in UP state for example: LOCAL(br-int): addr:6a:c2:62:af:d9:4f config: PORT_DOWN state: LINK_DOWN current: 10MB-FD COPPER speed: 10 Mbps now, 0 Mbps max Controlling the port state with the following command: #ovs-ofctl mod-port <bridge name> <port name> up/down 2.Add a note that every restart of OVS's service requires redeploying the instance, otherwise no traffic will be directed to the instance. 3.PMD and core masking. First,Add short description of PMD in the introduction[It's important because number of PMDs effect the performance] . [taken from dpdk.org] A Poll Mode Driver (PMD) consists of APIs, provided through the BSD driver running in user space, to configure the devices and their respective queues. In addition, a PMD accesses the RX and TX descriptors directly without any interrupts (with the exception of Link Status Change interrupts) to quickly receive, process and deliver packets in the user’s application. Second,Add the following command that shows how many PMDs the OVS-DPDK created,use tuna tool; #tuna -t ovs-vswitchd -CP | grep pmd In case you want to control the number of PMD and mask it to specific pCPU please use the following DPDK command: (Using 4,20 pCPU): ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x00100010 See section 4: https://github.com/openvswitch/ovs/blob/master/INSTALL.DPDK.md
Assigning to Martin for review.
TODO: backport to OSP8 once approved by tech review .
Eyal: Could you explain what the mechanism for the local bridges being up causing the packet loss issue? Also, if restarting OVS causes an issue, that sounds like a bug. https://bugs.launchpad.net/ubuntu/+source/neutron/+bug/1460164 for instance is a place where this kind of issue has been fixed in the past.
Hi, On OVS+DPDK2.4 we experienced an issue of packet drop to the kernel while the ports weren't in DOWN state, the workaround was to set those ports to DOWN state. This bug is fixed at OVS2.5 so I'm not sure if we should add it to the documentation. Franck, our PM, can answer that question:) About the bug, if it's already fixed, and by restarting the agent we get the connectivity back, I suggest to add a note which explains the issue and the solution. Thanks, Eyal Dannon
Adding in Terry for comment 7.
Redirecting the needinfo to Franck since I'm not up to date on which versions of OVS are going to be in what products/channels and he would be better able to answer whether this should be documented. If it should, I'm ok with the related doc changes. Re: the note about redeploying when restarting OVS, if this is still an issue on perf team's setup, I'd like to see a bug opened on it so we can track it and fix it. Otherwise, we can drop the comment or at least reword it to something like: If traffic is not being directed to existing instances after restarting the Open vSwitch service, redeploying the instances should fix the issue.
Starting from RHOSP10, OVS and OVS-DPDK will be provided by FD-production channel, and users requiring the "latest greatest" will have the possibility to switch to FD-beta. For RHOSP8 and RHOSP9, OVS-DPDK is provided by RHOSP (no FD-production) but as it is outdated, what we want to document for RHOSP8 and RHOSP9 is the use of the beta channel and OVS-DPDK 2.5. FD-production - 1 package - GA with full support openvswitch (OVS 2.4) with RHEL 7.3 moving to OVS-DPDK (OVS 2.5, DPDK 2.2 static dependencies), FD-beta - 2 packages - latest features, no GSS but best effort from R&D openvswitch OVS-DPDK (OVS 2.5, DPDK 2.2 static dependencies), dpdk standalone (DPDK 16.04), not anymore included in RHEL extra
Hi Terry, Based on Franck's feedback in comment 10, are there any changes that need to be made to the draft in comment 3?
Martin: It sounds like at least with rhel 7.3 everything should work if the bug is fixed in OVS 2.5. It sounds like the wiht OVS 2.4 the bug will remain and so we still need the port state documentation for it? Eyal, do you know if there are plans to backport the fix to OVS 2.4?
As Franck and Martin wrote, we should document the use of the beta channel and OVS-DPDK 2.5, what means we shouldn't mention this bug in the documentation because it fixed at OVS 2.5 .
For OSP10 * The manual DPDK configuration was removed at the request of PM * Deepti added in the director-based steps for OSP10 here: https://access.redhat.com/documentation/en/red-hat-openstack-platform/10/single/network-functions-virtualization-configuration-guide/ *** This bug has been marked as a duplicate of bug 1384774 ***