Bug 1359856

Summary: DPDK options parsed to OVS, "-l {CPUs}" are ignored
Product: Red Hat OpenStack Reporter: Eyal Dannon <edannon>
Component: openvswitch-dpdkAssignee: Flavio Leitner <fleitner>
Status: CLOSED NOTABUG QA Contact: Eyal Dannon <edannon>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.0 (Liberty)CC: edannon, fleitner, nyechiel, oblaut, rkhan, yrachman
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-08 07:27:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eyal Dannon 2016-07-25 14:40:38 UTC
Description of problem:

When openvswitch-dpdk’s service starts, the following arguments are taken from /etc/sysconfig/openvswitch; 
DPDK_OPTIONS="-l 2,8 -n 4 --socket-mem 1024,0" are the cores openvswitch-dpdk binded to.
It causes to OVS+DPDK PMD thread to open freely without controlling number of PMD threads. 


DPDK_OPTIONS="-l 2,8 -n 4 --socket-mem 1024,0"


[root@panther03 ~]# ps -ef | grep ovs-vswitchd | grep -v grep
root     137875      1  0 16:00 ?        00:00:00 ovs-vswit... --dpdk -l 2,8 -n 4 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
root     137876 137875 99 16:00 ?        00:00:12 ovs-vswitchd --dpdk -l 2,8 -n 4 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor

[root@panther03 ~]# cat /proc/13787{5,6}/status  | grep Cpus_allowed_list
Cpus_allowed_list:	2
Cpus_allowed_list:	2

and I'm getting an empty list while using "perf top -C8"

The PMDs can only be configured manually[as Flavio suggested].

[root@panther03 ~]# ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x00100010
[root@panther03 ~]# tuna -t ovs-vswitchd -CP | grep pmd
  137927  OTHER     0        4        51            1           pmd37  
  137928  OTHER     0       20        65            1           pmd38  


Version-Release number of selected component (if applicable):

OVS 2.5 ,OVS+DPDK 2.4

How reproducible:
always

Comment 2 Flavio Leitner 2016-08-03 14:04:49 UTC
Looks like there is a misunderstanding how the pmd-cpu-mask works.

The summary says "-l 2,8" which means CPU#2 and CPU#8, so the mask is actually:
>>> print "%x" % ((1 << 2) | (1<< 8))
104

Therefore the correct command would be:
# ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=104

Let me know if that fixes the issue.
Thanks,
fbl

Comment 3 Eyal Dannon 2016-08-08 07:45:02 UTC
Flavio explained the different between the values and their usages, here's the conclusions, 

-l option goes directly from OVS to DPDK, so OVS doesn't know anything about --dpdk -l or -c ,what matters to OVS is other_config:pmd-cpu-mask
So we need to specify either -l or -c for DPDK and pmd-cpu-mask for OVS

In OVS 2.6 hopefully there will be only one parameter.

Thanks,