Bug 1359856 - DPDK options parsed to OVS, "-l {CPUs}" are ignored
Summary: DPDK options parsed to OVS, "-l {CPUs}" are ignored
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openvswitch-dpdk
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Flavio Leitner
QA Contact: Eyal Dannon
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-07-25 14:40 UTC by Eyal Dannon
Modified: 2016-08-11 10:03 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-08 07:27:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Eyal Dannon 2016-07-25 14:40:38 UTC
Description of problem:

When openvswitch-dpdk’s service starts, the following arguments are taken from /etc/sysconfig/openvswitch; 
DPDK_OPTIONS="-l 2,8 -n 4 --socket-mem 1024,0" are the cores openvswitch-dpdk binded to.
It causes to OVS+DPDK PMD thread to open freely without controlling number of PMD threads. 


DPDK_OPTIONS="-l 2,8 -n 4 --socket-mem 1024,0"


[root@panther03 ~]# ps -ef | grep ovs-vswitchd | grep -v grep
root     137875      1  0 16:00 ?        00:00:00 ovs-vswit... --dpdk -l 2,8 -n 4 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor
root     137876 137875 99 16:00 ?        00:00:12 ovs-vswitchd --dpdk -l 2,8 -n 4 --socket-mem 1024 0 -- unix:/var/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --no-chdir --log-file=/var/log/openvswitch/ovs-vswitchd.log --pidfile=/var/run/openvswitch/ovs-vswitchd.pid --detach --monitor

[root@panther03 ~]# cat /proc/13787{5,6}/status  | grep Cpus_allowed_list
Cpus_allowed_list:	2
Cpus_allowed_list:	2

and I'm getting an empty list while using "perf top -C8"

The PMDs can only be configured manually[as Flavio suggested].

[root@panther03 ~]# ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=0x00100010
[root@panther03 ~]# tuna -t ovs-vswitchd -CP | grep pmd
  137927  OTHER     0        4        51            1           pmd37  
  137928  OTHER     0       20        65            1           pmd38  


Version-Release number of selected component (if applicable):

OVS 2.5 ,OVS+DPDK 2.4

How reproducible:
always

Comment 2 Flavio Leitner 2016-08-03 14:04:49 UTC
Looks like there is a misunderstanding how the pmd-cpu-mask works.

The summary says "-l 2,8" which means CPU#2 and CPU#8, so the mask is actually:
>>> print "%x" % ((1 << 2) | (1<< 8))
104

Therefore the correct command would be:
# ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=104

Let me know if that fixes the issue.
Thanks,
fbl

Comment 3 Eyal Dannon 2016-08-08 07:45:02 UTC
Flavio explained the different between the values and their usages, here's the conclusions, 

-l option goes directly from OVS to DPDK, so OVS doesn't know anything about --dpdk -l or -c ,what matters to OVS is other_config:pmd-cpu-mask
So we need to specify either -l or -c for DPDK and pmd-cpu-mask for OVS

In OVS 2.6 hopefully there will be only one parameter.

Thanks,


Note You need to log in before you can comment on or make changes to this bug.