With this enhancement, the `OvsDpdkCoreList` parameter is now optional. If you set `OvsDpdkCoreList`, you pin the `ovs-vswitchd` non-pmd threads to the first core that you list in the parameter. If you exclude `OvsDpdkCoreList`, you enable the `ovs-vswitchd` non-pmd threads to use any non-isolated cores.
DescriptionChristophe Fontaine
2020-03-04 08:46:43 UTC
The parameter "OvsDpdkCoreList" (which sets the option "dpdk-lcore-mask") should be optional so we can deploy without it.
This option is a dpdk option which modifies a non-dpdk related behaviour: all revalidator and handler threads will be pinned to the first core defined in the mask, while the expected behaviour is to pin to a set of CPUs. (on most – if not all – Nfv deployments, this will be the core 0)
Also, this option creates 1 thread for every other cores defined in the mask and these threads don't process anything.
The default behaviour seems preferable, as revalidator and handler threads will be scheduled on the default CpuSet of ovs-vswitchd (ie unpinned CPUs).
- Current behaviour, with the mask set:
[root@nfvcpt-0 heat-admin]# ovs-vsctl get Open_Vswitch . other_config
{dpdk-extra=" -n 4", dpdk-init="true", dpdk-lcore-mask="3003", dpdk-socket-mem="4096", n-handler-threads="4", n-revalidator-threads="4", pmd-cpu-mask="1c01c"}
[root@nfvcpt-0 heat-admin]# for i in $(ls /proc/$(pidof ovs-vswitchd)/task/) ; do THREAD=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/comm) ; AFF=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/status | awk '/Cpus_allowed_list/ {print $2}') ; echo -e $THREAD ' \t' $AFF ; done
ovs-vswitchd 0
eal-intr-thread 0
rte_mp_handle 0
lcore-slave-1 1
lcore-slave-12 12
lcore-slave-13 13
dpdk_watchdog1 0
urcu2 0
ct_clean17 0
pmd4 15
pmd13 3
pmd5 16
pmd6 14
pmd10 2
pmd7 4
handler58 0
handler64 0
handler61 0
handler62 0
revalidator63 0
revalidator60 0
revalidator65 0
revalidator59 0
- Without the mask set:
[root@nfvcpt-0 heat-admin]# ovs-vsctl set Open_Vswitch . other_config='{dpdk-extra=" -n 4", dpdk-init="true", dpdk-socket-mem="4096", n-handler-threads="4", n-revalidator-threads="4", pmd-cpu-mask="1c01c »}'
[root@nfvcpt-0 heat-admin]# for i in $(ls /proc/$(pidof ovs-vswitchd)/task/) ; do THREAD=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/comm) ; AFF=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/status | awk '/Cpus_allowed_list/ {print $2}') ; echo -e $THREAD ' \t' $AFF ; done
ovs-vswitchd 0-1,12-13
eal-intr-thread 1,12-13
rte_mp_handle 1,12-13
dpdk_watchdog1 0-1,12-13
urcu2 0-1,12-13
ct_clean3 0-1,12-13
pmd12 15
pmd13 3
pmd14 16
pmd15 14
pmd16 2
pmd17 4
handler64 0-1,12-13
handler65 0-1,12-13
handler58 0-1,12-13
handler59 0-1,12-13
revalidator60 0-1,12-13
revalidator61 0-1,12-13
revalidator63 0-1,12-13
revalidator62 0-1,12-13
ovs-vswitchd 0-1,12-13
This was not proposed as an RFE exception for 16.1.3 before Oct 23rd. It has no exception+ flag approval, so we are dropping it from the advisory in 16.1.3. Code remains.
The parameter "OvsDpdkCoreList" (which sets the option "dpdk-lcore-mask") should be optional so we can deploy without it. This option is a dpdk option which modifies a non-dpdk related behaviour: all revalidator and handler threads will be pinned to the first core defined in the mask, while the expected behaviour is to pin to a set of CPUs. (on most – if not all – Nfv deployments, this will be the core 0) Also, this option creates 1 thread for every other cores defined in the mask and these threads don't process anything. The default behaviour seems preferable, as revalidator and handler threads will be scheduled on the default CpuSet of ovs-vswitchd (ie unpinned CPUs). - Current behaviour, with the mask set: [root@nfvcpt-0 heat-admin]# ovs-vsctl get Open_Vswitch . other_config {dpdk-extra=" -n 4", dpdk-init="true", dpdk-lcore-mask="3003", dpdk-socket-mem="4096", n-handler-threads="4", n-revalidator-threads="4", pmd-cpu-mask="1c01c"} [root@nfvcpt-0 heat-admin]# for i in $(ls /proc/$(pidof ovs-vswitchd)/task/) ; do THREAD=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/comm) ; AFF=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/status | awk '/Cpus_allowed_list/ {print $2}') ; echo -e $THREAD ' \t' $AFF ; done ovs-vswitchd 0 eal-intr-thread 0 rte_mp_handle 0 lcore-slave-1 1 lcore-slave-12 12 lcore-slave-13 13 dpdk_watchdog1 0 urcu2 0 ct_clean17 0 pmd4 15 pmd13 3 pmd5 16 pmd6 14 pmd10 2 pmd7 4 handler58 0 handler64 0 handler61 0 handler62 0 revalidator63 0 revalidator60 0 revalidator65 0 revalidator59 0 - Without the mask set: [root@nfvcpt-0 heat-admin]# ovs-vsctl set Open_Vswitch . other_config='{dpdk-extra=" -n 4", dpdk-init="true", dpdk-socket-mem="4096", n-handler-threads="4", n-revalidator-threads="4", pmd-cpu-mask="1c01c »}' [root@nfvcpt-0 heat-admin]# for i in $(ls /proc/$(pidof ovs-vswitchd)/task/) ; do THREAD=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/comm) ; AFF=$(cat /proc/$(pidof ovs-vswitchd)/task/$i/status | awk '/Cpus_allowed_list/ {print $2}') ; echo -e $THREAD ' \t' $AFF ; done ovs-vswitchd 0-1,12-13 eal-intr-thread 1,12-13 rte_mp_handle 1,12-13 dpdk_watchdog1 0-1,12-13 urcu2 0-1,12-13 ct_clean3 0-1,12-13 pmd12 15 pmd13 3 pmd14 16 pmd15 14 pmd16 2 pmd17 4 handler64 0-1,12-13 handler65 0-1,12-13 handler58 0-1,12-13 handler59 0-1,12-13 revalidator60 0-1,12-13 revalidator61 0-1,12-13 revalidator63 0-1,12-13 revalidator62 0-1,12-13 ovs-vswitchd 0-1,12-13