Description of problem: when change n-handler-threads and n-revalidator-threads through ovs-vsctl, configuration didn't effected. Version-Release number of selected component (if applicable): openvswitch2.17-2.17.0-105.el8fdp/openvswitch3.1-3.1.0-39.el8fdp How reproducible: 100% Steps to Reproduce: 1.keep numa0 cpus except core0 as isolation cpus, any other cpu of numa1 and core0 of numa0 as housekeeping. The Housekeeping cpu have 29. # cat /proc/cmdline BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-499.rt7.288.el8.x86_64 root=/dev/mapper/rhel_dell--per750--03-root ro ksdevice=bootif pci=realloc resume=/dev/mapper/rhel_dell--per750--03-swap rd.lvm.lv=rhel_dell-per750-03/root rd.lvm.lv=rhel_dell-per750-03/swap console=ttyS0,115200n81 skew_tick=1 isolcpus=managed_irq,domain,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54 intel_pstate=disable nosoftlockup tsc=reliable nohz=on nohz_full=2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54 rcu_nocbs=2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54 irqaffinity=0,1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55 crashkernel=1G-4G:384M,4G-16G:512M,16G-64G:1G,64G-128G:2G,128G-:4G rcutree.kthread_prio=51 default_hugepagesz=1G hugepagesz=1G hugepages=48 intel_iommu=on iommu=pt intel_idle.max_cstate=0 processor.max_cstate=0 intel_pstate=disable # numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 node 0 size: 31257 MB node 0 free: 951 MB node 1 cpus: 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 node 1 size: 31178 MB node 1 free: 2283 MB node distances: node 0 1 0: 10 20 1: 20 10 2. configure n-revalidator-threads=6. and other_config:n-handler-threads=24. OVS still configure handler to 31. ovs-vsctl set Open_vSwitch . other_config:n-revalidator-threads=6 ovs-vsctl set Open_vSwitch . other_config:n-handler-threads=24 3. check the n-revalidator-threads change to 6, but handler threads still 31. 2023-07-11T02:07:13.277Z|00049|ofproto_dpif_upcall|INFO|Overriding n-handler-threads to 31, setting n-revalidator-threads to 6 2023-07-11T02:07:13.277Z|00050|ofproto_dpif_upcall|INFO|Starting 37 threads Actual results: 1. n-handler-threads still 31 Expected results: 1. n-handler-threads shoule be 24 Additional info:
Hi @mhou, n-handler-threads is not available on the newest openvswitch and kernel when using per-CPU dispatch mode. Openvswitch defaults to per-CPU dispatch in the latest kernel and OVS versions, and there is no way to change it. In effect n-handler-threads is only intended to be used in per-vport mode in older kernels. You can check which dispatch mode your system is running via `ovs-appctl dpif-netlink/dispatch-mode`. It is most likely in per-CPU dispatch mode which ignores n-handler-threads. The reason you have 31 Handlers threads is to balance the upcalls that could happen on all the CPUs. The way that is calculated is via you will have a minimum of what ever number of active CPUs, 29 in this case, plus an additional extra threads calculated via the next prime number from 29, which is 31 in this case I am working on ensuring that setting this parameter warns the user that making any changes has no effect and updating the documenttation if it's not clear enough that this setting is only intended to be used in per-vport mode and will not work in per-CPU mode
Hello Michael Current mode actually is per-cpu mode on my test bed. # ovs-appctl dpif-netlink/dispatch-mode ovs-system: per-cpu dispatch mode The current ovs document didn't contain any section to describe what difference between per-CPU and per-vport. http://www.openvswitch.org/support/dist-docs/ovs-vswitchd.conf.db.5.txt other_config : n-handler-threads: optional string, containing an inte‐ ger, at least 1 Attempts to specify the number of threads for software datapaths to use for handling new flows. Some datapaths may choose to ig‐ nore this and it will be set to a sensible option for the data‐ path type. This configuration is per datapath. If you have more than one software datapath (e.g. some system bridges and some netdev bridges), then the total number of threads is n-handler-threads times the number of software datapaths. .... other_config : n-handler-threads optional string, containing an integer, at least 1