Description of problem: ovn-controller sends an echo message to the openflow connection every 5 seconds and this is hardcoded. If ovn-controller takes >5 seconds to process logical flows, this results in 100% cpu usage all the time. So we should provide the option to increase this probe interval. Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
Hi Numan, Can you please help describe how to config the option?So I can add the case to environment. And to reproduce this issue,is it the right way that create a lot of logical ports? Thanks!
Verified on the latest version: [root@dell-per740-04 ~]# uname -a Linux dell-per740-04.rhts.eng.pek2.redhat.com 3.10.0-1062.el7.x86_64 #1 SMP Thu Jul 18 20:25:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux [root@dell-per740-04 ~]# rpm -qa | grep openvswitch openvswitch2.11-2.11.0-21.el7fdp.x86_64 openvswitch-selinux-extra-policy-1.0-13.el7fdp.noarch kernel-kernel-networking-openvswitch-ovn_ha-1.0-41.noarch [root@dell-per740-04 ~]# rpm -qa | grep ovn ovn2.11-2.11.0-36.el7fdp.x86_64 ovn2.11-host-2.11.0-36.el7fdp.x86_64 ovn2.11-central-2.11.0-36.el7fdp.x86_64 [root@dell-per740-04 ~]# [root@dell-per740-04 ~]# ovs-vsctl get open . external_ids:ovn-openflow-probe-interval "5" [root@dell-per740-04 ~]# top top - 22:11:36 up 5 days, 59 min, 2 users, load average: 2.04, 2.48, 2.67 Tasks: 566 total, 3 running, 563 sleeping, 0 stopped, 0 zombie %Cpu(s): 3.9 us, 1.5 sy, 0.0 ni, 94.6 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 65213648 total, 31088348 free, 7404080 used, 26721220 buff/cache KiB Swap: 32767996 total, 32767996 free, 0 used. 54395516 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 169046 openvsw+ 10 -10 4252852 1.6g 17936 R 99.2 2.6 61169:54 ovs-vswitchd 169120 root 10 -10 1534852 1.2g 1752 R 90.7 1.9 6289:55 ovn-controller 9 root 20 0 0 0 0 S 0.7 0.0 27:52.60 rcu_sched 1114 root 20 0 69412 24844 23728 S 0.7 0.0 80:35.58 systemd-journal 2080 root 20 0 730380 60424 15752 S 0.7 0.1 4:50.40 rsyslogd 268574 root 20 0 162452 2760 1580 R 0.7 0.0 0:00.07 top 37 root 20 0 0 0 0 S 0.3 0.0 1:40.84 kworker/5:0 [root@dell-per740-04 ~]# ovs-vsctl set open . external_ids:ovn-openflow-probe-interval=30 [root@dell-per740-04 ~]# ovs-vsctl get open . external_ids:ovn-openflow-probe-interval "30" [root@dell-per740-04 ~]# top top - 22:27:29 up 5 days, 1:15, 2 users, load average: 2.29, 2.16, 2.36 Tasks: 566 total, 4 running, 562 sleeping, 0 stopped, 0 zombie %Cpu(s): 5.1 us, 3.8 sy, 0.0 ni, 91.1 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st KiB Mem : 65213648 total, 31018080 free, 7461380 used, 26734188 buff/cache KiB Swap: 32767996 total, 32767996 free, 0 used. 54329968 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 169046 openvsw+ 10 -10 4258772 1.6g 17936 R 99.5 2.6 61202:52 ovs-vswitchd 169120 root 10 -10 1534852 1.2g 1752 R 19.1 1.9 6305:46 ovn-controller 269789 root 20 0 162452 2644 1524 R 11.8 0.0 0:00.03 top 1 root 20 0 216728 29880 4228 S 0.0 0.0 0:49.62 systemd 2 root 20 0 0 0 0 S 0.0 0.0 0:00.15 kthreadd 4 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0H 6 root 20 0 0 0 0 S 0.0 0.0 0:04.98 ksoftirqd/0 7 root rt 0 0 0 0 S 0.0 0.0 0:05.06 migration/0
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2943