Bug 1957296
| Summary: | 99% CPU when leaving a RAFT cluster where other servers are not accessible | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux Fast Datapath | Reporter: | Dan Williams <dcbw> | ||||
| Component: | ovsdb3.1 | Assignee: | Adrián Moreno <amorenoz> | ||||
| Status: | CLOSED WONTFIX | QA Contact: | Zhiqiang Fang <zfang> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | low | ||||||
| Version: | FDP 21.B | CC: | ctrautma, fleitner, jhsiao, mmichels, ralongi | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2024-07-09 19:47:33 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Dan, Sorry for the delay to get to this. Is this ticket still valid? If yes, should we move it to ovsdb3.1? fbl @fleitner Still happens on 2.17 and 3.1 Also doesn't matter valgrind vs. not valgrind, still sucks CPU. |
Created attachment 1779828 [details] OVN NB database that triggers the issue openvswitch2.13-2.13.0-79.el8fdp $ valgrind /usr/sbin/ovsdb-server --unixctl=/tmp/nbctl.sock /tmp/ovnkube-master-zvldj_nbdb --remote=punix:/tmp/nb.sock $ ovs-appctl -t /tmp/nbctl.sock cluster/leave OVN_Northbound 2021-05-05T14:32:19Z|00001|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.13.2 2021-05-05T14:32:19Z|00002|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:32:19Z|00003|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:32:19Z|00007|reconnect|INFO|ssl:10.0.187.238:9643: connecting... 2021-05-05T14:32:19Z|00008|reconnect|INFO|ssl:10.0.187.238:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:19Z|00012|reconnect|INFO|ssl:10.0.231.27:9643: connecting... 2021-05-05T14:32:19Z|00013|reconnect|INFO|ssl:10.0.231.27:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:20Z|00014|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:32:20Z|00015|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:32:20Z|00019|reconnect|INFO|ssl:10.0.187.238:9643: connecting... 2021-05-05T14:32:20Z|00020|reconnect|INFO|ssl:10.0.187.238:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:20Z|00021|reconnect|INFO|ssl:10.0.187.238:9643: waiting 2 seconds before reconnect 2021-05-05T14:32:20Z|00025|reconnect|INFO|ssl:10.0.231.27:9643: connecting... 2021-05-05T14:32:20Z|00026|reconnect|INFO|ssl:10.0.231.27:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:20Z|00027|reconnect|INFO|ssl:10.0.231.27:9643: waiting 2 seconds before reconnect 2021-05-05T14:32:21Z|00028|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:32:21Z|00029|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:32:22Z|00030|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:32:22Z|00031|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:32:22Z|00035|reconnect|INFO|ssl:10.0.187.238:9643: connecting... 2021-05-05T14:32:22Z|00036|reconnect|INFO|ssl:10.0.187.238:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:22Z|00037|reconnect|INFO|ssl:10.0.187.238:9643: waiting 4 seconds before reconnect 2021-05-05T14:32:22Z|00041|reconnect|INFO|ssl:10.0.231.27:9643: connecting... 2021-05-05T14:32:22Z|00042|reconnect|INFO|ssl:10.0.231.27:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:22Z|00043|reconnect|INFO|ssl:10.0.231.27:9643: waiting 4 seconds before reconnect 2021-05-05T14:32:23Z|00044|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:32:23Z|00045|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:32:26Z|00049|reconnect|INFO|ssl:10.0.187.238:9643: connecting... 2021-05-05T14:32:26Z|00050|reconnect|INFO|ssl:10.0.187.238:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:26Z|00051|reconnect|INFO|ssl:10.0.187.238:9643: continuing to reconnect in the background but suppressing further logging 2021-05-05T14:32:26Z|00055|reconnect|INFO|ssl:10.0.231.27:9643: connecting... 2021-05-05T14:32:26Z|00056|reconnect|INFO|ssl:10.0.231.27:9643: connection attempt failed (Protocol not available) 2021-05-05T14:32:26Z|00057|reconnect|INFO|ssl:10.0.231.27:9643: continuing to reconnect in the background but suppressing further logging 2021-05-05T14:32:29Z|00058|memory|INFO|67088 kB peak resident set size after 10.6 seconds 2021-05-05T14:32:29Z|00059|memory|INFO|cells:6319 monitors:0 raft-connections:2 2021-05-05T14:32:29Z|00060|raft|INFO|term 4: 10893 ms timeout expired, starting election 2021-05-05T14:32:40Z|00067|raft|INFO|term 5: 10951 ms timeout expired, starting election 2021-05-05T14:32:51Z|00080|raft|INFO|term 6: 10871 ms timeout expired, starting election 2021-05-05T14:33:01Z|00087|raft|INFO|term 7: 10287 ms timeout expired, starting election 2021-05-05T14:33:12Z|00094|raft|INFO|term 8: 10767 ms timeout expired, starting election 2021-05-05T14:33:19Z|00101|socket_util|ERR|Dropped 55 log messages in last 56 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:33:19Z|00102|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:33:19Z|00103|raft|WARN|Dropped 55 log messages in last 56 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:33:19Z|00104|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:33:34Z|00117|raft|INFO|Dropped 1 log messages in last 10 seconds (most recently, 10 seconds ago) due to excessive rate 2021-05-05T14:33:34Z|00118|raft|INFO|term 10: 10606 ms timeout expired, starting election 2021-05-05T14:34:19Z|00155|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:34:19Z|00156|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:34:19Z|00157|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:34:19Z|00158|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:34:37Z|00171|raft|INFO|Dropped 5 log messages in last 53 seconds (most recently, 10 seconds ago) due to excessive rate 2021-05-05T14:34:37Z|00172|raft|INFO|term 16: 10223 ms timeout expired, starting election 2021-05-05T14:35:19Z|00203|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:35:19Z|00204|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:35:19Z|00205|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:35:19Z|00206|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:35:30Z|00219|raft|INFO|Dropped 4 log messages in last 43 seconds (most recently, 11 seconds ago) due to excessive rate 2021-05-05T14:35:30Z|00220|raft|INFO|term 21: 10938 ms timeout expired, starting election 2021-05-05T14:36:19Z|00257|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:36:19Z|00258|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:36:19Z|00259|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:36:19Z|00260|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:36:34Z|00267|raft|INFO|Dropped 5 log messages in last 53 seconds (most recently, 11 seconds ago) due to excessive rate 2021-05-05T14:36:34Z|00268|raft|INFO|term 27: 10916 ms timeout expired, starting election 2021-05-05T14:36:52Z|00287|raft|INFO|5600: starting to leave cluster 5322 2021-05-05T14:36:52Z|00288|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:02Z|00295|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:06Z|00302|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00303|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00304|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00305|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00306|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00307|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00308|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00309|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00310|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:06Z|00311|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:09Z|00312|memory|INFO|peak resident set size grew 53% in last 280.2 seconds, from 67088 kB to 102560 kB 2021-05-05T14:37:09Z|00313|memory|INFO|cells:6319 monitors:0 raft-connections:2 2021-05-05T14:37:12Z|00314|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:12Z|00315|poll_loop|INFO|Dropped 13076 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:12Z|00316|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:18Z|00323|poll_loop|INFO|Dropped 13119 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:18Z|00324|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:19Z|00325|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:37:19Z|00326|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:37:19Z|00327|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:37:19Z|00328|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:37:22Z|00335|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:24Z|00336|poll_loop|INFO|Dropped 12699 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:24Z|00337|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:30Z|00344|poll_loop|INFO|Dropped 13141 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:30Z|00345|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:32Z|00346|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:36Z|00347|poll_loop|INFO|Dropped 13162 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:36Z|00348|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:42Z|00355|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:42Z|00356|poll_loop|INFO|Dropped 13054 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:42Z|00357|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:48Z|00364|poll_loop|INFO|Dropped 13179 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:48Z|00365|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:37:52Z|00366|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:37:54Z|00373|poll_loop|INFO|Dropped 13148 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:37:54Z|00374|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:00Z|00375|poll_loop|INFO|Dropped 13060 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:00Z|00376|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:02Z|00383|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:38:06Z|00384|poll_loop|INFO|Dropped 13192 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:06Z|00385|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:12Z|00392|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:38:12Z|00393|poll_loop|INFO|Dropped 12885 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:12Z|00394|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:18Z|00401|poll_loop|INFO|Dropped 12508 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:18Z|00402|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:19Z|00403|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:38:19Z|00404|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:38:19Z|00405|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:38:19Z|00406|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:38:22Z|00407|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:38:24Z|00408|poll_loop|INFO|Dropped 12195 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:24Z|00409|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:30Z|00416|poll_loop|INFO|Dropped 12376 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:30Z|00417|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (98% CPU usage) 2021-05-05T14:38:32Z|00418|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:38:36Z|00425|poll_loop|INFO|Dropped 12395 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:36Z|00426|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (98% CPU usage) 2021-05-05T14:38:42Z|00433|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:38:42Z|00434|poll_loop|INFO|Dropped 12716 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:42Z|00435|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (98% CPU usage) 2021-05-05T14:38:48Z|00436|poll_loop|INFO|Dropped 12724 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:48Z|00437|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:38:52Z|00444|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:38:54Z|00445|poll_loop|INFO|Dropped 12932 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:38:54Z|00446|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:00Z|00453|poll_loop|INFO|Dropped 12842 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:00Z|00454|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:02Z|00455|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:39:06Z|00462|poll_loop|INFO|Dropped 12925 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:06Z|00463|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:12Z|00464|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:39:12Z|00465|poll_loop|INFO|Dropped 12901 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:12Z|00466|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:18Z|00473|poll_loop|INFO|Dropped 12887 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:18Z|00474|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:19Z|00475|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:39:19Z|00476|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:39:19Z|00477|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:39:19Z|00478|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:39:22Z|00485|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:39:24Z|00486|poll_loop|INFO|Dropped 12891 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:24Z|00487|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:30Z|00494|poll_loop|INFO|Dropped 12807 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:30Z|00495|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:32Z|00496|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:39:36Z|00497|poll_loop|INFO|Dropped 12916 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:36Z|00498|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (98% CPU usage) 2021-05-05T14:39:42Z|00505|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:39:42Z|00506|poll_loop|INFO|Dropped 13071 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:42Z|00507|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:39:48Z|00514|poll_loop|INFO|Dropped 12839 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:48Z|00515|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (98% CPU usage) 2021-05-05T14:39:52Z|00516|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:39:54Z|00523|poll_loop|INFO|Dropped 12963 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:39:54Z|00524|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (98% CPU usage) 2021-05-05T14:40:00Z|00525|poll_loop|INFO|Dropped 12992 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:00Z|00526|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:02Z|00533|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:40:06Z|00534|poll_loop|INFO|Dropped 12978 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:06Z|00535|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:12Z|00542|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:40:12Z|00543|poll_loop|INFO|Dropped 12917 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:12Z|00544|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:18Z|00551|poll_loop|INFO|Dropped 13076 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:18Z|00552|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:19Z|00553|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:40:19Z|00554|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:40:19Z|00555|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:40:19Z|00556|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:40:22Z|00557|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:40:24Z|00558|poll_loop|INFO|Dropped 13019 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:24Z|00559|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:30Z|00566|poll_loop|INFO|Dropped 13123 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:30Z|00567|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:32Z|00568|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:40:36Z|00575|poll_loop|INFO|Dropped 12941 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:36Z|00576|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:42Z|00583|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:40:42Z|00584|poll_loop|INFO|Dropped 12828 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:42Z|00585|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:48Z|00586|poll_loop|INFO|Dropped 12977 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:48Z|00587|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:40:52Z|00594|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:40:54Z|00595|poll_loop|INFO|Dropped 13166 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:40:54Z|00596|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:41:00Z|00603|poll_loop|INFO|Dropped 13162 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:41:00Z|00604|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:41:06Z|00611|poll_loop|INFO|Dropped 13185 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:41:06Z|00612|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:41:12Z|00613|raft|INFO|Dropped 1 log messages in last 10 seconds (most recently, 10 seconds ago) due to excessive rate 2021-05-05T14:41:12Z|00614|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:41:12Z|00615|poll_loop|INFO|Dropped 13163 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:41:12Z|00616|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:41:18Z|00623|poll_loop|INFO|Dropped 13106 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:41:18Z|00624|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage) 2021-05-05T14:41:19Z|00625|socket_util|ERR|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:41:19Z|00626|socket_util|ERR|9643:10.0.172.29: bind: Cannot assign requested address 2021-05-05T14:41:19Z|00627|raft|WARN|Dropped 59 log messages in last 59 seconds (most recently, 1 seconds ago) due to excessive rate 2021-05-05T14:41:19Z|00628|raft|WARN|pssl:9643:10.0.172.29: listen failed (Cannot assign requested address) 2021-05-05T14:41:22Z|00635|raft|INFO|sending remove request (joining=false, leaving=true) 2021-05-05T14:41:24Z|00636|poll_loop|INFO|Dropped 13093 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate 2021-05-05T14:41:24Z|00637|poll_loop|INFO|wakeup due to 0-ms timeout at ../ovsdb/raft.c:1987 (99% CPU usage)