Description of problem: Client using Openvswitch and NOT OVN have very high number of duplicate openvswitch mounts making it slow to ssh to compute nodes and 'runc' processes using 100% cpu. After stracing one of those process we noticed it kept timing out while accessing /proc/$pid/mountinfo. We cat the file and found 32767 tmpfs mount: 117756 68588 0:24 /openvswitch /run/openvswitch rw,nosuid,nodev shared:26 - tmpfs tmpfs rw,seclabel,mode=755 We stopped systemd ovsdb-server on the node, umount /run/openvswitch (might not have been necessary) and after that all the mounts were gone, the processes were all back to normal. This was experienced on OSP16.1.1. Client updated his environment to OSP16.1.2 and the same thing can be seen. We need your help to find the source of those entries. Version-Release number of selected component (if applicable): OSP16.1.2 openvswitch2.13-2.13.0-60.el8fdp.x86_64 How reproducible: 100% on this environment (compute nodes) Steps to Reproduce: 1. Unknown 2. 3. Actual results: Duplicate openvswitch mount entries in /proc/mounts Expected results: Only one entry. Additional info: We have sosreport with the issue.
The tested work around is a reboot of overcloud nodes that have excessive mounts as detected like this: cat /proc/mounts | grep 'tmpfs /run/openvswitch' | wc -l
KCS Solution is published and linked here. We would like to find a resolution that is less invasive than Overcloud node reboots.
Hello, The customer reports the following seems to clean up the mounts: # systemctl stop ovs-dbserver # umount /run/openvswitch # systemctl start ovs-dbserver # podman stop neutron_ovs_agent # sleep 5 # podman start neutron_ovs_agent - Cory
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0817
*** Bug 2218577 has been marked as a duplicate of this bug. ***