Bug 2039880 - Log level too low for control plane metrics
Summary: Log level too low for control plane metrics
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.10
Hardware: All
OS: All
Target Milestone: ---
: 4.10.0
Assignee: Martin Kennelly
QA Contact: Anurag saxena
Depends On:
TreeView+ depends on / blocked
Reported: 2022-01-12 16:22 UTC by Martin Kennelly
Modified: 2022-03-10 16:39 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Last Closed: 2022-03-10 16:39:13 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift ovn-kubernetes pull 899 0 None open Bug 2039880: Metrics: Increase log level for CP recorder 2022-01-12 16:27:16 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:39:26 UTC

Description Martin Kennelly 2022-01-12 16:22:05 UTC
Description of problem:
Increasing log level to decrease log noise.
Pods maybe created and deleted before port binding
events hit the libovsdb cache causing unwanted
log noise. Its fine to discard metrics that do not hit the libovsdb
cache in an order we expect.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Spawn 4.10 cluster
2. Within the logs, you may see log messages starting with "metrics: ", 

Actual results:
Log messages that do not add any value

Expected results:
Log messages hidden in normal operator and only enabled if log level increased.

Additional info:
Upstream PR: https://github.com/ovn-org/ovn-kubernetes/pull/2744/files

Comment 2 Martin Kennelly 2022-01-22 14:54:30 UTC
To verify this bug:
1) Spawn a cluster with ovn-kubernetes CNI plugin
2) View logs of all non-leader OVN-Kubernetes pods container 'ovnkube-master'. You should not see any logs that contain the following strings:
  * "Add Logical Switch Port event expected pod with UID"
  * 'Unexpected last event type'
  * 'Add port binding event expected pod with UID'
  * 'Cache entry expected pod with UID'

Comment 4 zhaozhanqi 2022-01-25 03:25:35 UTC
Verified this bug on 4.10.0-0.nightly-2022-01-24-020644

$ oc logs ovnkube-master-cfkzn -c ovnkube-master -n openshift-ovn-kubernetes | grep Metrics:

Comment 7 errata-xmlrpc 2022-03-10 16:39:13 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.