Bug 1907433 - Excessive logging in image operator
Summary: Excessive logging in image operator
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image Registry
Version: 4.7
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.10.0
Assignee: Oleg Bulatov
QA Contact: wewang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-12-14 14:01 UTC by Vadim Rutkovsky
Modified: 2022-03-10 16:02 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-10 16:02:33 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift cluster-image-registry-operator pull 721 0 None open Bug 1907433: Decrease log level for event queue messages 2021-09-22 13:43:06 UTC
Red Hat Product Errata RHSA-2022:0056 0 None None None 2022-03-10 16:02:50 UTC

Description Vadim Rutkovsky 2020-12-14 14:01:37 UTC
```
I1214 12:07:06.020389       1 controllerimagepruner.go:308] get event from image pruner workqueue
I1214 12:07:06.020656       1 controllerimagepruner.go:323] event from image pruner workqueue successfully processed
I1214 12:07:06.020761       1 controller.go:375] add event to workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:06.020833       1 controller.go:324] get event from workqueue
I1214 12:07:06.020783       1 imageregistrycertificates.go:91] get event from workqueue
I1214 12:07:06.020937       1 caconfig.go:107] missing the cloud-provider-config configmap: configmap "cloud-provider-config" not found
I1214 12:07:06.021072       1 imageregistrycertificates.go:97] ImageRegistryCertificatesController: event from workqueue successfully processed
I1214 12:07:06.094235       1 controller.go:339] event from workqueue successfully processed
I1214 12:07:16.032802       1 controllerimagepruner.go:351] add event to image pruner workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:16.033944       1 controllerimagepruner.go:308] get event from image pruner workqueue
I1214 12:07:16.033079       1 controller.go:375] add event to workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:16.034138       1 controller.go:324] get event from workqueue
I1214 12:07:16.034212       1 controllerimagepruner.go:323] event from image pruner workqueue successfully processed
I1214 12:07:16.033099       1 imageregistrycertificates.go:91] get event from workqueue
I1214 12:07:16.034301       1 caconfig.go:107] missing the cloud-provider-config configmap: configmap "cloud-provider-config" not found
I1214 12:07:16.034413       1 imageregistrycertificates.go:97] ImageRegistryCertificatesController: event from workqueue successfully processed
I1214 12:07:16.123466       1 controller.go:339] event from workqueue successfully processed
I1214 12:07:26.045909       1 controllerimagepruner.go:308] get event from image pruner workqueue
I1214 12:07:26.045884       1 controllerimagepruner.go:351] add event to image pruner workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:26.046110       1 controllerimagepruner.go:323] event from image pruner workqueue successfully processed
I1214 12:07:26.046138       1 controller.go:324] get event from workqueue
I1214 12:07:26.046128       1 controller.go:375] add event to workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:26.046646       1 imageregistrycertificates.go:91] get event from workqueue
I1214 12:07:26.046683       1 caconfig.go:107] missing the cloud-provider-config configmap: configmap "cloud-provider-config" not found
I1214 12:07:26.046790       1 imageregistrycertificates.go:97] ImageRegistryCertificatesController: event from workqueue successfully processed
I1214 12:07:26.110483       1 controller.go:339] event from workqueue successfully processed
I1214 12:07:36.065777       1 controllerimagepruner.go:308] get event from image pruner workqueue
I1214 12:07:36.065749       1 controllerimagepruner.go:351] add event to image pruner workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:36.065847       1 controller.go:375] add event to workqueue due to *v1.ConfigMap, Namespace=openshift-image-registry, Name=openshift-master-controllers (update)
I1214 12:07:36.065887       1 controller.go:324] get event from workqueue
```

Image-registry is watching `openshift-master-controllers` configmap (to watch active leader?) and tries to fetch "cloud-provider-config" configmap. These events are repeated often, which makes crio log a lot of lines and write to disk.

Comment 9 wewang 2021-09-23 03:47:46 UTC
Verified in version:
Version:
4.9.0-0.ci.test-2021-09-23-025106-ci-ln-7lcj44k-latest

No excessive logging info below in image registry operator now
[wewang@localhost ~]$ oc logs -f pod/cluster-image-registry-operator-7b7ddfdf46-88gts -n openshift-image-registry |grep -E "get event from workqueue|event from workqueue successfully processed|get event from image pruner workqueue|event from image pruner workqueue successfully processed|ImageConfigController: event from workqueue processed|ImageRegistryCertificatesController: event from workqueue successfully processed|NodeCADaemonController: event from workqueue successfully processed"

Comment 16 errata-xmlrpc 2022-03-10 16:02:33 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056


Note You need to log in before you can comment on or make changes to this bug.