Bug 1891758 - the authentication operator may spam DeploymentUpdated event endlessly
Summary: the authentication operator may spam DeploymentUpdated event endlessly
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: apiserver-auth
Version: 4.6
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 4.7.0
Assignee: Standa Laznicka
QA Contact: pmali
URL:
Whiteboard:
Depends On:
Blocks: 1891795
TreeView+ depends on / blocked
 
Reported: 2020-10-27 09:40 UTC by Standa Laznicka
Modified: 2021-02-24 15:29 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: A missed condition in authentication operator's code. Consequence: Authentication operator's log would get flooded with messages about an update to a deployment, even though no update happened. Fix: Ensure the deployment's generation is taken into account while deciding whether to update the operator's status. Result: The authentication operator's log should no longer receive messages about a deployment being updated when no such update occurs.
Clone Of:
Environment:
Last Closed: 2021-02-24 15:28:28 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:29:04 UTC

Description Standa Laznicka 2020-10-27 09:40:31 UTC
Description of problem:
In cases when authentication.operator resource's generation gets updated prior to the openshift-authentication/oauth-openshift deployment's generation, 

Version-Release number of selected component (if applicable):
4.6

How reproducible:
not sure about the exact percent, but quite often

Steps to Reproduce:
1. perform a configuration step that increases the generation of the openshift-authentication/oauth-openshift deployment

Actual results:
the authentication operator keeps logging and spamming an event that the openshift-authentication/oauth-openshift deployment changed


Expected results:
the deployment change event only gets reported once

Additional info:

Comment 2 pmali 2020-11-09 10:50:42 UTC
I can still see 5 events generated after single configuration change with the latest cluster version available today.

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.7.0-0.nightly-2020-10-27-051128   True        False         6h49m   Cluster version is 4.7.0-0.nightly-2020-10-27-051128

I1109 10:37:02.362866       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"3515d0fd-7e4b-4096-8502-1f048d06e472", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed
I1109 10:37:05.560729       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"3515d0fd-7e4b-4096-8502-1f048d06e472", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed
I1109 10:37:08.760578       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"3515d0fd-7e4b-4096-8502-1f048d06e472", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed
I1109 10:37:14.963472       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"3515d0fd-7e4b-4096-8502-1f048d06e472", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed
I1109 10:37:18.160347       1 event.go:282] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"3515d0fd-7e4b-4096-8502-1f048d06e472", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'DeploymentUpdated' Updated Deployment.apps/oauth-openshift -n openshift-authentication because it changed

Comment 3 Standa Laznicka 2020-11-09 11:20:15 UTC
Do these events continue indefinitely or are these all the events there are? It's perfectly natural to see _some_ of these in the authentication operator since it publishes them as it picks the configuration that might change from the moment the operator was started. The real question is - do you see them being posted in a loop even though no changes to the deployment are actually happening?

Comment 4 pmali 2020-11-09 14:56:52 UTC
Its not in loop and not even occurring when no changes to the deployment. As well, As per our discussion on slack. Marking as Verified.

Comment 7 errata-xmlrpc 2021-02-24 15:28:28 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.