Bug 1936022 - DNS operator performs spurious updates in response to API's defaulting of daemonset's terminationGracePeriod and service's clusterIPs
Summary: DNS operator performs spurious updates in response to API's defaulting of dae...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 4.8
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.8.0
Assignee: Miciah Dashiel Butler Masters
QA Contact: jechen
Depends On:
Blocks: 1942228
TreeView+ depends on / blocked
Reported: 2021-03-06 00:16 UTC by Miciah Dashiel Butler Masters
Modified: 2022-08-04 22:39 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2021-07-27 22:51:38 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github openshift cluster-dns-operator pull 243 0 None open Bug 1936022: Fix spurious reconciliation of DNS daemonset and service 2021-03-06 00:26:05 UTC
Red Hat Product Errata RHSA-2021:2438 0 None None None 2021-07-27 22:52:05 UTC

Description Miciah Dashiel Butler Masters 2021-03-06 00:16:44 UTC
Description of problem:

When the DNS operator reconciles its resources, the operator gets the DNS daemonset and service objects from the API to determine whether the operator needs to create or update these objects.  For each object, if the object does not exist, the operator creates it, and if the service does exist, the operator compares it with what the operator expects to get in order to determine whether an update is needed for that object.  In this comparison, if the API has set the default value the service's clusterIPs field or the default value for the daemonset's terminationGracePeriodSeconds field, the operator detects the update and tries to set the field field back to the empty value.  The operator should not update the daemonset or service in response to API defaulting.

Version-Release number of selected component (if applicable):

The clusterIPs field is new in Kubernetes 1.20 (OpenShift 4.7). The terminationGracePeriodSeconds field was ignored before OpenShift 4.7.  Thus versions of OpenShift before 4.7 are unaffected by this issue.  

Steps to Reproduce:

1. Launch a new cluster.  

2. Check the DNS operator's logs:

    oc -n openshift-dns-operator logs deploy/dns-operator -c dns-operator

Actual results:

The DNS operator's logs have "updated dns daemonset" and "updated dns service" repeated over and over.  In a CI run, I see over 30 occurrences of each.  

Expected results:

The DNS operator should ignore default values that the API sets and should not log "updated dns daemonset" or "updated dns service" unless the objects are updated outside of API defaulting.

Comment 2 jechen 2021-03-25 18:49:15 UTC
Verified in 4.8.0-0.nightly-2021-03-25-063034

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE     STATUS
version   4.8.0-0.nightly-2021-03-25-063034   True        False         115m      Cluster version is 4.8.0-0.nightly-2021-03-25-063034

$ oc -n openshift-dns-operator logs deploy/dns-operator -c dns-operator

I0325 15:55:57.046086       1 request.go:655] Throttling request took 1.019540716s, request: GET:
time="2021-03-25T15:55:59Z" level=info msg="reconciling request: /default"
time="2021-03-25T15:55:59Z" level=info msg="reconciling request: /default"

DNS operator does not repetitively log "updated dns daemonset" or "updated dns service" as expected

Comment 5 errata-xmlrpc 2021-07-27 22:51:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.