Bug 1981784 - Problematic Deployment creates infinite number Replicasets causing etcd to reach quota limit
Summary: Problematic Deployment creates infinite number Replicasets causing etcd to re...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-apiserver
Version: 4.7
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
: 4.6.z
Assignee: Filip Krepinsky
QA Contact: zhou ying
URL:
Whiteboard:
: 1982724 (view as bug list)
Depends On: 1981775
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-07-13 11:59 UTC by Filip Krepinsky
Modified: 2021-09-22 20:06 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: caused if deployment is created with unresolved image (no image stream) and no image.openshift.io/triggers annotation. By creating image stream and subsequently patching our deployment, new image resolution will occur on new replica sets. This will result in inconsistent state between deployment controller and apiserver's imagepolicy plugin. Consequence: deployment controller creates replica sets in infinite loop Fix: responsibilities of apiserver's imagepolicy plugin were lowered Result: inconsistent image resolution should not occur in the deployments and thus it should not cause creation of infinite replica sets anymore
Clone Of: 1981775
Environment:
Last Closed: 2021-09-22 20:06:32 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift apiserver-library-go pull 56 0 None Merged Bug 1981784: fix Deployment creates a huge number of ReplicaSets 2021-09-15 10:35:50 UTC
Github openshift kubernetes pull 894 0 None Merged Bug 1981784: UPSTREAM: <drop>: bump(apiserver-library-go) 2021-09-15 10:35:56 UTC
Red Hat Product Errata RHBA-2021:3517 0 None None None 2021-09-22 20:06:34 UTC

Comment 1 Filip Krepinsky 2021-07-21 12:05:06 UTC 
*** Bug 1982724 has been marked as a duplicate of this bug. ***
Comment 2 Filip Krepinsky 2021-08-26 11:05:18 UTC 
> (Patch Manager) - 4.6 is only open for bugs of >= high severity. Please close or reassess the bugzilla severity.

@maszulik can we bump the severity?
Comment 3 Maciej Szulik 2021-09-01 10:44:31 UTC 
(In reply to Filip Krepinsky from comment #2)
> > (Patch Manager) - 4.6 is only open for bugs of >= high severity. Please close or reassess the bugzilla severity.
> 
> @maszulik can we bump the severity?

Yes, this definitely deserves higher severity since this might affect cluster stability.
Comment 6 zhou ying 2021-09-03 03:46:16 UTC 
Can't reproduce with latest ocp4.6 now:

[root@localhost roottest]# oc version 
Client Version: 4.9.0-202109020218.p0.git.96e95ce.assembly.stream-96e95ce
Server Version: 4.6.0-0.nightly-2021-09-02-111008
Kubernetes Version: v1.19.0+1cec875
[root@localhost roottest]# oc new-project zhouy

[root@localhost roottest]# oc create -f /tmp/dd.yaml 
deployment.apps/i-spawn-replicas created

[root@localhost roottest]# oc create -f /tmp/is.yaml
imagestream.image.openshift.io/imagestreamname created
[root@localhost roottest]# oc get imagestream.image.openshift.io/imagestreamname -o json | jq '.spec'
{
  "lookupPolicy": {
    "local": true
  }
}
[root@localhost roottest]# oc get rs
NAME                          DESIRED   CURRENT   READY   AGE
i-spawn-replicas-84778cc586   0         0         0       68s
[root@localhost roottest]# vi /tmp/patch.yaml
[root@localhost roottest]# oc patch deployment i-spawn-replicas --type merge --patch "$(cat /tmp/patch.yaml)"
deployment.apps/i-spawn-replicas patched
[root@localhost roottest]#  oc get rs | wc -l
3
[root@localhost roottest]#  oc get rs | wc -l
3
[root@localhost roottest]#  oc get rs | wc -l
3
Comment 9 errata-xmlrpc 2021-09-22 20:06:32 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.45 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:3517
Note You need to log in before you can comment on or make changes to this bug.