Bug 1981770

Summary: Problematic Deployment creates infinite number Replicasets causing etcd to reach quota limit
Product: OpenShift Container Platform Reporter: Filip Krepinsky <fkrepins>
Component: kube-apiserverAssignee: Filip Krepinsky <fkrepins>
Status: CLOSED ERRATA QA Contact: zhou ying <yinzhou>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.7CC: aos-bugs, dbaker, fkrepins, maszulik, mfojtik, skrenger, sreber, xxia, yinzhou
Target Milestone: ---   
Target Release: 4.8.z   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: caused if deployment is created with unresolved image (no image stream) and no image.openshift.io/triggers annotation. By creating image stream and subsequently patching our deployment, new image resolution will occur on new replica sets. This will result in inconsistent state between deployment controller and apiserver's imagepolicy plugin. Consequence: deployment controller creates replica sets in infinite loop Fix: responsibilities of apiserver's imagepolicy plugin were lowered Result: inconsistent image resolution should not occur in the deployments and thus it should not cause creation of infinite replica sets anymore
 Story Points: ---
Clone Of: 1976775 Environment:
Last Closed: 2021-08-10 11:27:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1976775    
Bug Blocks: 1981775    

Comment 1 Filip Krepinsky 2021-07-21 11:58:06 UTC 
*** Bug 1982717 has been marked as a duplicate of this bug. ***
Comment 4 zhou ying 2021-08-03 07:33:10 UTC 
Can't reproduce with latest payload:

[root@localhost roottest]# oc get clusterversion 
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.8.0-0.nightly-2021-07-31-065602   True        False         44m     Cluster version is 4.8.0-0.nightly-2021-07-31-065602

[root@localhost roottest]# oc create -f /tmp/depl.yaml 
deployment.apps/i-spawn-replicas created
[root@localhost roottest]# oc get deployment i-spawn-replicas -o json | jq '.spec'
{
  "progressDeadlineSeconds": 600,
  "replicas": 0,
  "revisionHistoryLimit": 10,
  "selector": {
    "matchLabels": {
      "app": "i-spawn-replicas"
    }
  },
  "strategy": {
    "rollingUpdate": {
      "maxSurge": "25%",
      "maxUnavailable": "25%"
    },
    "type": "RollingUpdate"
  },
  "template": {
    "metadata": {
      "creationTimestamp": null,
      "labels": {
        "app": "i-spawn-replicas"
      },
      "name": "i-spawn-replicas"
    },
    "spec": {
      "containers": [
        {
          "image": "imagestreamname",
          "imagePullPolicy": "Always",
          "name": "container1",
          "resources": {},
          "terminationMessagePath": "/dev/termination-log",
          "terminationMessagePolicy": "File"
        }
      ],
      "dnsPolicy": "ClusterFirst",
      "restartPolicy": "Always",
      "schedulerName": "default-scheduler",
      "securityContext": {},
      "terminationGracePeriodSeconds": 30
    }
  }
}
[root@localhost roottest]# oc get deploy
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
i-spawn-replicas   0/0     0            0           7m3s
[root@localhost roottest]# oc get rs
NAME                          DESIRED   CURRENT   READY   AGE
i-spawn-replicas-84778cc586   0         0         0       7m9s
[root@localhost roottest]# oc get rs i-spawn-replicas-84778cc586 -o json | jq '.spec'
{
  "replicas": 0,
  "selector": {
    "matchLabels": {
      "app": "i-spawn-replicas",
      "pod-template-hash": "84778cc586"
    }
  },
  "template": {
    "metadata": {
      "creationTimestamp": null,
      "labels": {
        "app": "i-spawn-replicas",
        "pod-template-hash": "84778cc586"
      },
      "name": "i-spawn-replicas"
    },
    "spec": {
      "containers": [
        {
          "image": "imagestreamname",
          "imagePullPolicy": "Always",
          "name": "container1",
          "resources": {},
          "terminationMessagePath": "/dev/termination-log",
          "terminationMessagePolicy": "File"
        }
      ],
      "dnsPolicy": "ClusterFirst",
      "restartPolicy": "Always",
      "schedulerName": "default-scheduler",
      "securityContext": {},
      "terminationGracePeriodSeconds": 30
    }
  }
}
[root@localhost roottest]# vi /tmp/is.yaml
[root@localhost roottest]# oc create -f /tmp/is.yaml 
imagestream.image.openshift.io/imagestreamname created
[root@localhost roottest]# oc get is
NAME              IMAGE REPOSITORY                                                         TAGS   UPDATED
imagestreamname   image-registry.openshift-image-registry.svc:5000/zhouy/imagestreamname          
[root@localhost roottest]#  oc get imagestream.image.openshift.io/imagestreamname -o json | jq '.spec'
{
  "lookupPolicy": {
    "local": true
  }
}
[root@localhost roottest]# oc get rs
NAME                          DESIRED   CURRENT   READY   AGE
i-spawn-replicas-84778cc586   0         0         0       8m37s
[root@localhost roottest]# vi /tmp/patch.yaml
[root@localhost roottest]# oc patch deployment i-spawn-replicas --type merge --patch "$(cat /tmp/patch.yaml)"
deployment.apps/i-spawn-replicas patched
[root@localhost roottest]# oc get rs |wc -l
3
[root@localhost roottest]# oc get rs |wc -l
3
[root@localhost roottest]# oc get rs |wc -l
3
[root@localhost roottest]# oc get rs |wc -l
3
Comment 6 errata-xmlrpc 2021-08-10 11:27:39 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.4 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2983