Bug 1756417

Summary: Schema change in ClusterResourceQuota from 4.1 to 4.2 results in breakage
Product: OpenShift Container Platform Reporter: Greg Blomquist <gblomqui>
Component: openshift-apiserverAssignee: Stefan Schimanski <sttts>
Status: CLOSED ERRATA QA Contact: Xingxing Xia <xxia>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.2.0CC: aos-bugs, eparis, lseelye, mfojtik, sttts
Target Milestone: ---   
Target Release: 4.3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: ClusterResourceQuota in 4.2 and 4.3 do not allow non-strings as limit values because the OpenAPI schema is wrong. Consequence: Integer quota values cannot be set in ClusterResourceQuota objects although that was possible in 4.1. Fix: The OpenAPI schema for ClusterResourceQuota was fixed to allow integers. Result: Integers can be used as quota values in ClusterResourceQuota again.
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-05-13 21:25:44 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1755125    

Description Greg Blomquist 2019-09-27 15:07:42 UTC
This bug was initially created as a copy of Bug #1755125

I am copying this bug because: 

The original bug was tracking a change to 4.2.0 repo.

This copy is tracking the change to 4.3.0

 - https://github.com/openshift/cluster-config-operator/pull/92



Description of problem:

While using build 4.2.0-0.nightly-2019-09-17-212934 (verified later with 4.2.0-0.nightly-2019-09-23-154647) it was noticed that the schema appears to have changed for the ClusterResourceQuota object compared to 4.1.16. Specifically, spec.quota.hard.

The change appears to be a requirement that the value of that attribute be expressed as a string, when previously, an integer was acceptable. This causes a breakage for SRE on new OSD clusters created with a 4.2 image as the cluster quota configuration is applied (c.f., https://github.com/openshift/managed-cluster-config/blob/840d1c7dd7dfb268dbf678ff3ff8135540a4e1db/deploy/resource-quotas/15-resource.clusterresourcequota.yaml)

The behaviour of clusters upgraded from 4.1.16 to 4.2 is that for ClusterResourceQuota configurations which were already applied, they have no error.


Version-Release number of selected component (if applicable):

* 4.2.0-0.nightly-2019-09-17-212934
* 4.2.0-0.nightly-2019-09-23-154647

I don't know if this is apparent in earlier versions.

How reproducible:

Every time.


Steps to Reproduce:
1. Install a new cluster with a 4.2.0 nightly image and attempt to apply this manifest:

kind: ClusterResourceQuota
apiVersion: quota.openshift.io/v1
metadata:
  name: loadbalancer-quota
spec:
  selector:
    annotations: null
    labels:
      matchExpressions:
      - key: managed.openshift.io/service-lb-quota-exempt
        operator: DoesNotExist
  quota:
    hard:
      services.loadbalancers: 2


2. Observe error:
The ClusterResourceQuota "loadbalancer-quota" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"quota.openshift.io/v1", "kind":"ClusterResourceQuota", "metadata":map[string]interface {}{"annotations":map[string]interface {}{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"quota.openshift.io/v1\",\"kind\":\"ClusterResourceQuota\",\"metadata\":{\"annotations\":{},\"name\":\"loadbalancer-quota\"},\"spec\":{\"quota\":{\"hard\":{\"services.loadbalancers\":2}},\"selector\":{\"annotations\":null,\"labels\":{\"matchExpressions\":[{\"key\":\"managed.openshift.io/service-lb-quota-exempt\",\"operator\":\"DoesNotExist\"}]}}}}\n"}, "creationTimestamp":"2019-09-24T20:10:26Z", "generation":1, "name":"loadbalancer-quota", "uid":"58e43920-df07-11e9-942e-02f4d851fc82"}, "spec":map[string]interface {}{"quota":map[string]interface {}{"hard":map[string]interface {}{"services.loadbalancers":2}}, "selector":map[string]interface {}{"annotations":interface {}(nil), "labels":map[string]interface {}{"matchExpressions":[]interface {}{map[string]interface {}{"key":"managed.openshift.io/service-lb-quota-exempt", "operator":"DoesNotExist"}}}}}}: validation failure list:
spec.quota.hard.services.loadbalancers in body must be of type string: "integer"




Actual results:

spec.quota.hard.services.loadbalancers in body must be of type string: "integer"


Expected results:

clusterresourcequota.quota.openshift.io/loadbalancer-quota created


Additional info:

Comment 3 Xingxing Xia 2019-10-24 08:48:04 UTC
Was on other work, sorry delayed on this bug.
Tested in 4.3.0-0.nightly-2019-10-24-004924 env, still meet error:
$ oc create -f - << EOF
kind: ClusterResourceQuota
apiVersion: quota.openshift.io/v1
metadata:
  name: loadbalancer-quota
spec:
  selector:
    annotations: null
    labels:
      matchExpressions:
      - key: managed.openshift.io/service-lb-quota-exempt
        operator: DoesNotExist
  quota:
    hard:
      services.loadbalancers: 2
EOF
The ClusterResourceQuota "loadbalancer-quota" is invalid: spec.quota.hard.services.loadbalancers: Invalid value: "integer": spec.quota.hard.services.loadbalancers in body must be of type string: "integer"
[xxia@fedora29 my]$ oc version
Client Version: v4.3.0
Server Version: 4.3.0-0.nightly-2019-10-24-004924
Kubernetes Version: v1.16.0-beta.2+d34dc8a

Comment 4 Xingxing Xia 2019-10-24 09:32:35 UTC
In above env checked below, not sure why still having "additionalProperties.type: string":
$ oc rsh -n openshift-cluster-version deployments/cluster-version-operator
sh-4.2# ls /release-manifests
...
0000_03_quota-openshift_01_clusterresourcequota.crd.yaml
...
sh-4.2# cat /release-manifests/0000_03_quota-openshift_01_clusterresourcequota.crd.yaml
          properties:
            quota:
              description: Quota defines the desired quota
              properties:
                hard:
                  additionalProperties:
                    anyOf:
                    - type: string
                    - type: integer
                    type: string
                  description: 'hard is the set of desired hard limits for each named

Comment 5 Michal Fojtik 2019-11-06 20:07:48 UTC
Stefan, ping.

Comment 8 Xingxing Xia 2019-11-25 05:48:01 UTC
Verified in 4.3.0-0.nightly-2019-11-24-183610 env with above steps, the issue is fixed.

Comment 10 errata-xmlrpc 2020-05-13 21:25:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0062

Comment 11 Red Hat Bugzilla 2023-09-14 05:43:52 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days