Bug 1473370

Summary: ResourceQuota controller observed making excessive LIST calls at scale
Product: OpenShift Container Platform Reporter: Eric Paris <eparis>
Component: NodeAssignee: Derek Carr <decarr>
Status: CLOSED ERRATA QA Contact: DeShuai Ma <dma>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: aos-bugs, decarr, dma, jokerman, mmccomas, xtian
Target Milestone: ---   
Target Release: 3.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: too many LIST calls were made by resource quota controller Consequence: this can impact performance of clusters at largest scales. Fix: reduced LIST calls made by resource quota controller by using shared informer caches. Result: LIST operations made to the master were reduced and information was pulled from a shared cache in the controller.
 Story Points: ---
Clone Of: 1473042 Environment:
Last Closed: 2017-11-28 22:04:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1473042    
Bug Blocks:    

Comment 2 DeShuai Ma 2017-09-28 06:36:11 UTC 
Do some regression on openshift v3.7.0-0.127.0, quota works well, verify the bug

[root@ip-172-18-9-60 ~]# oc create quota quota --hard=services=10,secrets=10
resourcequota "quota" created
[root@ip-172-18-9-60 ~]# oc get quota
NAME      AGE
quota     8s
[root@ip-172-18-9-60 ~]# oc describe quota
Name:		quota
Namespace:	dma
Resource	Used	Hard
--------	----	----
secrets		9	10
services	0	10
[root@ip-172-18-9-60 ~]# oc get secrets
NAME                       TYPE                                  DATA      AGE
builder-dockercfg-rqbbz    kubernetes.io/dockercfg               1         40s
builder-token-4f1w2        kubernetes.io/service-account-token   4         40s
builder-token-vtnk7        kubernetes.io/service-account-token   4         40s
default-dockercfg-8kmtq    kubernetes.io/dockercfg               1         40s
default-token-02cf2        kubernetes.io/service-account-token   4         40s
default-token-kftvq        kubernetes.io/service-account-token   4         40s
deployer-dockercfg-p008f   kubernetes.io/dockercfg               1         40s
deployer-token-g3jqd       kubernetes.io/service-account-token   4         40s
deployer-token-lggtl       kubernetes.io/service-account-token   4         40s
[root@ip-172-18-9-60 ~]# oc get svc | wc -l
No resources found.
0
[root@ip-172-18-9-60 ~]# oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/services/ExternalSvc.yaml
service "my-svc" created
[root@ip-172-18-9-60 ~]# oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/secrets/secret.yaml
secret "test-secret" created
[root@ip-172-18-9-60 ~]# oc get secrets | grep -v NAME | wc -l
10
[root@ip-172-18-9-60 ~]# oc get svc |grep -v NAME | wc -l
1
[root@ip-172-18-9-60 ~]# oc describe quota
Name:		quota
Namespace:	dma
Resource	Used	Hard
--------	----	----
secrets		10	10
services	1	10
[root@ip-172-18-9-60 ~]# oc create -f https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/secrets/secret-datastring-image.json
Error from server (Forbidden): error when creating "https://raw.githubusercontent.com/openshift-qe/v3-testfiles/master/secrets/secret-datastring-image.json": secrets "secret-datastring-image" is forbidden: exceeded quota: quota, requested: secrets=1, used: secrets=10, limited: secrets=10
Comment 5 errata-xmlrpc 2017-11-28 22:04:10 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2017:3188