Red Hat Bugzilla – Bug 1477601
Two secrets triggered quota in Starter tier
Last modified: 2017-08-16 10:02:10 EDT
Description of problem:
On deploying an application in the Starter tier, my app was configured to use two custom secrets. The second secret triggered a quota violation and I was unable to create:
Error from server (Forbidden): secrets "secret2" is forbidden: exceeded quota: object-counts, requested: secrets=1, used: secrets=20, limited: secrets=20
However at the time I saw only these (7) secrets in my project:
$ oc get secrets
NAME TYPE DATA AGE
builder-token-6zp7m kubernetes.io/service-account-token 4 25s
builder-token-xj3mp kubernetes.io/service-account-token 0 25s
default-token-40m18 kubernetes.io/service-account-token 0 25s
default-token-hm93d kubernetes.io/service-account-token 4 25s
deployer-token-kk97n kubernetes.io/service-account-token 4 25s
deployer-token-ncqxb kubernetes.io/service-account-token 0 25s
sshsecret kubernetes.io/ssh-auth 1 21s
Even more strangely, in the web console my quota page showed 12 of 20 used.
Version-Release number of selected component (if applicable):
$ oc version
features: Basic-Auth GSSAPI Kerberos SPNEGO
May be difficult to reproduce, at the time I could reproduce it, today I can create additional secrets without issue.
Steps to Reproduce:
1. Start new project.
2. Create a secret.
3. Create another secret.
Second custom secret was rejected for quota violation.
If a quota of 20 secrets, and we get 6 automatically for the default service accounts, I would expect to be able to create 14 for my project.
during the initial project creation, it is possible for the automatically created secrets to be retried by the secret-creating controller.
if that occurrs, quota can temporarily indicate higher objects counts in use. on reconciliation, the in-use number reduces to match what is actually being used. that can take anywhere from a few seconds to a few minutes, depending on cluster size.
can you verify that quota indicates the correct number in use now?
Count is now correct. I also may have seen some indication it was related to deleting the project, then recreating it, if that is any help. I tend to do that when working on a template to clean everything up and re-try. In any case the issue does seem to go away at some point after it's hit, shortly after creating or recreating a project.
This is working as design, with temporary overallocation possible and reconciliation coming behind and recalculating quota periodically