We've found a layering violation between the openshift and kube control planes. SCC requires annotations on namespaces to set default UIDs to create pods and clusterresourcequota (CRQ) requires reconciliation to free quota to create pods. The controllers which do these things are in the openshift-controller-manager even though they have no logical openshift dependency. In 4.1, we partially fixed this by creating these resources as CRDs so they were always available, but we missed the controllers that are responsible for keeping these resources functional inside of the cluster. We need to pull the "openshift.io/namespace-security-allocation" and "openshift.io/cluster-quota-reconciliation" controllers into a spot above the openshift-apiserver so that our platform can continue to create pods even if part of the openshift-control-plane is down. Best option known option: new image used in a new container in the existing kube-controller-manager static pod. This gives us resiliency during disaster recovery that a normal pod would not provide us. It doesn't require a new operator or a change to topology and it does not complicate a rebase.
static pod def, rbac: https://github.com/openshift/cluster-kube-controller-manager-operator/pull/297 temporary lock NamespaceSCCAllocationController: https://github.com/openshift/openshift-controller-manager/pull/28 temporary lock ClusterPolicyController: https://github.com/openshift/cluster-policy-controller/pull/3 CI https://github.com/openshift/release/pull/5075 remove quota,sec controllers from OCM: https://github.com/openshift/openshift-controller-manager/pull/37 These changes are merged, setting to MODIFIED to be picked up by QA
What have strong CI on this. Marking verified to free up our bot.
The bug should only be moved to VERIFIED by QE.
Referenced PR for validation by QE https://github.com/openshift/openshift-controller-manager/pull/37
Bugs should never move from MODIFIED->VERIFIED. Bugs must move from MODIFIED->ON_QA->VERIFIED via ART automation in order to not cause other confusion in a release.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0062
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days