Description of problem: Autoscaler log report error “E0618 07:23:15.255288 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:serviceaccount:openshift-machine-api:cluster-autoscaler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope” Version-Release number of selected component (if applicable): 4.8.0-0.nightly-2021-06-14-145150 How reproducible: always Steps to Reproduce: 1. Create a clusterautoscaler 2. Create a machineautoscaler 3. Add workload 4. Check autoscaler logs Actual results: Autoscale logs always output error msg: oc logs -f cluster-autoscaler-default-75c55cf9d7-kwtt8 I0618 06:25:05.288288 1 klogx.go:86] Pod openshift-machine-api/scale-up-6cc4bdd5db-69c8z is unschedulable I0618 06:25:05.289389 1 klogx.go:86] Pod openshift-machine-api/scale-up-6cc4bdd5db-jlrxc is unschedulable I0618 06:25:05.290411 1 klogx.go:86] Pod openshift-machine-api/scale-up-6cc4bdd5db-shjtv is unschedulable I0618 06:25:05.294384 1 scale_up.go:453] No expansion options I0618 06:25:05.297314 1 scale_down.go:917] No candidates for scale down E0618 06:25:07.144282 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.CSIDriver: failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User "system:serviceaccount:openshift-machine-api:cluster-autoscaler" cannot list resource "csidrivers" in API group "storage.k8s.io" at the cluster scope I0618 06:25:15.454335 1 klogx.go:86] Pod openshift-machine-api/scale-up-6cc4bdd5db-shjtv is unschedulable I0618 06:25:15.454660 1 klogx.go:86] Pod openshift-machine-api/scale-up-6cc4bdd5db-svp5v is unschedulable Expected results: Autoscaler logs doesn’t have such error msgs. Additional info:
i think we just need to update the role for the machine-api service account. i am starting to investigate.
Failed to verify oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.9.0-0.nightly-2021-06-24-082405 True False 61m Cluster version is 4.9.0-0.nightly-2021-06-24-082405 E0624 16:04:42.548972 1 reflector.go:138] k8s.io/client-go/informers/factory.go:134: Failed to watch *v1beta1.CSIStorageCapacity: failed to list *v1beta1.CSIStorageCapacity: csistoragecapacities.storage.k8s.io is forbidden: User "system:serviceaccount:openshift-machine-api:cluster-autoscaler" cannot list resource "csistoragecapacities" in API group "storage.k8s.io" at the cluster scope I0624 16:04:46.391301 1 static_autoscaler.go:319] 2 unregistered nodes present
verified oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.9.0-0.nightly-2021-06-27-223612 True False 3m43s Cluster version is 4.9.0-0.nightly-2021-06-27-223612 I0628 08:53:12.609381 1 scale_down.go:917] No candidates for scale down I0628 08:53:22.628438 1 static_autoscaler.go:401] No unschedulable pods I0628 08:53:22.628875 1 scale_down.go:917] No candidates for scale down W0628 08:53:32.712601 1 clusterstate.go:432] AcceptableRanges have not been populated yet. Skip checking I0628 08:53:33.288638 1 static_autoscaler.go:401] No unschedulable pods I0628 08:53:33.691161 1 pre_filtering_processor.go:66] Skipping ip-10-0-242-215.us-east-2.compute.internal - node group min size reached I0628 08:53:34.088723 1 scale_down.go:917] No candidates for scale down I0628 08:53:44.909621 1 klogx.go:86] Pod openshift-machine-api/scale-up-6cc4bdd5db-2k59x is unschedulable
@ancollin i have created https://bugzilla.redhat.com/show_bug.cgi?id=1995595 and am working on the backports
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759