Description of problem: When trying to create PV on vmware storage it fails with below error. *************** I0411 19:39:55.969270 29228 rbac.go:116] RBAC DENY: user "system:serviceaccount:kube-system:vsphere-cloud-provider" groups ["system:serviceaccounts" "system:serviceaccounts:kube-system" "system:authenticated"] cannot "list" resource "nodes" cluster-wide E0411 19:39:55.971008 29345 reflector.go:205] github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:227: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:vsphere-cloud-provider" cannot list nodes at the cluster scope: User "system:serviceaccount:kube-system:vsphere-cloud-provider" cannot list all nodes in the cluster *************** Version-Release number of selected component (if applicable): 3.9 How reproducible: Always Steps to Reproduce: 1. Configure OCP to access VMWare vSphere [1] 2. Try to create pv using vsphere storage [2] [1] https://docs.openshift.com/container-platform/3.9/install_config/configuring_vsphere.html [2] https://docs.openshift.com/container-platform/3.9/install_config/persistent_storage/persistent_storage_vsphere.html#install-config-persistent-storage-persistent-storage-vsphere Actual results: PV creation fails with following error. *************** I0411 19:39:55.969270 29228 rbac.go:116] RBAC DENY: user "system:serviceaccount:kube-system:vsphere-cloud-provider" groups ["system:serviceaccounts" "system:serviceaccounts:kube-system" "system:authenticated"] cannot "list" resource "nodes" cluster-wide E0411 19:39:55.971008 29345 reflector.go:205] github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:227: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:vsphere-cloud-provider" cannot list nodes at the cluster scope: User "system:serviceaccount:kube-system:vsphere-cloud-provider" cannot list all nodes in the cluster *************** The error occur because the required clusterrole for user vsphere-cloud-provider is missing in file /usr/share/ansible/openshift-ansible/roles/openshift_cloud_provider/files/vsphere-svc.yml. Expected results: vsphere-cloud-provider should be created with clusterrole and PV creation should be successful. Master Log: *************** I0411 19:39:55.969270 29228 rbac.go:116] RBAC DENY: user "system:serviceaccount:kube-system:vsphere-cloud-provider" groups ["system:serviceaccounts" "system:serviceaccounts:kube-system" "system:authenticated"] cannot "list" resource "nodes" cluster-wide E0411 19:39:55.971008 29345 reflector.go:205] github.com/openshift/origin/vendor/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:227: Failed to list *v1.Node: nodes is forbidden: User "system:serviceaccount:kube-system:vsphere-cloud-provider" cannot list nodes at the cluster scope: User "system:serviceaccount:kube-system:vsphere-cloud-provider" cannot list all nodes in the cluster *************** Node Log (of failed PODs): PV Dump: PVC Dump: StorageClass Dump (if StorageClass used by PV/PVC): Additional info: File /usr/share/ansible/openshift-ansible/roles/openshift_cloud_provider/files/vsphere-svc.yml do not have "kind: ClusterRole" entry after 'roleRef' in ClusterRoleBinding section. As temporary workaround, adding "kind: ClusterRole" entry after 'roleRef' in ClusterRoleBinding section helps to create PV. Example entries after changes. ************************************** apiVersion: v1 groupNames: null kind: ClusterRoleBinding metadata: annotations: openshift.io/reconcile-protect: "false" labels: kubernetes.io/bootstrapping: rbac-defaults name: system:vsphere-cloud-provider roleRef: kind: ClusterRole <------ Added entry name: system:vsphere-cloud-provider subjects: - kind: ServiceAccount name: vsphere-cloud-provider namespace: kube-system userNames: - system:serviceaccount:kube-system:vsphere-cloud-provider *************************************
https://github.com/openshift/ose/pull/1246
Verified on v3.9.30. vSphere VMDK could successfully provision without the need of sa/clusterrole/glusterrolebinding.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2018:1796