Description of problem: After deploying gluster block provisioner pod these logs are visible in the logs : --snip-- E0624 11:21:14.159454 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched E0624 11:21:17.668742 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched E0624 11:21:20.904421 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched E0624 11:21:23.082461 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched E0624 11:21:26.824756 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched E0624 11:21:30.945627 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched E0624 11:21:35.296369 1 leaderelection.go:286] Failed to update lock: endpoints "gluster.org-glusterblock" is forbidden: User "system:serviceaccount:glusterfs:glusterblock-storage-provisioner" cannot update endpoints in the namespace "glusterfs": no RBAC policy matched --/snip-- The RCA is that, the cluster role attached to this pod does not carry the permissions for updating endpoints. If I add below permissions to the subjected cluster role, provisioner works perfectly. Resources: endpoints Verbs: create delete get list update watch Version-Release number of selected component (if applicable): OCS 3.11.4 builds. How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
After updating the role for 'endpoint' update, I was able to provision the volumes successfully. I0624 11:21:45.478908 1 controller.go:1026] provision "glusterfs/blockvolume" class "glusterfs-storage-block": volume "pvc-2b355dc4-9671-11e9-9539-005056b26dc1" provisioned I0624 11:21:45.478950 1 controller.go:1040] provision "glusterfs/blockvolume" class "glusterfs-storage-block": trying to save persistentvolume "pvc-2b355dc4-9671-11e9-9539-005056b26dc1" I0624 11:21:45.591400 1 controller.go:1047] provision "glusterfs/blockvolume" class "glusterfs-storage-block": persistentvolume "pvc-2b355dc4-9671-11e9-9539-005056b26dc1" saved I0624 11:21:45.591436 1 controller.go:1088] provision "glusterfs/blockvolume" class "glusterfs-storage-block": succeeded
This PR in upstream caused this change: https://github.com/kubernetes-incubator/external-storage/commit/8e3bfd30818c65efdcca497780417fca1a88925e So, we have to accommodate the changes in our deployment based on or considering "existing OCS" deployments .
Verified in build openshift-ansible-3.11.129-1.git.0.11838de.el7.noarch and i see that i am able to create a blockvolume with out any issues. I also see the following change in the file /usr/share/ansible/openshift-ansible/roles/openshift_storage_glusterfs/files/glusterblock-provisioner.yml due to which the bug was raised. - apiGroups: [""] resources: ["endpoints"] verbs: ["get", "list", "watch", "create", "update", "patch"] Based on the above comments moving the bug to verified state.