Bug 1824481

Description of problem:

When the password is changed in vSphere for the user that is configured for the OpenShift cluster, storage can still be provisioned despite having a password that is no longer valid. 


Version-Release number of selected component (if applicable):

Client Version: 4.3.9
Server Version: 4.3.9
Kubernetes Version: v1.16.2


How reproducible:

Always

Steps to Reproduce:

1. Deploy an OCP cluster with vSphere integration. Check that dynamically provisioning storage works.
2. Change the user's pasword in vSphere. 
3. Provision a new PVC, it will be created.

Actual results:

 The PVCs are provisioned.

Expected results:

 PVC provisioning should fail with "Cannot complete login due to an incorrect user name or password."

Additional info:

 Even 24 hours after the password change in vSphere, OCP can still provision storage. How long is the session kept open with vSphere after the first successful login?

 Changing the password in the "vsphere-creds" secret does not make any change. You can change it to another password that is not correct, and storage will still be provisioned.

 Manually deleting the kube-apiserver, kube-controller-manager pods openshift, apiserver pods, and controller-manager pods does not make a difference. 

 Only by modifying or re-creating the "cloud-provider-config" triggers the recreation of several pods (79 in my test cluster), and then OpenShift starts using the new credentials in the "vsphere-creds" secret.