Description of problem: kube-controller-manager is unable to locate the credentials of the vCenter host, even though they are defined and cased identically to the hostname in the cloud-provider-config. Version-Release number of selected component (if applicable): 4.2.20 UPI on VMWare How reproducible: Consistently Steps to Reproduce: 1. Attempt to provision storage with the "thin" storage class 2. PVC will stay suck in pending 3. Actual results: vCenter hostname is case-sensitive in a way that is opaque to administrators Expected results: vCenter hostname should not be case-sensitive, or should be documented that it is. Additional info: Log Messages: E0318 18:58:01.956856 1 connection.go:65] Failed to create govmomi client. err: ServerFaultCode: Cannot complete login due to an incorrect user name or password. E0318 18:58:01.956888 1 credentialmanager.go:79] credentials not found for server "VCENTER.host.local" E0318 18:58:01.956899 1 nodemanager.go:404] Failed to get credentials from Secret Credential Manager with err: Credentials not found E0318 18:58:01.956907 1 vsphere.go:616] failed connecting to vcServer "VCENTER.host.local" with error Credentials not found I0318 18:58:01.956947 1 pv_controller.go:1492] failed to provision volume for claim "openshift-monitoring/openshift-monitoring-alertman-pvc-alertmanager-main-2" with StorageClass "thin": Credentials not found E0318 18:58:01.956980 1 goroutinemap.go:150] Operation for "provision-openshift-monitoring/openshift-monitoring-alertman-pvc-alertmanager-main-2[9c8615ed-6916-11ea-a483-005056184020]" failed. No retries permitted until 2020-03-18 19:00:03.956966958 +0000 UTC m=+113504.770261005 (durationBeforeRetry 2m2s). Error: "Credentials not found" This appears to be induced by the way the vCenter server name is calculated when the secret containing the credentials is parsed[https://github.com/kubernetes/kubernetes/blob/release-1.15/staging/src/k8s.io/legacy-cloud-providers/vsphere/credentialmanager.go#L138]. The credentialKey is converted to lower case and is then parsed to lookup the server name.
These bits are handled by the cloud team, moving accordingly.
PR to fix: https://github.com/kubernetes/kubernetes/pull/90796
QE do not have a vSphere with vCenter hostname in uppercase, therefore unit tested this. --- PASS: TestParseSecretConfig (0.00s) credentialmanager_test.go:372: Executing Testcase: Valid username and password credentialmanager_test.go:372: Executing Testcase: Invalid username key with valid password key credentialmanager_test.go:372: Executing Testcase: Missing username credentialmanager_test.go:372: Executing Testcase: Missing password credentialmanager_test.go:372: Executing Testcase: FQIN stored as lowercase credentialmanager_test.go:372: Executing Testcase: IP with unknown key PASS ok k8s.io/legacy-cloud-providers/vsphere 0.404s
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:2409