Bug 1821945 - vCenter hostname must be all lower case
Summary: vCenter hostname must be all lower case
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cloud Compute
Version: 4.5
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.5.0
Assignee: Danil Grigorev
QA Contact: Jianwei Hou
Depends On:
TreeView+ depends on / blocked
Reported: 2020-04-07 22:17 UTC by rvanderp
Modified: 2020-07-13 17:26 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed: 2020-07-13 17:26:08 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github kubernetes kubernetes issues 90795 0 None closed vCenter hostname must be all lower case 2021-02-09 18:26:00 UTC
Github kubernetes kubernetes pull 90796 0 None closed Ensure the server FQIN is stored and searched in lowercase (vsphere) 2021-02-09 18:26:00 UTC
Github openshift origin pull 24973 0 None closed Bug 1821945: Ensure the server FQIN is stored and searched in lowercase (vsphere) 2021-02-09 18:26:00 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:26:21 UTC

Description rvanderp 2020-04-07 22:17:09 UTC
Description of problem:
kube-controller-manager is unable to locate the credentials of the vCenter host, even though they are defined and cased identically to the hostname in the cloud-provider-config.

Version-Release number of selected component (if applicable):
4.2.20 UPI on VMWare

How reproducible:

Steps to Reproduce:
1. Attempt to provision storage with the "thin" storage class
2. PVC will stay suck in pending

Actual results:
vCenter hostname is case-sensitive in a way that is opaque to administrators

Expected results:
vCenter hostname should not be case-sensitive, or should be documented that it is.

Additional info:

Log Messages:
E0318 18:58:01.956856       1 connection.go:65] Failed to create govmomi client. err: ServerFaultCode: Cannot complete login due to an incorrect user name or password.
E0318 18:58:01.956888       1 credentialmanager.go:79] credentials not found for server "VCENTER.host.local"
E0318 18:58:01.956899       1 nodemanager.go:404] Failed to get credentials from Secret Credential Manager with err: Credentials not found
E0318 18:58:01.956907       1 vsphere.go:616] failed connecting to vcServer "VCENTER.host.local" with error Credentials not found
I0318 18:58:01.956947       1 pv_controller.go:1492] failed to provision volume for claim "openshift-monitoring/openshift-monitoring-alertman-pvc-alertmanager-main-2" with StorageClass "thin": Credentials not found
E0318 18:58:01.956980       1 goroutinemap.go:150] Operation for "provision-openshift-monitoring/openshift-monitoring-alertman-pvc-alertmanager-main-2[9c8615ed-6916-11ea-a483-005056184020]" failed. No retries permitted until 2020-03-18 19:00:03.956966958 +0000 UTC m=+113504.770261005 (durationBeforeRetry 2m2s). Error: "Credentials not found"

This appears to be induced by the way the vCenter server name is calculated when the secret containing the credentials is parsed[https://github.com/kubernetes/kubernetes/blob/release-1.15/staging/src/k8s.io/legacy-cloud-providers/vsphere/credentialmanager.go#L138].  The credentialKey is converted to lower case and is then parsed to lookup the server name.

Comment 1 Maciej Szulik 2020-04-08 09:12:28 UTC
These bits are handled by the cloud team, moving accordingly.

Comment 2 Danil Grigorev 2020-05-06 11:36:35 UTC
PR to fix: https://github.com/kubernetes/kubernetes/pull/90796

Comment 6 Jianwei Hou 2020-05-20 06:26:52 UTC
QE do not have a vSphere with vCenter hostname in uppercase, therefore unit tested this.

--- PASS: TestParseSecretConfig (0.00s)
    credentialmanager_test.go:372: Executing Testcase: Valid username and password
    credentialmanager_test.go:372: Executing Testcase: Invalid username key with valid password key
    credentialmanager_test.go:372: Executing Testcase: Missing username
    credentialmanager_test.go:372: Executing Testcase: Missing password
    credentialmanager_test.go:372: Executing Testcase: FQIN stored as lowercase
    credentialmanager_test.go:372: Executing Testcase: IP with unknown key
ok  	k8s.io/legacy-cloud-providers/vsphere	0.404s

Comment 7 errata-xmlrpc 2020-07-13 17:26:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.