Bug 1821945

Summary:	vCenter hostname must be all lower case
Product:	OpenShift Container Platform	Reporter:	rvanderp
Component:	Cloud Compute	Assignee:	Danil Grigorev <dgrigore>
Cloud Compute sub component:	Other Providers	QA Contact:	Jianwei Hou <jhou>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	low
Priority:	unspecified	CC:	agarcial, aos-bugs, mfojtik
Version:	4.5
Target Milestone:	---
Target Release:	4.5.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-07-13 17:26:08 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description rvanderp 2020-04-07 22:17:09 UTC

Description of problem:
kube-controller-manager is unable to locate the credentials of the vCenter host, even though they are defined and cased identically to the hostname in the cloud-provider-config.

Version-Release number of selected component (if applicable):
4.2.20 UPI on VMWare

How reproducible:
Consistently

Steps to Reproduce:
1. Attempt to provision storage with the "thin" storage class
2. PVC will stay suck in pending
3.

Actual results:
vCenter hostname is case-sensitive in a way that is opaque to administrators

Expected results:
vCenter hostname should not be case-sensitive, or should be documented that it is.

Additional info:

Log Messages:
E0318 18:58:01.956856 1 connection.go:65] Failed to create govmomi client. err: ServerFaultCode: Cannot complete login due to an incorrect user name or password.
E0318 18:58:01.956888 1 credentialmanager.go:79] credentials not found for server "VCENTER.host.local"
E0318 18:58:01.956899 1 nodemanager.go:404] Failed to get credentials from Secret Credential Manager with err: Credentials not found
E0318 18:58:01.956907 1 vsphere.go:616] failed connecting to vcServer "VCENTER.host.local" with error Credentials not found
I0318 18:58:01.956947 1 pv_controller.go:1492] failed to provision volume for claim "openshift-monitoring/openshift-monitoring-alertman-pvc-alertmanager-main-2" with StorageClass "thin": Credentials not found
E0318 18:58:01.956980 1 goroutinemap.go:150] Operation for "provision-openshift-monitoring/openshift-monitoring-alertman-pvc-alertmanager-main-2[9c8615ed-6916-11ea-a483-005056184020]" failed. No retries permitted until 2020-03-18 19:00:03.956966958 +0000 UTC m=+113504.770261005 (durationBeforeRetry 2m2s). Error: "Credentials not found"

This appears to be induced by the way the vCenter server name is calculated when the secret containing the credentials is parsed[https://github.com/kubernetes/kubernetes/blob/release-1.15/staging/src/k8s.io/legacy-cloud-providers/vsphere/credentialmanager.go#L138]. The credentialKey is converted to lower case and is then parsed to lookup the server name.

Comment 1 Maciej Szulik 2020-04-08 09:12:28 UTC

These bits are handled by the cloud team, moving accordingly.

Comment 2 Danil Grigorev 2020-05-06 11:36:35 UTC

PR to fix: https://github.com/kubernetes/kubernetes/pull/90796

Comment 6 Jianwei Hou 2020-05-20 06:26:52 UTC

QE do not have a vSphere with vCenter hostname in uppercase, therefore unit tested this.

--- PASS: TestParseSecretConfig (0.00s)
    credentialmanager_test.go:372: Executing Testcase: Valid username and password
    credentialmanager_test.go:372: Executing Testcase: Invalid username key with valid password key
    credentialmanager_test.go:372: Executing Testcase: Missing username
    credentialmanager_test.go:372: Executing Testcase: Missing password
    credentialmanager_test.go:372: Executing Testcase: FQIN stored as lowercase
    credentialmanager_test.go:372: Executing Testcase: IP with unknown key
PASS
ok  	k8s.io/legacy-cloud-providers/vsphere	0.404s

Comment 7 errata-xmlrpc 2020-07-13 17:26:08 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409