Bug 1777137

Summary:	OpenShift Authentication pods crash when badly formatted certificate is provided
Product:	OpenShift Container Platform	Reporter:	Robert Bost <rbost>
Component:	apiserver-auth	Assignee:	Standa Laznicka <slaznick>
Status:	CLOSED ERRATA	QA Contact:	pmali
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.2.z	CC:	aos-bugs, mfojtik, nagrawal, scheng, slaznick, sttts, xxia
Target Milestone:	---
Target Release:	4.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	4.5
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: Lack of validation of CMs/secrets might cause the authentication operator to rollout a new deployment of oauth-server with invalid/missing files. Consequence: oauth-server pods won't be able to run properly and might crash. Fix: Make the authentication operator validate the input CMs and secrets. Result: New deployment should not get rolled out when the CMs and secrets referenced in the oauth/cluster configuration contain invalid data.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2020-10-27 15:54:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Robert Bost 2019-11-27 01:50:29 UTC

Description of problem:

The following error was seen in oauth pod logs after configuring a new idp client keypair. I would expect the authentication-operator to check formatting before proprogating to Pods:

panic: Error building BasicAuthPasswordIdentityProvider client: error loading x509 keypair from cert file /var/config/user/idp/0/secret/v4-0-config-user-idp-0-tls-client-cert/tls.crt and key file /var/config/user/idp/0/secret/v4-0-config-user-idp-0-tls-client-key/tls.key: tls: failed to find any PEM data in certificate input

goroutine 1 [running]:
github.com/openshift/oauth-server/pkg/oauthserver.(*OAuthServerConfig).buildHandlerChainForOAuth(0xc000461500, 0x1be6b00, 0xc0003a03e0, 0xc0002a8c40, 0x16b57a0, 0xc0005e9638)
	/go/src/github.com/openshift/oauth-server/pkg/oauthserver/oauth_apiserver.go:303 +0xee
github.com/openshift/oauth-server/vendor/k8s.io/apiserver/pkg/server.completedConfig.New.func1(0x1be6b00, 0xc0003a03e0, 0x1be6b00, 0xc0003a03e0)
	/go/src/github.com/openshift/oauth-server/vendor/k8s.io/apiserver/pkg/server/config.go:444 +0x45
github.com/openshift/oauth-server/vendor/k8s.io/apiserver/pkg/server.NewAPIServerHandler(0x18dc66b, 0xf, 0x1c1b7c0, 0xc0004bbec0, 0xc0003a0380, 0x0, 0x0, 0x0)
	/go/src/github.com/openshift/oauth-server/vendor/k8s.io/apiserver/pkg/server/handler.go:96 +0x2fc
github.com/openshift/oauth-server/vendor/k8s.io/apiserver/pkg/server.completedConfig.New(0xc0002a8c40, 0x0, 0x0, 0x18dc66b, 0xf, 0x1c378a0, 0x2bc4f80, 0xc0002a8c40, 0x0, 0x0)
	/go/src/github.com/openshift/oauth-server/vendor/k8s.io/apiserver/pkg/server/config.go:446 +0x124
github.com/openshift/oauth-server/pkg/oauthserver.completedOAuthConfig.New(0xc0003a0360, 0xc000461508, 0x1c378a0, 0x2bc4f80, 0x4, 0x1c00dc0, 0xc000610e00)
	/go/src/github.com/openshift/oauth-server/pkg/oauthserver/oauth_apiserver.go:286 +0x70
github.com/openshift/oauth-server/pkg/cmd/oauth-server.RunOsinServer(0xc000258600, 0xc00009c720, 0xd07, 0xf07)
	/go/src/github.com/openshift/oauth-server/pkg/cmd/oauth-server/server.go:40 +0x8a
github.com/openshift/oauth-server/pkg/cmd/oauth-server.(*OsinServer).RunOsinServer(0xc00048ce80, 0xc00009c720, 0x5ac320, 0x16a1ce0)
	/go/src/github.com/openshift/oauth-server/pkg/cmd/oauth-server/cmd.go:91 +0x35c
github.com/openshift/oauth-server/pkg/cmd/oauth-server.NewOsinServer.func1(0xc0001fc500, 0xc0004a4a40, 0x0, 0x2)
	/go/src/github.com/openshift/oauth-server/pkg/cmd/oauth-server/cmd.go:39 +0xf4
github.com/openshift/oauth-server/vendor/github.com/spf13/cobra.(*Command).execute(0xc0001fc500, 0xc0004a49e0, 0x2, 0x2, 0xc0001fc500, 0xc0004a49e0)
	/go/src/github.com/openshift/oauth-server/vendor/github.com/spf13/cobra/command.go:760 +0x2ae
github.com/openshift/oauth-server/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc0001fc280, 0xc0001fc280, 0x0, 0x0)
	/go/src/github.com/openshift/oauth-server/vendor/github.com/spf13/cobra/command.go:846 +0x2ec
github.com/openshift/oauth-server/vendor/github.com/spf13/cobra.(*Command).Execute(...)
	/go/src/github.com/openshift/oauth-server/vendor/github.com/spf13/cobra/command.go:794
main.main()
	/go/src/github.com/openshift/oauth-server/cmd/oauth-server/main.go:41 +0x2cf


Version-Release number of selected component (if applicable): 4.2


How reproducible: always

Steps to Reproduce:
1. Create a secret with badly formatted certificate (i.e. missing header or footer lines
2. Operator will proceed and oauth-server pods will crash

Actual results: Crashing pods

Expected results: Operator to validate certificate

Comment 1 Standa Laznicka 2019-11-27 08:55:45 UTC

We are looking into improving input validation in 4.4 as a part of stabilization for the authentication operator. Until then, I'd suggest formatting your certificates properly.

Comment 13 errata-xmlrpc 2020-10-27 15:54:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196