Bug 1728754 - Instructions for adding default API server default certificate fails to configure serving cert
Summary: Instructions for adding default API server default certificate fails to confi...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-apiserver
Version: 4.1.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.1.z
Assignee: David Eads
QA Contact: Xingxing Xia
URL:
Whiteboard: 4.1.7
: 1711447 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-10 15:33 UTC by David Eads
Modified: 2019-11-12 15:26 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-25 05:32:32 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:1809 None None None 2019-07-25 05:32:37 UTC
Github openshift api pull 375 'None' 'closed' '[release-4.1] Bug 1728754: remove broken default serving cert setting' 2019-12-02 02:41:52 UTC
Github openshift cluster-kube-apiserver-operator pull 519 'None' 'closed' '[release-4.1] Bug 1728754: remove broken default serving cert' 2019-12-02 02:41:52 UTC

Description David Eads 2019-07-10 15:33:49 UTC
This bug was initially created as a copy of Bug #1720770

I am copying this bug because: 



Document URL: https://docs.openshift.com/container-platform/4.1/authentication/certificates/api-server.html#add-default-api-server_api-server-certificates

Section Number and Name: 

Describe the issue: 

Customer followed instructions for setting the default API server serving certificate.  The API server is not presenting the certificate and one of the kube-apiserver pods is stuck in CrashLoopBackoff.

Suggestions for improvement: 

Additional information: 

On the customer's cluster, they have 3 masters.  In the process of applying the change, the first instance of the kube-apiserver pod is stuck in CrashLoopBackoff.  Investigating the logs of the kube-apiserver container reveals that the container is unable to find the serving cert.  See stack trace below.  I can reproduce the same problem on my AWS cluster as well.

	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/hypershift/main.go:45 +0x2b8
E0614 15:04:19.542054       1 pathrecorder.go:107] registered "/readyz/crd-informer-synced" from goroutine 1 [running]:
runtime/debug.Stack(0x4e8a780, 0xc008619fb0, 0xc002860260)
	/opt/rh/go-toolset-1.11/root/usr/lib/go-toolset-1.11-golang/src/runtime/debug/stack.go:24 +0xa7
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).trackCallers(0xc001ece690, 0xc002860260, 0x1b)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:109 +0x89
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/mux.(*PathRecorderMux).Handle(0xc001ece690, 0xc002860260, 0x1b, 0xaa2ed40, 0xc00b8c3880)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/mux/pathrecorder.go:173 +0x86
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz.InstallPathHandler(0xaa27f80, 0xc001ece690, 0x5969c5c, 0x7, 0xc0056ede00, 0x1d, 0x1e)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz/healthz.go:120 +0x3c1
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server.(*GenericAPIServer).installReadyz(0xc00638edc0, 0xc0003bc6c0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/healthz.go:55 +0x1a8
github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server.preparedGenericAPIServer.Run(0xc00638edc0, 0xc0003bc6c0, 0xc00638edc0, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/apiserver/pkg/server/genericapiserver.go:290 +0x88
github.com/openshift/origin/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app.Run(0xc0001ff8c0, 0xc0003bc6c0, 0x0, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:153 +0x145
github.com/openshift/origin/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app.NewAPIServerCommand.func1(0xc001228780, 0x0, 0x0, 0x0, 0x1, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/cmd/kube-apiserver/app/server.go:115 +0x109
github.com/openshift/origin/pkg/cmd/openshift-kube-apiserver.RunOpenShiftKubeAPIServerServer(0xc001334800, 0xc0003bc6c0, 0x24, 0x7ffda2d38c27)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/openshift-kube-apiserver/server.go:70 +0x57e
github.com/openshift/origin/pkg/cmd/openshift-kube-apiserver.(*OpenShiftKubeAPIServerServer).RunAPIServer(0xc0003dde60, 0xc0003bc6c0, 0xc000aa38c0, 0xc000519800)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/openshift-kube-apiserver/cmd.go:129 +0x817
github.com/openshift/origin/pkg/cmd/openshift-kube-apiserver.NewOpenShiftKubeAPIServerServerCommand.func1(0xc00074b900, 0xc000394380, 0x0, 0x2)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/pkg/cmd/openshift-kube-apiserver/cmd.go:61 +0x10c
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).execute(0xc00074b900, 0xc0003942a0, 0x2, 0x2, 0xc00074b900, 0xc0003942a0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:760 +0x2cc
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc00074b180, 0xc001228000, 0xc00074bb80, 0xc00074b900)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:846 +0x2fd
github.com/openshift/origin/vendor/github.com/spf13/cobra.(*Command).Execute(0xc00074b180, 0xc00074b180, 0x0)
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/github.com/spf13/cobra/command.go:794 +0x2b
main.main()
	/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/cmd/hypershift/main.go:45 +0x2b8
F0614 15:04:19.543371       1 cmd.go:71] open /etc/kubernetes/static-pod-certs/secrets/user-serving-cert/tls.crt: no such file or directory

Comment 2 Xingxing Xia 2019-07-18 09:48:54 UTC
Verified in 4.1.0-0.nightly-2019-07-18-023612. Now defaultServingCertificate setting is removed (not allowed)
oc edit apiserver cluster
...
spec:
  servingCerts:
    defaultServingCertificate:
      name: some-secret
...

Attempted to save and exit, it failed and warned the setting is forbidden. And the Doc already removed it too.

Comment 3 Xingxing Xia 2019-07-18 10:03:54 UTC
Another bug 1731105 is reported after this.

Comment 5 errata-xmlrpc 2019-07-25 05:32:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1809

Comment 6 Luis Sanchez 2019-11-12 15:26:16 UTC
*** Bug 1711447 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.