Bug 1738368 (CVE-2019-11248)

Summary:	CVE-2019-11248 kubernetes: /debug/pprof endpoint exposed on kubelet's healthz port
Product:	[Other] Security Response	Reporter:	Sam Fowler <sfowler>
Component:	vulnerability	Assignee:	Red Hat Product Security <security-response-team>
Status:	CLOSED ERRATA	QA Contact:
Severity:	medium	Docs Contact:
Priority:	medium
Version:	unspecified	CC:	ahardin, bleanhar, bmontgom, ccoleman, dedgar, eparis, go-sig, hchiramm, ichavero, jbrooks, jburrell, jcajka, jchaloup, jgoulding, jmulligan, jokerman, kramdoss, madam, mchappel, nstielau, rhs-bugs, sisharma, sponnaga, strigazi, tstclair
Target Milestone:	---	Keywords:	Security
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	kubernetes 1.15.0, kubernetes 1.14.4, kubernetes 1.13.8, kubernetes 1.12.10	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-10-27 10:47:35 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1738369, 1738417, 1738418
Bug Blocks:	1738370

Description Sam Fowler 2019-08-07 01:26:00 UTC

The Kubernetes debugging endpoint /debug/pprof is exposed over the unauthenticated Kubelet healthz port. Versions prior to 1.15.0, 1.14.4, 1.13.8, and 1.12.10 are affected.The issue is of medium severity, but only exposed locally by the default configuration.

By default, the Kubelet exposes unauthenticated healthz endpoints on port :10248, but only over localhost. If your nodes are using a non-localhost healthzBindAddress (--health-bind-address), and an older version, you may be vulnerable. If your nodes are using the default localhost healthzBindAddress, it is only exposed to pods or processes running in the host network namespace.

The `go pprof` endpoint is exposed over the Kubelet's healthz port. This debugging endpoint can potentially leak sensitive information such as internal Kubelet memory addresses and configuration, or for limited denial of service.


Upstream Issue:

https://github.com/kubernetes/kubernetes/issues/81023


External References:

https://groups.google.com/forum/#!topic/kubernetes-security-announce/pKELclHIov8

Comment 1 Sam Fowler 2019-08-07 01:27:25 UTC

Created kubernetes tracking bugs for this issue:

Affects: fedora-all [bug 1738369]

Comment 2 Sam Fowler 2019-08-07 04:37:13 UTC

The kubelet healthz endpoint is disabled by default in OCP 3:

https://github.com/openshift/origin/blob/release-3.11/pkg/cmd/server/kubernetes/node/options/options.go#L52

This can be confirmed by following the upstream advice to check the value of healthzBindAddress and healthzPort:

$ kubectl get --raw /api/v1/nodes/${NODE_NAME}/proxy/configz | jq -r '.kubeletconfig.healthzBindAddress, .kubeletconfig.healthzPort'
127.0.0.1
0

When, healthzPort is set to 0, the healthz endpoint is disabled:

https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/

Comment 4 Sam Fowler 2019-08-07 06:50:16 UTC

The kubelet healthz server is enabled by default on localhost in OCP 4:

$ kubectl get --raw /api/v1/nodes/${NODE_NAME}/proxy/configz | jq -r '.kubeletconfig.healthzBindAddress, .kubeletconfig.healthzPort'
127.0.0.1
10248

The /debug/pprof endpoint is enabled at localhost:10248 on all nodes and provides an interface for runtime profiling:

https://golang.org/pkg/net/http/pprof/

Comment 5 Sam Fowler 2019-08-07 06:50:23 UTC

Statement:

OpenShift Container Platform 3 is not vulnerable to this flaw as the kubelet healthz server is disabled by default. OpenShift Container Platform 4 enables the /debug/pprof endpoint on the kubelet healthz server to local traffic only.