Bug 1698722

Summary:	Prometheus openshift-monitoring/kube-controller-manager target still uses 10252 after secure KCM is re-enabled
Product:	OpenShift Container Platform	Reporter:	Xingxing Xia <xxia>
Component:	Master	Assignee:	Michal Fojtik <mfojtik>
Status:	CLOSED DUPLICATE	QA Contact:	Xingxing Xia <xxia>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4.1.0	CC:	aos-bugs, jokerman, mmccomas
Target Milestone:	---
Target Release:	4.1.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2019-04-11 09:46:19 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Xingxing Xia 2019-04-11 05:51:54 UTC

Description of problem:
Prometheus openshift-monitoring/kube-controller-manager target still uses 10252 after secure KCM is re-enabled.
In prometheus UI, saw "down":
http://10.0.138.175:10252/metrics
    down     endpoint="https" instance="10.0.138.175:10252" job="kube-controller-manager" namespace="openshift-kube-controller-manager" pod="kube-controller-manager-ip-10-0-138-175.ap-northeast-2.compute.internal" service="kube-controller-manager"     12.407s ago     1.468ms     Get http://10.0.138.175:10252/metrics: dial tcp 10.0.138.175:10252: connect: connection refused
http://10.0.142.164:10252/metrics
    down     endpoint="https" instance="10.0.142.164:10252" job="kube-controller-manager" namespace="openshift-kube-controller-manager" pod="kube-controller-manager-ip-10-0-142-164.ap-northeast-2.compute.internal" service="kube-controller-manager"     12.677s ago     1.299ms     Get http://10.0.142.164:10252/metrics: dial tcp 10.0.142.164:10252: connect: connection refused
http://10.0.148.220:10252/metrics
    down     endpoint="https" instance="10.0.148.220:10252" job="kube-controller-manager" namespace="openshift-kube-controller-manager" pod="kube-controller-manager-ip-10-0-148-220.ap-northeast-2.compute.internal" service="kube-controller-manager"     21.547s ago     506.6us     Get http://10.0.148.220:10252/metrics: dial tcp 10.0.148.220:10252: connect: connection refused

Version-Release number of selected component (if applicable):
4.0.0-0.nightly-2019-04-10-182914

How reproducible:
Always

Steps to Reproduce:
1. Visit prometheus UI, go to Targets page

Actual results:
1. Saw above "down": http://file.rdu.redhat.com/~xxia/kube-controller-manager.png

Expected results:
1. Should use https://...:10257/metrics

Additional info:
Secure 10257 is re-enabled per below result:
$ oc rsh -n openshift-kube-controller-manager kube-controller-manager-ip-10-0-138-175.ap-northeast-2.compute.internal
sh-4.2# curl -k https://10.0.138.175:10257/healthz 
ok
sh-4.2# curl http://10.0.138.175:10252/healthz
curl: (7) Failed connect to 10.0.138.175:10252; Connection refused

Comment 1 Xingxing Xia 2019-04-11 09:46:19 UTC


*** This bug has been marked as a duplicate of bug 1698201 ***