Description of problem:

This issue has the same symptoms as those described in https://bugzilla.redhat.com/show_bug.cgi?id=1731827 which was released as an errata fix.


Version-Release number of selected component (if applicable):

4.1.11


How reproducible:

Not exactly sure how to reproduce -

1. Install OpenShift 4.1.11. 
2. Wait a while.
3. View endpoints in GUI and try to curl endpoints.


Actual results:

500 internal server error in GUI (see private screenshot).

Also when curling endpoint:

➜  ~ curl -k -H "Authorization: Bearer $(oc sa get-token prometheus-k8s -n openshift-monitoring)"  https://172.31.63.40:10250/metrics
An error has occurred during metrics collection:

3 error(s) occurred:
* collected metric kubelet_container_log_filesystem_used_bytes label:<name:"container" value:"etcd-metrics" > label:<name:"namespace" value:"openshift-etcd" > label:<name:"pod" value:"etcd-member-control-plane-0.sandbox.redacted.com" > label:<name:"uid" value:"cfadc2d331e5127585aa2bbeff5e1f5d" > gauge:<value:4096 >  was collected before with the same name and label values
* collected metric kubelet_container_log_filesystem_used_bytes label:<name:"container" value:"dns" > label:<name:"namespace" value:"openshift-dns" > label:<name:"pod" value:"dns-default-ck872" > label:<name:"uid" value:"bfb3c3eb-ca74-11e9-8efb-0050569b68ae" > gauge:<value:8192 >  was collected before with the same name and label values
* collected metric kubelet_container_log_filesystem_used_bytes label:<name:"container" value:"openshift-apiserver" > label:<name:"namespace" value:"openshift-apiserver" > label:<name:"pod" value:"apiserver-l57n6" > label:<name:"uid" value:"55c03e12-cb27-11e9-aa4b-0050569b5be3" > gauge:<value:901120 >  was collected before with the same name and label values


Expected results:

Endpoints should be up and available.


Additional info: