Bug 1366365

Summary:	Heapster "Failed to find node" warning and verbose logging
Product:	OpenShift Container Platform	Reporter:	Matt Wringe <mwringe>
Component:	Hawkular	Assignee:	Matt Wringe <mwringe>
Status:	CLOSED ERRATA	QA Contact:	chunchen <chunchen>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.3.0	CC:	aos-bugs, tdawson, wsun
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:
Clones:	1366367 (view as bug list)		Environment:
Last Closed:	2016-09-27 09:43:49 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1366367

Description Matt Wringe 2016-08-11 18:44:47 UTC

Description of problem:
When running Heapster, the logs will fill up with warning messages about "Failed to find node" as well as more verbose messages about each and every invocation to the nodes and how many metrics were returned.

The warning about "Failed to find node" is confusing and has nothing to do with Heapster being able to find or connect to OpenShift nodes.

These warning messages have been fixed upstream by changing the message and severity. See https://github.com/kubernetes/heapster/commit/c192564d3b551a2a632c067a9121fc6d239ebe24

We should backport this change into our images. 


Version-Release number of selected component (if applicable):
3.3.0, 3.2.1

How reproducible:
Always

Steps to Reproduce:
1. deploy metrics and view the Heapster logs

Actual results:
no warnings or errors in the logs unless a real error or issue has occured.

Comment 1 Troy Dawson 2016-08-11 19:52:42 UTC

This is fixed in rpm heapster-1.1.0-3.el7
That rpm is in image openshift3/metrics-heapster:3.3.0-3 and newer.

Comment 3 chunchen 2016-08-12 06:10:26 UTC

It's fixed, checked with the latest metrics images, please refer to the below testing results:

<------------snip---------->
he endpoint check has successfully completed.
Starting Heapster with the following arguments: --source=kubernetes:https://host-8-175-74.host.centralci.eng.rdu2.redhat.com:8443?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250 --sink=hawkular:https://hawkular-metrics:443?tenant=_system&labelToTenant=pod_namespace&labelNodeId=nodename&caCert=/hawkular-cert/hawkular-metrics-ca.certificate&user=hawkular&pass=4kzA67N0MAW8wX_&filter=label(container_name:^system.slice.*|^user.slice) --tls_cert=/secrets/heapster.cert --tls_key=/secrets/heapster.key --tls_client_ca=/secrets/heapster.client-ca --allowed_users=system:master-proxy --metric_resolution=10s
I0812 02:07:12.216218       1 heapster.go:65] heapster --source=kubernetes:https://host-8-175-74.host.centralci.eng.rdu2.redhat.com:8443?useServiceAccount=true&kubeletHttps=true&kubeletPort=10250 --sink=hawkular:https://hawkular-metrics:443?tenant=_system&labelToTenant=pod_namespace&labelNodeId=nodename&caCert=/hawkular-cert/hawkular-metrics-ca.certificate&user=hawkular&pass=4kzA67N0MAW8wX_&filter=label(container_name:^system.slice.*|^user.slice) --tls_cert=/secrets/heapster.cert --tls_key=/secrets/heapster.key --tls_client_ca=/secrets/heapster.client-ca --allowed_users=system:master-proxy --metric_resolution=10s
I0812 02:07:12.216312       1 heapster.go:66] Heapster version 1.1.0
I0812 02:07:12.216490       1 configs.go:60] Using Kubernetes client with master "https://host-8-175-74.host.centralci.eng.rdu2.redhat.com:8443" and version "v1"
I0812 02:07:12.216497       1 configs.go:61] Using kubelet port 10250
I0812 02:07:12.247606       1 driver.go:322] Initialised Hawkular Sink with parameters {_system https://hawkular-metrics:443?tenant=_system&labelToTenant=pod_namespace&labelNodeId=nodename&caCert=/hawkular-cert/hawkular-metrics-ca.certificate&user=hawkular&pass=4kzA67N0MAW8wX_&filter=label(container_name:^system.slice.*|^user.slice) 0xc8202dc000  5}
I0812 02:07:12.348419       1 heapster.go:92] Starting with Hawkular-Metrics Sink
I0812 02:07:12.348501       1 heapster.go:92] Starting with Metric Sink
I0812 02:07:12.362776       1 heapster.go:171] Starting heapster on port 8082

Comment 5 errata-xmlrpc 2016-09-27 09:43:49 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1933