Red Hat Bugzilla – Bug 1469295
After upgrading to 3.5 hawkular-metrics pod cannot start with error against permissions
Last modified: 2017-08-03 12:09:53 EDT
Description of problem:
Customer upgraded OpenShift cluster from 3.3 to 3.4 and then to 3.5 and tried to deploy metrics.
It deployed but eh hawkular-metrics pod fails with:
Starting Hawkular Metrics Error: the service account for Hawkular Metrics does not have permission to view resources in this namespace. View permissions are required for Hawkular Metrics to function properly. Usually this can be resolved by running: oc adm policy add-role-to-user view system:serviceaccount:openshift-infra:hawkular -n openshift-infra
Version-Release number of selected component (if applicable):
metrics-hawkular-metrics v3.5 4ede8a0257c8 = 3.5.0-22
metrics-heapster v3.5 56f0e1727405 = 3.5.0-16
metrics-cassandra v3.5 46585da34fbe = 3.5.0-19
Found BZ 1448462 and tested the commands there (cacert_output) but it had vastly different output than what that bug help so I opened this new one.
I will be attaching logs and that acert_output file shortly.
From the attached cacert_output the error is that the OpenShift Master endpoint is not accepting a connection and is closing it (Connection reset by peer)
This can mean a few things.
The master API is not available. It could be behind a firewall or not exposed in a way that the Hawkular Metrics pod can access it.
The master API is not available at the expected hostname. By default this is https://kubernetes.default.svc:443 but the system may be setup to use a different internal hostname for it (you can configure a different hostname by specifying the openshift_metrics_master_url property in your inventory file).
I am going to mark this as 'noabug' as it looks like they incorrectly set their metrics URL in their inventory file.