Bug 1340172 - Can't get Utilization / Metrics from OSE 3.2
Summary: Can't get Utilization / Metrics from OSE 3.2
Keywords:
Status: CLOSED DUPLICATE of bug 1332866
Alias: None
Product: Red Hat CloudForms Management Engine
Classification: Red Hat
Component: C&U Capacity and Utilization
Version: 5.6.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: GA
: 5.7.0
Assignee: Federico Simoncelli
QA Contact: Dave Johnson
URL:
Whiteboard: container:c&u
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-05-26 15:35 UTC by Lutz Lange
Modified: 2016-05-30 22:13 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-05-30 22:13:29 UTC
Category: ---
Cloudforms Team: ---
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Lutz Lange 2016-05-26 15:35:34 UTC
Description of problem:
   I have the metrics stack running and working in my OSE 3.2. 
   The provider is configured and refreshing.
   Metrics are working in the OSE Web Interface.
   Router is set up and reachable on port 5000 of the Master Addresse

Version-Release number of selected component (if applicable):
   OSE 3.2
   CF 4.1 Beta

I could not identify any relevant failures in the logs.
You can get access to my environment. Please ping my by mail.

Comment 2 Lutz Lange 2016-05-30 06:21:34 UTC
It looks like there is some special user required to access the metrics. My demouser account can't get at the metrics. I looked into the heapster and found only a system:master-proxy user in an allowed users file. Is that the reason why this fails? :


# curl --insecure -H "Authorization: Bearer $(oc whoami -t)" -H "Hawkular-tenant: _system" -k -X GET https://tmaster.ose.sademo.de/hawkular/metrics/metrics

<html><head><title>JBWEB000065: HTTP Status 403 - </title><style><!--H1 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:22px;} H2 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:16px;} H3 {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;font-size:14px;} BODY {font-family:Tahoma,Arial,sans-serif;color:black;background-color:white;} B {font-family:Tahoma,Arial,sans-serif;color:white;background-color:#525D76;} P {font-family:Tahoma,Arial,sans-serif;background:white;color:black;font-size:12px;}A {color : black;}A.name {color : black;}HR {color : #525D76;}--></style> </head><body><h1>JBWEB000065: HTTP Status 403 - </h1><HR size="1" noshade="noshade"><p><b>JBWEB000309: type</b> JBWEB000067: Status report</p><p><b>JBWEB000068: message</b> <u></u></p><p><b>JBWEB000069: description</b> <u>JBWEB000123: Access to the specified resource has been forbidden.</u></p><HR size="1" noshade="noshade"></body></html>

Is CF hitting the same issue?

Comment 3 Federico Simoncelli 2016-05-30 06:55:43 UTC
Please run:

  # oc get pods -n openshift-infra -o yaml | grep image:

and make sure that the images version is 3.2.

Deploying OpenShift metrics 3.1 on a 3.2 cluster doesn't work (exactly for those authentication issues).

Thanks.

Comment 4 Lutz Lange 2016-05-30 07:53:48 UTC
# oc get pods -n openshift-infra -o yaml | grep image

      image: registry.access.redhat.com/openshift3/metrics-cassandra:3.2.0
      imagePullPolicy: IfNotPresent
    imagePullSecrets:
      image: registry.access.redhat.com/openshift3/metrics-cassandra:3.2.0
      imageID: docker://ee2117c9848298ca5a0cbbce354fd4adff370435225324ab9d60cd9cd9a95c53
      image: registry.access.redhat.com/openshift3/metrics-hawkular-metrics:3.2.0
      imagePullPolicy: IfNotPresent
    imagePullSecrets:
      image: registry.access.redhat.com/openshift3/metrics-hawkular-metrics:3.2.0
      imageID: docker://e1fc1a42f4990747f55ec147a345c7f5de3988649d4e5131cc63f739a48edea9
      image: registry.access.redhat.com/openshift3/metrics-heapster:3.2.0
      imagePullPolicy: IfNotPresent
      image: registry.access.redhat.com/openshift3/metrics-heapster:3.2.0
      imageID: docker://13d3ba87920a0e518a8f1853f15ccd5e0ca27f2d259064d4bbaa4814ab75118f

this is all 3.2 images. And the metrics do work in the OSE Web Interface.

Comment 5 Federico Simoncelli 2016-05-30 08:00:19 UTC
(In reply to Lutz Lange from comment #2)
> It looks like there is some special user required to access the metrics. My
> demouser account can't get at the metrics. I looked into the heapster and
> found only a system:master-proxy user in an allowed users file. Is that the
> reason why this fails? :
> 
> 
> # curl --insecure -H "Authorization: Bearer $(oc whoami -t)" -H
> "Hawkular-tenant: _system" -k -X GET
> https://tmaster.ose.sademo.de/hawkular/metrics/metrics

What is the token you configured in ManageIQ?

If you followed the guide here:

  https://access.redhat.com/documentation/en/red-hat-cloudforms/4.1-beta/managing-providers/chapter-4-containers-providers

The token to use is the one that you get from the command "oc get -n management-infra secrets management-admin-token ...".

In the command above you're using is your own token (that of course has not enough permissions).

Anyway debugging your commands may not be relevant to the issue you're experiencing in ManageIQ. Can you attach the evm logs? Thanks.

Comment 6 Federico Simoncelli 2016-05-30 22:13:29 UTC
After investigating this we found that even if the metrics roles were enabled in the CFME configuration, the backend didn't pick up the changes:

 $ rake evm:status
 automate:database_operations:database_owner:ems_inventory:ems_operations:event:reporting:scheduler:smartstate:user_interface:web_services:websocket

(The roles ems_metrics_collector:ems_metrics_coordinator:ems_metrics_processor were missing).

*** This bug has been marked as a duplicate of bug 1332866 ***


Note You need to log in before you can comment on or make changes to this bug.