Bug 1464871 - hawkular-openshift-agent-configmap.yaml should be changed since there is no hawkular-metrics-certificate secret in Metrics 3.6.0
hawkular-openshift-agent-configmap.yaml should be changed since there is no h...
Status: VERIFIED
Product: OpenShift Container Platform
Classification: Red Hat
Component: Metrics (Show other bugs)
3.6.0
Unspecified Unspecified
high Severity high
: ---
: 3.7.0
Assigned To: Matt Wringe
Junqi Zhao
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-06-26 03:15 EDT by Junqi Zhao
Modified: 2017-10-05 13:47 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
secrets under openshift-infra namespace (8.90 KB, text/plain)
2017-06-26 03:22 EDT, Junqi Zhao
no flags Details

  None (edit)
Description Junqi Zhao 2017-06-26 03:15:41 EDT
Description of problem:
In https://github.com/openshift/origin-metrics/blob/enterprise/hawkular-openshift-agent/hawkular-openshift-agent-configmap.yaml
ca_cert_file: secret:openshift-infra/hawkular-metrics-certificate/hawkular-metrics-ca.certificate

But there is no hawkular-metrics-certificate secret in Metrics 3.6.0, could not get application level metrics.
# oc get secret hawkular-metrics-certificate -n openshift-infra
Error from server (NotFound): secrets "hawkular-metrics-certificate" not found

hawkular-openshift-agent is in CrashLoopBackOff status.
# oc get po -n default
NAME                             READY     STATUS             RESTARTS   AGE
docker-registry-3-dw9xz          1/1       Running            0          6h
hawkular-openshift-agent-rl3f5   0/1       CrashLoopBackOff   2          15m
hawkular-openshift-agent-tlh69   0/1       CrashLoopBackOff   2          15m
registry-console-1-pn85f         1/1       Running            0          6h
router-1-s0wh1                   1/1       Running            0          6h

# oc logs hawkular-openshift-agent-rl3f5 -n default
I0626 07:08:43.834605       1 hawkular-openshift-agent.go:69] Hawkular OpenShift Agent: Version: 1.2.2.Final, Commit: de35fdddc8a9f90b8094f372c6cf9576908dd780
I0626 07:08:43.834930       1 emitter_server.go:48] Agent emitter will emit metrics
I0626 07:08:43.834936       1 emitter_server.go:64] Agent emitter will NOT emit status
I0626 07:08:43.834940       1 emitter_server.go:70] Agent emitter will provide a health probe
I0626 07:08:43.834943       1 emitter_server.go:95] Agent will start the emitter endpoint at [:8080]
E0626 07:08:43.865156       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:08:43.865175       1 metrics_storage.go:216] Could not get the requested secret. This may mean the secret has not yet been created yet or the agent does not have the required permission to access the secret. Will attempt again every 5 seconds for the next 5 minutes
E0626 07:08:48.870860       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:08:53.875770       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:08:58.880059       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:09:03.884410       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:09:08.888818       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:09:13.893427       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:09:18.898125       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found
E0626 07:09:23.902304       1 metrics_storage.go:200] Error trying to get Secret named [hawkular-metrics-certificate] from namespace [openshift-infra]: err=secrets "hawkular-metrics-certificate" not found


Version-Release number of selected component (if applicable):
# openshift version
openshift v3.6.122
kubernetes v1.6.1+5115d708d7
etcd 3.2.0

Images from brew registry
metrics-hawkular-openshift-agent   v3.6                18d769922c5a        2 days ago          274 MB


How reproducible:
Always

Steps to Reproduce:
1. Deploy Metrics using ansible
2. Deploy the hawkular-openshift-agent
   1) copy files from:  https://github.com/openshift/origin-metrics/tree/enterprise/hawkular-openshift-agent
   2) Change the IMAGE parameter according to you environment in hawkular-openshift-agent.yaml and execute the following commands:
      oc process -f hawkular-openshift-agent.yaml | oc create -n default  -f -
      oc adm policy add-cluster-role-to-user hawkular-openshift-agent system:serviceaccount:default:hawkular-openshift-agent
3. Check hawkular-openshift-agent pods' log.

Actual results:
Could not get the requested secret named [hawkular-metrics-certificate] from namespace [openshift-infra]

Expected results:
Should be no error info in pod log

Additional info:
Comment 1 Junqi Zhao 2017-06-26 03:22 EDT
Created attachment 1291879 [details]
secrets under openshift-infra namespace
Comment 2 Matt Wringe 2017-09-29 14:13:16 EDT
For 3.6, it using ansible to install HOSA. See https://github.com/openshift/openshift-docs/pull/5419#issuecomment-333195877 for instructions.
Comment 3 Junqi Zhao 2017-09-29 20:52:01 EDT
We are using ansible to install HOSA, this issue does not happen now.

Set it to VERIFIED.

Note You need to log in before you can comment on or make changes to this bug.