Bug 1688969

Summary: Integrate coredns metrics with the monitoring component
Product: OpenShift Container Platform Reporter: Ravi Sankar <rpenta>
Component: NetworkingAssignee: Dan Mace <dmace>
Networking sub component: router QA Contact: Hongan Li <hongli>
Status: CLOSED ERRATA Docs Contact:
Severity: medium    
Priority: medium CC: aos-bugs, dmace
Version: 4.1.0   
Target Milestone: ---   
Target Release: 4.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Feature: Report coredns metrics to prometheus monitoring component. Reason: Gives better visibility into the dns queries. Result: Helps during troubleshooting and making dns optimizations.
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-04 10:45:52 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ravi Sankar 2019-03-14 19:49:58 UTC
Description of problem:
CoreDNS provides metrics plugin to report prometheus metrics.
Enable prometheus metrics in the corefile and integrate with openshift-monitoring component to get better visibility into the dns queries.

Comment 1 Ravi Sankar 2019-04-18 03:22:17 UTC
Fixed by https://github.com/openshift/cluster-dns-operator/pull/98

Comment 3 Hongan Li 2019-04-24 02:55:10 UTC
verified with 4.1.0-0.nightly-2019-04-23-223857 and issue has been fixed.

$ oc -n openshift-dns get svc
NAME          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)                  AGE
dns-default   ClusterIP   172.30.0.10   <none>        53/UDP,53/TCP,9153/TCP   50m

$ oc get servicemonitor/dns-default -n openshift-dns -o yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  creationTimestamp: 2019-04-24T02:07:52Z
  generation: 1
  name: dns-default
  namespace: openshift-dns
  ownerReferences:
  - apiVersion: apps/v1
    controller: true
    kind: DaemonSet
    name: dns-default
    uid: 67022044-6634-11e9-b683-0670efaa028e
  resourceVersion: "14873"
  selfLink: /apis/monitoring.coreos.com/v1/namespaces/openshift-dns/servicemonitors/dns-default
  uid: c415c9fc-6635-11e9-960e-0a02360899c2
spec:
  endpoints:
  - bearerTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
    interval: 30s
    path: /metrics
    port: metrics
    scheme: http
    tlsConfig:
      caFile: /etc/prometheus/configmaps/serving-certs-ca-bundle/service-ca.crt
      serverName: dns-default.openshift-dns.svc
  namespaceSelector:
    matchNames:
    - openshift-dns
  selector: {}

And can see DNS status on Prometheus Web UI:
http://10.128.0.6:9153/metrics
	up 	endpoint="metrics" instance="10.128.0.6:9153" job="dns-default" namespace="openshift-dns" pod="dns-default-wljh9" service="dns-default" 	10.353s ago 	6.618ms

Comment 6 errata-xmlrpc 2019-06-04 10:45:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0758