Description of problem: After deploying monitoring, Prometheus is unable to access etcd and the logs show x509 error. Sounds similar to https://bugzilla.redhat.com/show_bug.cgi?id=1645417 Version-Release number of selected component (if applicable): 3.11 How reproducible: 1. Deploy Prometheus monitoring 2. Prometheus is unable to monitor etcd due to not having a secret 3. Create the kube-etcd-client-certs secret in the openshift-monitoring project 4. Prometheus does not detect and use this new secret and so still cannot monitor etcd 5. A temporary workaround is to manually add the secret in the GUI which allows the daemon set to be created -- but this does not persist Actual results: The Prometheus operator does not automatically create a secret to monitor etcd. A manually added secret does not get noticed by Prometheus operator. A manually added secret using the 'add this secret to an application' dialogue in the GUI works around this issue but does not persist. This error is seen in Prometheus log: level=error ts=2019-08-15T11:12:15.605123238Z caller=scrape.go:148 component="scrape manager" scrape_pool=openshift-monitoring/etcd/0 msg="Error creating HTTP client" err="unable to use specified CA cert /etc/prometheus/secrets/kube-etcd-client-certs/etcd-client-ca.crt: open /etc/prometheus/secrets/kube-etcd-client-certs/etcd-client-ca.crt: no such file or directory Expected results: 1. Prometheus should create the correct secret for monitoring etcd or 2. Prometheus should detect that a secret for monitoring etcd has been created. 3. Also, a secret added in the GUI should persist.
oc v3.11.98 kubernetes v1.11.0+d4cacc0 features: Basic-Auth GSSAPI Kerberos SPNEGO Server https://master.redacted.com:8443 openshift v3.11.98 kubernetes v1.11.0+d4cacc0
*** Bug 1772948 has been marked as a duplicate of this bug. ***