Description of problem: Two prometheus instances running (with separate service accounts) for failover reasons. Changed the 'prometheus-metrics-viewer' rolebinding to contain both service accounts. Added both the user in the 'passwd.yml' file in the logging-elasticsearch secret. But this did not help for the second prometheus instance Actual results: Can only insert one user Expected results: Both users can we inserted. Additional info: Customer reports: Encountered the last piece which is the 'sg_role_mapping.yml' file. Looks like the PROMETHEUS_USER env variable from the elasticsearch DC gets inserted into the role mapping file when the elasticsearch container starts [1]. So far he solved it by using the bearer token from the first prometheus instance in the second one. [1] https://github.com/openshift/origin-aggregated-logging/blob/master/elasticsearch/run.sh#L82
I'm closing this as "Won't fix" because it impacts only the tech preview stack. By the way, running redundant Prometheus instances is usually achieved with a replica count > 1 rather than running 2 separate deployments. Prometheus in OpenShift 3.11 will be deployed by the cluster monitoring operator which will run multiple Prometheus instances out-of-the-box for redundancy.