Created attachment 1388714[details]
metrics logs
+++ This bug was initially created as a clone of Bug #1469423 +++
Description of problem:
Set openshift_metrics_cassandra_replicas=2 to deploy,it take a long time, about 15 minutes, to make hawkular-metrics and heapster pod get ready.
Same phenomenon if set openshift_metrics_hawkular_replicas as non-default value
Version-Release number of selected component (if applicable):
# openshift version
openshift v3.9.0-0.31.0
kubernetes v1.9.1+a0ce1bc657
etcd 3.2.8
Metrics version
metrics-heapster-v3.9.0-0.34.0.0
metrics-hawkular-metrics-v3.9.0-0.34.0.0
metrics-cassandra-v3.9.0-0.34.0.0
How reproducible:
Always
Steps to Reproduce:
1. Set openshift_metrics_cassandra_replicas=2 in inventory file, and deploy metrics via ansible
2.
3.
Actual results:
It take a long time, about 15 minutes, to make hawkular-metrics and heapster pod get ready
Expected results:
hawkular-metrics and heapster pod should not wait for so long to become ready.
It takes around 9 minutes to reach Running status for all pods, much more quickly than before, but one hawkular-cassandra pod had restarted for 3 times, 2 times for hawkular-metrics-schema pod, since the restart mechanism is expected, close this defect as VERIFIED.
NAME READY STATUS RESTARTS AGE IP NODE
hawkular-cassandra-1-gb77z 1/1 Running 0 8m 10.129.0.12 *****-qeos-nrr-1
hawkular-cassandra-2-vxq6n 1/1 Running 3 8m 10.128.0.10 *****-qeos-master-etcd-1
hawkular-metrics-2h2mw 0/1 Running 0 7m 10.129.0.11 *****-qeos-nrr-1
hawkular-metrics-schema-qznfv 1/1 Running 2 9m 10.129.0.9 *****-qeos-nrr-1
heapster-dkpjx 0/1 Running 0 7m 10.128.0.9 *****-qeos-master-etcd-1
parameters
openshift_metrics_install_metrics=true
openshift_metrics_cassandra_storage_type=dynamic
openshift_metrics_cassandra_replicas=2
metrics version: v3.10.27-1
# openshift version
openshift v3.10.27