Created attachment 1388714 [details] metrics logs +++ This bug was initially created as a clone of Bug #1469423 +++ Description of problem: Set openshift_metrics_cassandra_replicas=2 to deploy,it take a long time, about 15 minutes, to make hawkular-metrics and heapster pod get ready. Same phenomenon if set openshift_metrics_hawkular_replicas as non-default value Version-Release number of selected component (if applicable): # openshift version openshift v3.9.0-0.31.0 kubernetes v1.9.1+a0ce1bc657 etcd 3.2.8 Metrics version metrics-heapster-v3.9.0-0.34.0.0 metrics-hawkular-metrics-v3.9.0-0.34.0.0 metrics-cassandra-v3.9.0-0.34.0.0 How reproducible: Always Steps to Reproduce: 1. Set openshift_metrics_cassandra_replicas=2 in inventory file, and deploy metrics via ansible 2. 3. Actual results: It take a long time, about 15 minutes, to make hawkular-metrics and heapster pod get ready Expected results: hawkular-metrics and heapster pod should not wait for so long to become ready.
I am moving to ON_QA since this ticket depended on the work done in bug 1560695.
It takes around 9 minutes to reach Running status for all pods, much more quickly than before, but one hawkular-cassandra pod had restarted for 3 times, 2 times for hawkular-metrics-schema pod, since the restart mechanism is expected, close this defect as VERIFIED. NAME READY STATUS RESTARTS AGE IP NODE hawkular-cassandra-1-gb77z 1/1 Running 0 8m 10.129.0.12 *****-qeos-nrr-1 hawkular-cassandra-2-vxq6n 1/1 Running 3 8m 10.128.0.10 *****-qeos-master-etcd-1 hawkular-metrics-2h2mw 0/1 Running 0 7m 10.129.0.11 *****-qeos-nrr-1 hawkular-metrics-schema-qznfv 1/1 Running 2 9m 10.129.0.9 *****-qeos-nrr-1 heapster-dkpjx 0/1 Running 0 7m 10.128.0.9 *****-qeos-master-etcd-1 parameters openshift_metrics_install_metrics=true openshift_metrics_cassandra_storage_type=dynamic openshift_metrics_cassandra_replicas=2 metrics version: v3.10.27-1 # openshift version openshift v3.10.27