Created attachment 1294520 [details] inventory file used for logging deployment Description of problem: .all index is missing when ops cluster is enabled. Issue did not exist if ops cluster is disabled. Version-Release number of selected component (if applicable): # openshift version openshift v3.6.133 kubernetes v1.6.1+5115d708d7 etcd 3.2.1 # rpm -qa | grep ansible openshift-ansible-callback-plugins-3.6.133-1.git.0.950bb48.el7.noarch openshift-ansible-docs-3.6.133-1.git.0.950bb48.el7.noarch openshift-ansible-lookup-plugins-3.6.133-1.git.0.950bb48.el7.noarch openshift-ansible-filter-plugins-3.6.133-1.git.0.950bb48.el7.noarch openshift-ansible-playbooks-3.6.133-1.git.0.950bb48.el7.noarch ansible-2.2.3.0-1.el7.noarch openshift-ansible-3.6.133-1.git.0.950bb48.el7.noarch openshift-ansible-roles-3.6.133-1.git.0.950bb48.el7.noarch # Images tested with ${brew_registry}/openshift3/logging-curator v3.6 3bc7ecc4f17d About an hour ago 222.1 MB ${brew_registry}/openshift3/logging-kibana v3.6 8da2ca80b6a4 2 hours ago 342.4 MB ${brew_registry}/openshift3/logging-elasticsearch v3.6 dc20042569d0 2 hours ago 404.2 MB ${brew_registry}/openshift3/logging-auth-proxy v3.6 bddefb8b639a 2 hours ago 214.8 MB ${brew_registry}/openshift3/logging-fluentd v3.6 eed0e8e709c0 2 hours ago 231.5 MB How reproducible: Always Steps to Reproduce: 1.Deploy logging v3.6 with ops cluster enabled 2.Wait until EFK are all on, and see "update_mapping" in both es and es-ops logs $ oc get po NAME READY STATUS RESTARTS AGE logging-curator-1-8r3gr 1/1 Running 0 1h logging-curator-ops-1-kr440 1/1 Running 0 1h logging-es-data-master-3137qzx4-1-j464p 1/1 Running 0 1h logging-es-ops-data-master-n3mjpg36-1-bs58q 1/1 Running 0 1h logging-fluentd-588rp 1/1 Running 0 1h logging-fluentd-vjcm8 1/1 Running 0 1h logging-kibana-1-zzqt1 2/2 Running 0 1h logging-kibana-ops-1-qh5xs 2/2 Running 0 1h 3.Login the non-ops logging UI --> see user indices there with log entries 4.Login the ops logging UI --> only .operations* exist Actual results: .all index is missing Expected results: .all index should exist Additional info: screenshots attached es and es-ops logs attached inventory file attached
Created attachment 1294522 [details] es log
Created attachment 1294523 [details] es ops log
Created attachment 1294524 [details] ops logging UI
Created attachment 1294525 [details] non-ops logging UI
Closing since finally I get the .all index on the non-ops kibana UI after seeing "update_mapping" in es log for another 1 hour.
Tested today, .all index is not found on Kibana UI # rpm -qa | grep openshift-ansible openshift-ansible-3.6.139-1.git.0.4ff49c6.el7.noarch openshift-ansible-roles-3.6.139-1.git.0.4ff49c6.el7.noarch openshift-ansible-docs-3.6.139-1.git.0.4ff49c6.el7.noarch openshift-ansible-lookup-plugins-3.6.139-1.git.0.4ff49c6.el7.noarch openshift-ansible-callback-plugins-3.6.139-1.git.0.4ff49c6.el7.noarch openshift-ansible-playbooks-3.6.139-1.git.0.4ff49c6.el7.noarch openshift-ansible-filter-plugins-3.6.139-1.git.0.4ff49c6.el7.noarch Images from brew registry logging-elasticsearch v3.6.138-1 2b2b061286e6 47 hours ago 404.2 MB logging-kibana v3.6.138-1 43aea64f70a8 47 hours ago 342.4 MB logging-curator v3.6.138-1 bf5d8756dcfc 47 hours ago 221.5 MB logging-fluentd v3.6.138-1 e4ed3ac61d69 47 hours ago 231.5 MB logging-auth-proxy v3.6.138-1 9b98fea74082 47 hours ago 214.8 MB
ops disabled cluster, kibana UI have .all index, see the attached UI. bz #1467834 is only happened with ops enabled cluster.
Created attachment 1295707 [details] ops disabled cluster, .all index on Kibana UI
It works for me again after waiting for long, similar with those in comment #5. I'm not able to see .all index on OCP 3.6 with these v3.6 images deployed: openshift3/logging-auth-proxy 4cf6b1d60d2b openshift3/logging-kibana 4563b27eac07 openshift3/logging-elasticsearch 8809f390a819 openshift3/logging-fluentd a2ea005ef4f6 openshift3/logging-curator ea1887b8e441
s/not/now/ in comment #9