Description of problem: After a fresh install of the Aggregated Logging logs coming from the operations projects (i.e. default, openshift-infra and openshift) are not sent to the .operations.* index Version-Release number of selected component (if applicable): $ oc version oc v3.6.173.0.5 $ oc get po -o 'go-template={{range $pod := .items}}{{range $container := $pod.spec.containers}}oc exec -c {{$container.name}} {{$pod.metadata.name}} -- find /root/buildinfo -name Dockerfile-openshift*{{"\n"}}{{end}}{{end}}' | bash - /root/buildinfo/Dockerfile-openshift3-logging-curator-v3.6.173.0.21-17 /root/buildinfo/Dockerfile-openshift3-logging-elasticsearch-v3.6.173.0.5-5 /root/buildinfo/Dockerfile-openshift3-logging-fluentd-v3.6.173.0.21-17 /root/buildinfo/Dockerfile-openshift3-logging-fluentd-v3.6.173.0.21-17 /root/buildinfo/Dockerfile-openshift3-logging-kibana-v3.6.173.0.21-17 /root/buildinfo/Dockerfile-openshift3-logging-auth-proxy-v3.6.173.0.21-17 How reproducible: Always Steps to Reproduce: 1. Install a 3.6 cluster 2. Deploy EFK using ansible 3. Access Kibana and browse the .operations.* index pattern Actual results: Only logs from journald are shown Expected results: Logs from journald and infra projects to be shown Additional info: $ oc exec logging-es-data-master-moc45ja4-2-8q889 -- curl -s --key /etc/elasticsearch/secret/admin-key --cert /etc/elasticsearch/secret/admin-cert --cacert /etc/elasticsearch/secret/admin-ca https://localhost:9200/_cat/indices?h=index project.jenkins.a985ce8a-90c7-11e7-af7f-5254004a7695.2017.09.19 .searchguard.logging-es-data-master-moc45ja4 project.logging.4d1af73a-90c3-11e7-a156-5254004a7695.2017.09.19 .kibana.d033e22ae348aeb5660fc2140aec35850c4da997 project.default.34aa1a30-90bc-11e7-a156-5254004a7695.2017.09.18 project.default.34aa1a30-90bc-11e7-a156-5254004a7695.2017.09.15 project.default.34aa1a30-90bc-11e7-a156-5254004a7695.2017.09.19 .operations.2017.09.15 .operations.2017.09.19 .kibana .operations.2017.09.18 project.logging.4d1af73a-90c3-11e7-a156-5254004a7695.2017.09.18
Here is my ansible inventory used: #Aggregated logging openshift_logging_install_logging=true openshift_logging_image_version=v3.6 openshift_logging_kibana_hostname=kibana.apps36.example.com openshift_logging_es_cluster_size=1 openshift_logging_es_memory_limit=1G
Commit pushed to master at https://github.com/openshift/origin-aggregated-logging https://github.com/openshift/origin-aggregated-logging/commit/e29f72f7e38e8e70b975f1284efe8514c1f32ac9 Bug 1493188 - operations namespace logs not sent to .operations index https://bugzilla.redhat.com/show_bug.cgi?id=1493188 The format of the CONTAINER_NAME field changed from k8s_container-name.container-hash_ ... to k8s_container-name_ .... which caused problems for the retag filter. Since we don't care about the value of container-hash in this context, just ignore it. Also added some test to ensure that operations namespace logs end up in es-ops in the .operations.* indices and nowhere else.
The project logs can be list in .operations index with v3.6.173.0.39
@rich Let's use the v3.5 bug https://bugzilla.redhat.com/show_bug.cgi?id=1494310 to trace the v3.5 testing. Could you move it to ON_QA?
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3049