Still failing in: oc v3.9.14 kubernetes v1.9.1+a0ce1bc657 fluentd v3.9.14-2 Now checking with latest fluentd image for 3.9
Fails also for logging-fluentd-v3.9.25-1 $ sh estest.sh 1cfc5479-5207-11e8-925a-bab7b04c3d1f ^C [quicklab@master-0 ~]$ sh estest1.sh 1cfc5479-5207-11e8-925a-bab7b04c3d1f 4e2bb589-5208-11e8-925a-bab7b04c3d1f 5aa44177-5208-11e8-925a-bab7b04c3d1f 6896bdf0-5208-11e8-925a-bab7b04c3d1f 7a13bd50-5208-11e8-925a-bab7b04c3d1f 874460ce-5208-11e8-925a-bab7b04c3d1f 93be2569-5208-11e8-925a-bab7b04c3d1f a1259a95-5208-11e8-925a-bab7b04c3d1f ad6ce706-5208-11e8-925a-bab7b04c3d1f bae6d13f-5208-11e8-925a-bab7b04c3d1f c832f30d-5208-11e8-925a-bab7b04c3d1f d5ba3457-5208-11e8-925a-bab7b04c3d1f e3228722-5208-11e8-925a-bab7b04c3d1f f00afe2e-5208-11e8-925a-bab7b04c3d1f fdb7f12d-5208-11e8-925a-bab7b04c3d1f 0afcbdda-5209-11e8-925a-bab7b04c3d1f 17343b85-5209-11e8-925a-bab7b04c3d1f 23b27ca8-5209-11e8-925a-bab7b04c3d1f 2fdd3f1d-5209-11e8-925a-bab7b04c3d1f 3d22c2d5-5209-11e8-925a-bab7b04c3d1f [quicklab@master-0 ~]$ oc rsh -c elasticsearch $es_pod es_util --query=_cat/indices?h=index,docs.count | grep pelle project.pelle-foo.1cfc5479-5207-11e8-925a-bab7b04c3d1f.2018.05.07 1 [quicklab@master-0 ~]$ oc get po -o 'go-template={{range $pod := .items}}{{range $container := $pod.spec.containers}}oc exec -c {{$container.name}} {{$pod.metadata.name}} -- find /root/buildinfo -name Dockerfile-openshift*{{"\n"}}{{end}}{{end}}' | bash - /root/buildinfo/Dockerfile-openshift3-logging-curator-v3.9.14-2 /root/buildinfo/Dockerfile-openshift3-logging-elasticsearch-v3.9.14-2 /root/buildinfo/Dockerfile-openshift3-oauth-proxy-v3.9.14-2 /root/buildinfo/Dockerfile-openshift3-logging-fluentd-v3.9.25-1 /root/buildinfo/Dockerfile-openshift3-logging-fluentd-v3.9.25-1 /root/buildinfo/Dockerfile-openshift3-logging-fluentd-v3.9.25-1 /root/buildinfo/Dockerfile-openshift3-logging-kibana-v3.9.14-2 /root/buildinfo/Dockerfile-openshift3-logging-auth-proxy-v3.9.14-3
Master PR to add test to CI
https://github.com/openshift/origin-aggregated-logging/pull/1145
In addition to https://github.com/openshift/origin-aggregated-logging/pull/1145 we need - a hack/testing/test-indexing_after_ns_removal.sh which calls test/indexing_after_ns_removal.sh - need to verify that the kubernetes metadata in the record matches the expected data e.g. the pod uuid in the record must match the actual pod uuid, the namespace uuid must match the actual namespace uuid and must match the namespace uuid in the index name - the test must check to see that there are no orphaned records in the .orphaned index
Master PR including above comments: https://github.com/openshift/origin-aggregated-logging/pull/1190
OCP 3.6-3.10 is no longer on full support [1]. Marking CLOSED DEFERRED. If you have a customer case with a support exception or have reproduced on 3.11+, please reopen and include those details. When reopening, please set the Target Release to the appropriate version where needed. [1]: https://access.redhat.com/support/policy/updates/openshift
Re-opening this BZ, since this is still happening in v3.11.153 EFK images.
(In reply to Nicolas Nosenzo from comment #15) > Re-opening this BZ, since this is still happening in v3.11.153 EFK images. Can you provide examples of what you are seeing? * In what index are the logs found? * What is the ns/nsuuid of the expected ns
(In reply to Jeff Cantrill from comment #16) > Can you provide examples of what you are seeing? > > * In what index are the logs found? > * What is the ns/nsuuid of the expected ns Just found my lab wiped out, but I will come back with those examples early next week. Just fyi, I was using the test suite `indexing_after_ns_removal.sh`