Description of problem:
When fluentd is not able to find namespace or pod metadata it pushes that message to '.orphaned' index. Currently there is no way to see that index in kibana until es pod settings are modified manually.
Version-Release number of selected component (if applicable):
when someone need to look for messages in kibana under orphaned index.
Steps to Reproduce:
1. Create few pods which are logging at high rate (currently i was able to do it @ 1250 messages/mux/sec)
2. delete projects
3. see indexes, there will be a index project..orphaned.orphaned*
Messages are not visible in kibana.
Admins should be able to see logs in kibana
Pushing to 3.10 as the filter code still is not published in the code base.
Created attachment 1394934 [details]
kibana and orphan messages
This will be resolved by https://github.com/openshift/origin-aggregated-logging/pull/1048
The orphaned index can be view by cluster-admin in v3.9.27
.project.* and .orphaned index pattern are set by default
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.