Bug 1821998

If user needs to perform query on this field and it requires fielddata (sorting or aggregation) then they should be using `kubernetes.container_name.raw` field instead. This raw field is not tokenized [1] and is treated as a keyword [2].

[1] https://github.com/openshift/origin-aggregated-logging/blob/master/elasticsearch/index_templates/com.redhat.viaq-openshift-operations.template.json#L256-L261
[2] https://www.elastic.co/guide/en/elasticsearch/reference/5.6/keyword.html

(In reply to Lukas Vlcek from comment #3)
> If user needs to perform query on this field and it requires fielddata
> (sorting or aggregation) then they should be using
> `kubernetes.container_name.raw` field instead. This raw field is not
> tokenized [1] and is treated as a keyword [2].
> 
> [1]
> https://github.com/openshift/origin-aggregated-logging/blob/master/
> elasticsearch/index_templates/com.redhat.viaq-openshift-operations.template.
> json#L256-L261
> [2] https://www.elastic.co/guide/en/elasticsearch/reference/5.6/keyword.html


Can you please tell the steps to use kubernetes.container_name.raw ?

Here are my primitive steps, and my suspicion is since this raw field is not tokenised, I shouldn't be using 'MYUSER' index:

```
[root@master-2 ~]# curl -k -H "Authorization: Bearer $(oc whoami -t)" -X DELETE https://$es_pod_ip:9200/.kibana.974602627aec0ac9c80ba400b71c38a29eca3d89
{"acknowledged":true}


[root@master-2 ~]# curl -k -H "Authorization: Bearer $(oc whoami -t)" -X PUT "https://$es_pod_ip:9200/.kibana.974602627aec0ac9c80ba400b71c38a29eca3d89" -H 'Content-Type: application/json'  -d'{
>   "mappings": {
>     "type": {
>       "properties": {
>         "kubernetes.container_name.raw": {
>           "type": "text",
>           "fielddata": true
>         }
>       }
>     }
>   }
> }'

{"acknowledged":true,"shards_acknowledged":true,"index":".kibana.974602627aec0ac9c80ba400b71c38a29eca3d89"}

[root@master-2 ~]#
[root@master-2 ~]# oc exec -it -c elasticsearch $es_pod -- es_load_kibana_ui_objects MYUSER
[2020-04-08 22:27:43,332][INFO ][container.run            ] Adding Kibana dashboards and other UI objects for user MYUSER index .kibana.974602627aec0ac9c80ba400b71c38a29eca3d89
[2020-04-08 22:27:43,334][INFO ][container.run            ] Adding the index pattern for project.* . . .
[2020-04-08 22:27:43,718][INFO ][container.run            ] Adding the Kibana UI objects . . .
[2020-04-08 22:27:44,087][INFO ][container.run            ] Success


## I see this in the output. But still get the error.


            "type": {
                "properties": {
                    "kubernetes": {
                        "properties": {
                            "container_name": {
                                "properties": {
                                    "raw": {
                                        "fielddata": true,
                                        "type": "text"
                                    }
                                }
                            }
                        }
                    }
                }
            },



```

Can you please share the error now? I want to make sure which field it is complaining about, does it still complain about `kubernetes.container_name` or `kubernetes.container_name.raw` now?

 Here are the steps we followed, but unfortunately they do not persist even after saving the file.

Attached as screenshots.

Hi,

I tested this and it was fixed by changing "kubernetes.container_name" to "kubernetes.container_name.raw" and now get no error and can see nice pie chart :-)

Kibana --> Management --> Saved Objects --> Visualizations

Click "Kubernetes Logs by Namespace, DC, Pod, Container"

Down bottom of page edit "visState" textbox adjusting "kubernetes.container_name" to "kubernetes.container_name.raw"


    {
      "id": "5",
      "type": "terms",
      "schema": "segment",
      "params": {
        "field": "kubernetes.container_name.raw",
        "size": 50,
        "order": "desc",
        "orderBy": "1"
      }
    }


Click "Save visualization Object"

Visualize --> Kubernetes Logs by Namespace, DC, Pod, Container

I have applied the PR located here: https://github.com/openshift/origin-aggregated-logging/pull/1878 to an Elastic pod running in OCP v3.11.200.

From a host with access to the cluster and assuming that you have the patch file (attached here as k8s-visualizations-unified.patch, you can run the following commands to validate the patch works as intended:

$ export ESPOD=$(oc get pods -n openshift-logging -o=jsonpath="{range .items[*]}{.metadata.name}{\"\n\"}{end}"| grep es)
$ mkdir elasticsearch.$ESPOD
$ oc rsync -n openshift-logging -c elasticsearch logging-es-data-master-qm79y90e-1-kcc6k:/usr/share/elasticsearch/kibana_ui_objects elasticsearch.$ESPOD/
kibana_ui_objects/k8s-dashboard.json
kibana_ui_objects/k8s-search.json
kibana_ui_objects/k8s-visualizations.json
$ cp -a elasticsearch.$ESPOD elasticsearch
$ patch -p0 < k8s-visualizations-unified.patch
patching file elasticsearch/kibana_ui_objects/k8s-dashboard.json
patching file elasticsearch/kibana_ui_objects/k8s-search.json
patching file elasticsearch/kibana_ui_objects/k8s-visualizations.json
$ oc rsync -n openshift-logging -c elasticsearch elasticsearch/kibana_ui_objects/ $ESPOD:/usr/share/elasticsearch/kibana_ui_objects/
$ oc exec -n openshift-logging $ESPOD -- es_load_kibana_ui_objects <USER NAME>

Assuming you have applications running under that user that are generating logs, you should see that the dashboards are present and working as intended.

Created attachment 1680401 [details]
A patch generated from the upstream GIT PR request.

Attaching the patch used in the previous comment.

k8s-visualizations-unified.patch

Verified, The user can navagated to kibana without any error.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days