Bug 1656086
| Summary: | Users unable to access Kibana after upgrade to 3.11 | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Matthew Barnes <mbarnes> |
| Component: | Logging | Assignee: | Jeff Cantrill <jcantril> |
| Status: | CLOSED ERRATA | QA Contact: | Anping Li <anli> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.11.0 | CC: | aos-bugs, jcantril, rmeggins, travi |
| Target Milestone: | --- | Keywords: | OpsBlocker |
| Target Release: | 3.11.z | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: |
Cause: Bug in earlier releases of cluster-logging introduced Kibana index-patterns where the title was not properly replaced and was left with the place holder of '$TITLE$'
Consequence: User sees permission error 'no permissions for [indices:data/read/field_caps]'
Fix: Remove all index-patterns which have the bad data either by upgrading or running 'oc exec -c elasticsearch -n $NS $pod -- es_util --query=".kibana.*/_delete_by_query?pretty" -d "{\"query\":{\"match\":{\"title\":\"*TITLE*\"}}}"'
Result: The permission error is no longer generated
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-01-10 09:04:12 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
I believe this is a result of bad index-pattern seeding left over from 3.9 that is only manifesting itself now. It appears there are issues with any index-pattern that has source with 'title' equal to '$TITLE'. This was the placeholder for our index-pattern template.
Work around is to:
a. Delete the kibana index pattern
or
b. Delete only the offenders: 'oc exec -c elasticsearch -n $NS $pod -- es_util --query=".kibana.*/_delete_by_query?pretty" -d "{\"query\":{\"match\":{\"title\":\"*TITLE*\"}}}"'
Commits pushed to master at https://github.com/openshift/origin-aggregated-logging https://github.com/openshift/origin-aggregated-logging/commit/b51ce0f1de4a004136d9de36fcc96c11521ddd13 bug 1656086. remove index-patterns with title= https://github.com/openshift/origin-aggregated-logging/commit/b362507e0888d799c8689b2875fb275dcc10b8ba Merge pull request #1484 from openshift-cherrypick-robot/cherry-pick-1482-to-master [master] bug 1656086. remove index-patterns with title=$TITLE Commits pushed to master at https://github.com/openshift/origin-aggregated-logging https://github.com/openshift/origin-aggregated-logging/commit/710f6e766f5ec4960e21b8131117b7527810bc8f bug 1656086. code review fixes https://github.com/openshift/origin-aggregated-logging/commit/452005329a41c68565a17f88a0f9f0938dcd42f9 Merge pull request #1487 from openshift-cherrypick-robot/cherry-pick-1485-to-master [master] bug 1656086. code review fixes Reproduced with ose-logging-elasticsearch5:v3.11.51-3.The issue are fixed after upgraded to openshift3/ose-logging-elasticsearch5:v3.11.59. Before v3.11.59, the Kibana index docs.count is 2 green open .kibana.39a89dd158c8e9747943b00f84be79fc33fbd405 U5T4rM_zSN6rfC2sB62N4A 1 0 2 0 55kb 55kb In v3.11.59, the kibana index docs.count is 3 green open .kibana.39a89dd158c8e9747943b00f84be79fc33fbd405 U5T4rM_zSN6rfC2sB62N4A 1 0 3 0 54.8kb 54.8kb Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0024 |
After upgrading to OpenShift 3.11, users are encountering permission errors when trying to access Kibana. Error: [object Object]: [security_exception] no permissions for [indices:data/read/field_caps] and User [name=USERNAME, roles=[gen_user_efad75b1cdea4c3c1e42f8bf5fbeefdde90c0fb0, gen_kibana_efad75b1cdea4c3c1e42f8bf5fbeefdde90c0fb0]] at https://logs.insights.openshift.com/bundles/kibana.bundle.js?v=15653:227:21353 at processQueue (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:38:23621) at https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:38:23888 at Scope.$eval (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:39:4619) at Scope.$digest (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:39:2359) at Scope.$apply (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:39:5037) at done (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:37:25027) at completeRequest (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:37:28702) at XMLHttpRequest.xhr.onload (https://logs.insights.openshift.com/bundles/commons.bundle.js?v=15653:37:29634) Short term workaround from bug https://bugzilla.redhat.com/show_bug.cgi?id=1652224#c4 applies here as well.