Bug 1464971

Summary: Kibana not opening - SearchGuard no data for
Product: OpenShift Container Platform Reporter: Ruben Romero Montes <rromerom>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED DUPLICATE QA Contact: Xia Zhao <xiazhao>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.4.1CC: aos-bugs, bvincell, pportant
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-28 13:49:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logging-es and kibana logs none

Description Ruben Romero Montes 2017-06-26 10:25:02 UTC
Created attachment 1291942 [details]
logging-es and kibana logs

Description of problem:
Unable to open the Discover page in Kibana.

Version-Release number of selected component (if applicable):
3.4.1

How reproducible:
Unknown

Steps to Reproduce:
1. Install EFK stack
2. Try to access Kibana either directly or from "View Archive"

Actual results:
Unable to access Kibana. Page timeouts while loading

Expected results:
Display the Discovery page in Kibana

Additional info:

See the following logs for pod logging-es-80njvkm1-3-jvxvz
[2017-06-23 16:08:40,782][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for config while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,782][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for roles while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,782][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for rolesmapping while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,782][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for actiongroups while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,921][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for config while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,921][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for roles while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,921][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for rolesmapping while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)
[2017-06-23 16:08:40,921][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for actiongroups while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-80njvkm1-3-jvxvz)

It also happens in pod logging-es-54saglv2-3-hhqep
[2017-06-23 16:08:40,788][ERROR][com.floragunn.searchguard.configuration.ConfigurationLoader] No data for actiongroups while retrieving configuration for [config, roles, rolesmapping, actiongroups]  (index=.searchguard.logging-es-54saglv2-3-hhqep)

But not for the other pod logging-es-q07h8j30-4-4tsl1
[2017-06-23 16:08:39,104][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-q07h8j30-4-4tsl1] create_mapping [config]
[2017-06-23 16:08:40,264][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-q07h8j30-4-4tsl1] create_mapping [roles]
[2017-06-23 16:08:40,470][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-54saglv2-3-hhqep] create_mapping [config]
[2017-06-23 16:08:40,471][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-q07h8j30-4-4tsl1] create_mapping [rolesmapping]
[2017-06-23 16:08:40,648][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-q07h8j30-4-4tsl1] create_mapping [actiongroups]
[2017-06-23 16:08:40,650][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-54saglv2-3-hhqep] create_mapping [roles]
[2017-06-23 16:08:40,747][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-54saglv2-3-hhqep] create_mapping [rolesmapping]
[2017-06-23 16:08:40,829][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-54saglv2-3-hhqep] create_mapping [actiongroups]
[2017-06-23 16:08:43,166][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-80njvkm1-3-jvxvz] create_mapping [config]
[2017-06-23 16:08:43,414][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-80njvkm1-3-jvxvz] create_mapping [roles]
[2017-06-23 16:08:43,711][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-80njvkm1-3-jvxvz] create_mapping [rolesmapping]
[2017-06-23 16:08:43,819][INFO ][cluster.metadata         ] [Angela Del Toro] [.searchguard.logging-es-80njvkm1-3-jvxvz] create_mapping [actiongroups]

Comment 1 Ruben Romero Montes 2017-06-26 16:17:15 UTC
After scaling down and deleting manually all the files, the behaviour persists.

Comment 3 Jeff Cantrill 2017-06-28 13:49:12 UTC
Closing as a dup since this is because the SG perms are not seeded properly.  There is a fix to be backported to 3.4 to resolve.

*** This bug has been marked as a duplicate of bug 1449378 ***

Comment 4 Peter Portante 2017-06-28 15:42:41 UTC
This second set of logs seems to show that logging is working fine, but the second set of logs was captured too quickly after ES pods were started.  If we could get an update on this problem, if it is still a problem, that would be great.