Bug 1466962

Summary: Elasticsearch logging dead with exception: com.floragunn.searchguard.user.User cannot be cast to com.floragunn.searchguard.user.User
Product: OpenShift Container Platform Reporter: Peter Portante <pportant>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED DUPLICATE QA Contact: Xia Zhao <xiazhao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.5.1CC: aos-bugs, jgoulding, jwozniak, rmeggins, wgordon
Target Milestone: ---Keywords: OpsBlocker
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-07-05 11:51:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Peter Portante 2017-06-30 23:11:25 UTC
On a 3.5 install from the OpenShift Dedicated team, we see the following stack trace which is preventing operation of Elasticsearch:

[2017-06-30 17:38:57,598][WARN ][rest.suppressed          ] path: /_cluster/health, params: {pretty=true}
java.lang.ClassCastException: com.floragunn.searchguard.user.User cannot be cast to com.floragunn.searchguard.user.User
        at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:87)
        at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:170)
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:144)
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85)
        at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
        at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52)
        at org.elasticsearch.rest.BaseRestHandler$HeadersAndContextCopyClient.doExecute(BaseRestHandler.java:88)
        at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
        at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:853)
        at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.health(AbstractClient.java:873)
        at org.elasticsearch.rest.action.admin.cluster.health.RestClusterHealthAction.handleRequest(RestClusterHealthAction.java:62)
        at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:54)
        at org.elasticsearch.rest.RestController.executeHandler(RestController.java:198)
        at org.elasticsearch.rest.RestController$RestHandlerFilter.process(RestController.java:280)
        at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:261)
        at io.fabric8.elasticsearch.plugin.KibanaUserReindexFilter.process(KibanaUserReindexFilter.java:83)
        at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:264)
        at com.floragunn.searchguard.filter.SearchGuardRestFilter.process(SearchGuardRestFilter.java:65)
        at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:264)
        at com.floragunn.searchguard.filter.SearchGuardRestFilter.process(SearchGuardRestFilter.java:65)
        at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:264)
        at io.fabric8.elasticsearch.plugin.acl.DynamicACLFilter.process(DynamicACLFilter.java:158)
        at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:264)
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:161)
        at org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:153)
        at org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:101)
        at org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:451)
        at com.floragunn.searchguard.ssl.http.netty.SearchGuardSSLNettyHttpServerTransport.dispatchRequest(SearchGuardSSLNettyHttpServerTransport.java:162)
        at com.floragunn.searchguard.http.SearchGuardHttpServerTransport.dispatchRequest(SearchGuardHttpServerTransport.java:70)
        at org.elasticsearch.http.netty.HttpRequestHandler.messageReceived(HttpRequestHandler.java:61)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.elasticsearch.http.netty.pipelining.HttpPipeliningHandler.messageReceived(HttpPipeliningHandler.java:60)
        at org.jboss.netty.channel.SimpleChannelHandler.handleUpstream(SimpleChannelHandler.java:88)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.jboss.netty.handler.codec.http.HttpChunkAggregator.messageReceived(HttpChunkAggregator.java:145)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:108)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:459)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:536)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435)
        at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
        at org.elasticsearch.common.netty.OpenChannelsHandler.handleUpstream(OpenChannelsHandler.java:75)
        at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
        at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)

Comment 1 Peter Portante 2017-06-30 23:53:49 UTC
*** Bug 1466963 has been marked as a duplicate of this bug. ***

Comment 2 Peter Portante 2017-07-03 15:39:50 UTC
Also, the sgadmin.sh script is failing with:

Cannot retrieve cluster state due to: class_cast_exception: null. This is not an error, will keep on trying ...
   * Try running sgadmin.sh with -icl and -nhnv (If thats works you need to check your clustername as well as hostnames in your SSL certificates)
   * If this is not working, try running sgadmin.sh with --diagnose and see diagnose trace log file)

Comment 3 Jan Wozniak 2017-07-04 14:09:51 UTC
> On a 3.5 install from the OpenShift Dedicated team

Am I correct to assume this means they are using downstream images from brew? If yes, then I think we could inspect our images and synchronize upstream and downstream.

> java.lang.ClassCastException: com.floragunn.searchguard.user.User cannot be cast to com.floragunn.searchguard.user.User

When casting an instance of a class to apparently same class throws an exception, it could mean that this class was loaded by two different classloaders. And inspecting the ES downstream image, it looks like we are installing the searchguard plugin twice which could explain this exception. 

1) separately here: http://pkgs.devel.redhat.com/cgit/rpms/logging-elasticsearch-docker/tree/install.sh.prod?h=rhaos-3.5-rhel-7#n27

2) as a part of openshift-elasticsearch-plugin: http://pkgs.devel.redhat.com/cgit/rpms/logging-elasticsearch-docker/tree/install.sh.prod?h=rhaos-3.5-rhel-7#n36

The searchguard became part of openshift-elasticsearch-plugin a few months ago.
https://github.com/fabric8io/openshift-elasticsearch-plugin/issues/78

Comment 4 Jeff Cantrill 2017-07-05 11:51:09 UTC
Closing this one as a duplicate since https://bugzilla.redhat.com/show_bug.cgi?id=1467572#c5 identifies the latest images work properly.  My suspicion is that tested images were some mix of the originals that deployed SG as a plugin and the change to the openshift-elasticsearch-plugin that embed SG

*** This bug has been marked as a duplicate of bug 1467572 ***