Bug 1459221

Summary: NullPointerException in io.fabric8.elasticsearch.plugin.KibanaUserReindexAction.buildNewResult
Product: OpenShift Container Platform Reporter: Peter Portante <pportant>
Component: LoggingAssignee: Jeff Cantrill <jcantril>
Status: CLOSED ERRATA QA Contact: Xia Zhao <xiazhao>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.4.1CC: aos-bugs, rmeggins, smunilla
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-10 05:26:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Peter Portante 2017-06-06 14:45:45 UTC
Saw this stack trace multiple times on a 3.4.1 OCP instance, with logging-elasticsearch 3.4.1-26:

[2017-06-06 13:30:08,515][WARN ][rest.suppressed          ] path: /_mget, params: {ignore_unavailable=true, preference=1496755806317, timeout=0}
java.lang.NullPointerException
	at io.fabric8.elasticsearch.plugin.KibanaUserReindexAction.buildNewResult(KibanaUserReindexAction.java:180)
	at io.fabric8.elasticsearch.plugin.KibanaUserReindexAction.apply(KibanaUserReindexAction.java:124)
	at org.elasticsearch.action.support.TransportAction$ResponseFilterChain.proceed(TransportAction.java:210)
	at org.elasticsearch.action.support.TransportAction$FilteredActionListener.onResponse(TransportAction.java:237)
	at org.elasticsearch.action.support.TransportAction$FilteredActionListener.onResponse(TransportAction.java:223)
	at org.elasticsearch.action.get.TransportMultiGetAction$1.finishHim(TransportMultiGetAction.java:122)
	at org.elasticsearch.action.get.TransportMultiGetAction$1.onResponse(TransportMultiGetAction.java:104)
	at org.elasticsearch.action.get.TransportMultiGetAction$1.onResponse(TransportMultiGetAction.java:97)
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:89)
	at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:85)
	at org.elasticsearch.action.support.TransportAction$ResponseFilterChain.proceed(TransportAction.java:212)
	at com.floragunn.searchguard.filter.SearchGuardFilter.apply(SearchGuardFilter.java:173)
	at org.elasticsearch.action.support.TransportAction$ResponseFilterChain.proceed(TransportAction.java:210)
	at io.fabric8.elasticsearch.plugin.KibanaUserReindexAction.apply(KibanaUserReindexAction.java:163)
	at org.elasticsearch.action.support.TransportAction$ResponseFilterChain.proceed(TransportAction.java:210)
	at org.elasticsearch.action.support.TransportAction$FilteredActionListener.onResponse(TransportAction.java:237)
	at org.elasticsearch.action.support.TransportAction$FilteredActionListener.onResponse(TransportAction.java:223)
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction$2.handleResponse(TransportSingleShardAction.java:235)
	at org.elasticsearch.action.support.single.shard.TransportSingleShardAction$AsyncSingleAction$2.handleResponse(TransportSingleShardAction.java:221)
	at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:158)
	at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.handleResponse(SearchGuardMessageChannelHandler.java:56)
	at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:124)
	at com.floragunn.searchguard.ssl.transport.SearchGuardMessageChannelHandler.messageReceived(SearchGuardMessageChannelHandler.java:50)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
	at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
	at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
	at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
	at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
	at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
	at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
	at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
	at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
	at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

Comment 1 Jeff Cantrill 2017-06-08 00:50:38 UTC
Fixed in origin: https://github.com/fabric8io/openshift-elasticsearch-plugin/pull/83

Comment 2 Jeff Cantrill 2017-06-26 21:23:36 UTC
Commit pushed to master at https://github.com/openshift/origin-aggregated-logging

https://github.com/openshift/origin-aggregated-logging/commit/4187d2dff2077ffd8c8b3dab3d7d60ba3458b10b
bump openshift-elasticsearch-plugin to 2.4.4.9
bug 1456413. Fix Kibana security error message
bug 1459221. Fix null pointer in KibanaUserReindexAction
fix plugin improper evaluation of index pattern
fix plugin improper evaluation of how to retrieve a user from 2.4.4.6

Comment 4 Xia Zhao 2017-07-03 10:01:43 UTC
Tested with the latest v3.6 images on OCP 3.6.0, logging system worked fine and didn't meet this exception in es log. Set to verified.

Test env:
# openshift version
openshift v3.6.131
kubernetes v1.6.1+5115d708d7
etcd 3.2.1


ansible version:
openshift-ansible-playbooks-3.6.131-1.git.0.d87dfaa.el7.noarch

Images tested with:
openshift3/logging-elasticsearch    c601094a6111
openshift3/logging-kibana    c91b7ad68dc7
openshift3/logging-fluentd    82367a1102e0
openshift3/logging-curator    b609245a72f9
openshift3/logging-auth-proxy    39164e25543c

Comment 6 errata-xmlrpc 2017-08-10 05:26:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1716