Bugzilla will be upgraded to version 5.0 on a still to be determined date in the near future. The original upgrade date has been delayed.
First Last Prev Next    This bug is not in your last search results.
Bug 1586169 - [Infra] NPE seen at org.opendaylight.netvirt.fibmanager.FibRpcServiceImpl.createFibEntry
[Infra] NPE seen at org.opendaylight.netvirt.fibmanager.FibRpcServiceImpl.cre...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: opendaylight (Show other bugs)
13.0 (Queens)
Unspecified Unspecified
high Severity high
: z1
: 13.0 (Queens)
Assigned To: Josh Hershberg
Tomas Jamrisko
Infra
: Triaged, ZStream
Depends On:
Blocks:
  Show dependency tree / graph
 
Reported: 2018-06-05 12:01 EDT by Tim Rozet
Modified: 2018-10-18 03:23 EDT (History)
10 users (show)

See Also:
Fixed In Version: opendaylight-8.3.0-1.el7ost
Doc Type: Bug Fix
Doc Text:
Missing parameters from createFibEntry generate a Null Pointer Exception (NPE) during NAT setup. This bug may result in missing FIB entries from the routing table, causing NAT or routing to fail. This update adds the proper parameters to the RPC call. NPE is no longer seen in the OpenDaylight log, and NAT and routing function correctly.
Story Points: ---
Clone Of:
Environment:
N/A
Last Closed: 2018-07-19 09:53:05 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
trozet: needinfo-

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


External Trackers
Tracker ID Priority Status Summary Last Updated
OpenDaylight Bug NETVIRT-1133 None None None 2018-06-14 07:01 EDT
OpenDaylight gerrit 73293 None None None 2018-06-21 04:11 EDT
OpenDaylight gerrit 73294 None None None 2018-06-21 04:17 EDT
Red Hat Product Errata RHBA-2018:2215 None None None 2018-07-19 09:53 EDT

  None (edit)
Description Tim Rozet 2018-06-05 12:01:10 EDT 
Jun 05 13:28:22 controller-0 dockerd-current[19119]: java.lang.NullPointerException: null
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.netvirt.fibmanager.FibRpcServiceImpl.createFibEntry(FibRpcServiceImpl.java:91) [349:org.opendaylight.netvirt.fibmanager-impl:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.yangtools.yang.binding.util.RpcMethodInvokerWithInput.invokeOn(RpcMethodInvokerWithInput.java:32) [326:org.opendaylight.mdsal.yang-binding:0.12.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.yangtools.yang.binding.util.AbstractMappedRpcInvoker.invokeRpc(AbstractMappedRpcInvoker.java:53) [326:org.opendaylight.mdsal.yang-binding:0.12.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.binding.impl.BindingDOMRpcImplementationAdapter.invoke(BindingDOMRpcImplementationAdapter.java:83) [216:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.binding.impl.BindingDOMRpcImplementationAdapter.invokeRpc(BindingDOMRpcImplementationAdapter.java:70) [216:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.dom.broker.impl.GlobalDOMRpcRoutingTableEntry.invokeRpc(GlobalDOMRpcRoutingTableEntry.java:41) [219:org.opendaylight.controller.sal-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRoutingTable.invokeRpc(DOMRpcRoutingTable.java:178) [219:org.opendaylight.controller.sal-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRouter.invokeRpc(DOMRpcRouter.java:102) [219:org.opendaylight.controller.sal-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at Proxydd0e6907_5e3b_4eba_a05a_3f239c44a90a.invokeRpc(Unknown Source) [?:?]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at Proxyfae6851b_24a0_448f_a524_4b1638c5d6c5.invokeRpc(Unknown Source) [?:?]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke0(RpcServiceAdapter.java:65) [216:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.access$000(RpcServiceAdapter.java:43) [216:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter$RpcInvocationStrategy.invoke(RpcServiceAdapter.java:156) [216:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter.invoke(RpcServiceAdapter.java:96) [216:org.opendaylight.controller.sal-binding-broker-impl:1.7.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at com.sun.proxy.$Proxy316.createFibEntry(Unknown Source) [348:org.opendaylight.netvirt.fibmanager-api:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.netvirt.natservice.internal.NatTunnelInterfaceStateListener.hndlTepAddForDnatInEachRtr(NatTunnelInterfaceStateListener.java:898) [356:org.opendaylight.netvirt.natservice-impl:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.netvirt.natservice.internal.NatTunnelInterfaceStateListener.hndlTepAddForAllRtrs(NatTunnelInterfaceStateListener.java:412) [356:org.opendaylight.netvirt.natservice-impl:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.netvirt.natservice.internal.NatTunnelInterfaceStateListener.hndlTepEvntsForDpn(NatTunnelInterfaceStateListener.java:348) [356:org.opendaylight.netvirt.natservice-impl:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.netvirt.natservice.internal.NatTunnelInterfaceStateListener.add(NatTunnelInterfaceStateListener.java:186) [356:org.opendaylight.netvirt.natservice-impl:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.netvirt.natservice.internal.NatTunnelInterfaceStateListener.add(NatTunnelInterfaceStateListener.java:82) [356:org.opendaylight.netvirt.natservice-impl:0.6.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at org.opendaylight.genius.datastoreutils.AsyncDataTreeChangeListenerBase$DataTreeChangeHandler.run(AsyncDataTreeChangeListenerBase.java:167) [259:org.opendaylight.genius.mdsalutil-api:0.4.0.redhat-9]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:?]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:?]
Jun 05 13:28:22 controller-0 dockerd-current[19119]:         at java.lang.Thread.run(Thread.java:748) [?:?]



Jun 05 13:31:13 controller-0 dockerd-current[19119]: java.util.concurrent.TimeoutException: Delivery of system messages timed out and they were dropped.
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.remote.ReliableDeliverySupervisor$$anonfun$gated$1.applyOrElse(Endpoint.scala:346) [45:com.typesafe.akka.remote:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.actor.Actor.aroundReceive(Actor.scala:514) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.actor.Actor.aroundReceive$(Actor.scala:512) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.remote.ReliableDeliverySupervisor.aroundReceive(Endpoint.scala:203) [45:com.typesafe.akka.remote:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.actor.ActorCell.invoke(ActorCell.scala:496) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.Mailbox.run(Mailbox.scala:224) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.Mailbox.exec(Mailbox.scala:234) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [40:com.typesafe.akka.actor:2.5.4]
Jun 05 13:31:13 controller-0 dockerd-current[19119]:         at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [40:com.typesafe.akka.actor:2.5.4]
Comment 4 Michael Vorburger 2018-06-14 07:01:52 EDT 
Upstream NETVIRT-1133 seems to be the same, but was closed with Cannot Reproduce. Let's open that one if we have a reliable way to reproduce it here?
Comment 5 Tim Rozet 2018-06-14 10:27:08 EDT 
Yeah I saw it again last night after running OOO update.
Comment 6 Aswin Suryanarayanan 2018-06-14 12:36:04 EDT 
What OOO update do? Does it delete neutron resources ?
Comment 7 Tim Rozet 2018-06-15 14:09:14 EDT 
No, it restarts each ODL sequentially and updates it to a newer version.  I'm not 100% certain it is the same NPE, but I saw an NPE in Fib and figured it was the same one was I was glancing through the logs post update. I think there were 2 nova instances at the time which had FIP associated with them before update. We can try to reproduce.
Comment 8 Tim Rozet 2018-06-15 14:09:14 EDT 
No, it restarts each ODL sequentially and updates it to a newer version.  I'm not 100% certain it is the same NPE, but I saw an NPE in Fib and figured it was the same one was I was glancing through the logs post update. I think there were 2 nova instances at the time which had FIP associated with them before update. We can try to reproduce.
Comment 9 Josh Hershberg 2018-06-21 04:17:40 EDT 
Gerrit patches posted upstream. see attachments
Comment 21 Janki 2018-07-17 00:17:17 EDT 
I don't see this exception in karaf logs with opendaylight-8.3.0-1.el7ost.noarch.
Comment 23 errata-xmlrpc 2018-07-19 09:53:05 EDT 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:2215
Note You need to log in before you can comment on or make changes to this bug.
First Last Prev Next    This bug is not in your last search results.