Created attachment 1219706 [details]
node log for the panic error

Description of problem:
When trying to merge network via oadm pod-network command, sometimes there is panic error appears in the node log. And the node service gets restarted after that.

Version-Release number of selected component (if applicable):
openshift v3.4.0.24+52fd77b
kubernetes v1.4.0+776c994
etcd 3.1.0-rc.0
openvswitch 2.4.0

How reproducible:
sometimes

Steps to Reproduce:
1. Setup multinode env with multitenant plugin

2. Create a lot of projects and pods and then delete them
$ for i in `seq 1 100` ; do oc new-project bmengtestpro$i ; oc create -f //raw.githubusercontent.com/openshift-qe/v3-testfiles/master/networking/pod-for-ping.json  -n bmengtestpro$i ; oc delete project bmengtestpro$i ; done

3. Create another two projects with some pods in them

4. Try to merge the two projects via oadm pod-network
# oadm pod-network join-projects project1 --to project2

5. Check the node log 

Actual results:
5. Panic error and traceback shows in node log. And the project cannot be merged.

Expected results:
Should not panic and merge network works.

Additional info:
I have never experienced this error when testing with openvswitch 2.5.0, but I am not pretty sure that this is an ovs 2.4.0 only issue. I can just say that it will have a high rate to reproduce this with ovs 2.4.0.

Full trace log attached.