From RedHat Customer Service, this bug seems to be fixed in openvswitch-2.1.2-2.el7_0.1. Where can we find it? Why it's not in RDO repository?

> Known bug
> https://github.com/openvswitch/ovs/commit/3601bd879
> 
> Fixed in upstream 2.1.3. Is it possible to have it backported to el7?

https://repos.fedorapeople.org/repos/openstack/openstack-juno/epel-7/openvswitch-2.1.2-2.el7.centos.1.x86_64.rpm was built[*] from openvswitch-2.1.2-2.el7_0.1 SRPM which has changelog entry:
- updated with git snapshot up to 3f0588b

which includes 3601bd879.

[*] https://copr.fedoraproject.org/coprs/jruzicka/rdo-juno-epel-7/build/52323/


> Fixed in upstream 2.1.3. Is it possible to have it backported to el7?

Has that been confirmed? There are only few unrelated patches 3f0588b..v2.1.3

We'll be able to rebase to 2.3.0 after centos 7.1 is available (kernel changes are required).

Nope it seems redhat customer support is wrong, we still have the error with their package:
2015-02-23T02:23:57.879Z|646607|dpif(revalidator_7)|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),tunnel(tun_id=0x68,src=192.168.40.104,dst=192.168.40.101,tos=0x0,ttl=64,flags(key)),in_port(8),skb_mark(0),eth(src=02:00:c0:a8:c7:d2,dst=02:00:c0:a8:c7:d0),eth_type(0x8100),vlan(vid=199,pcp=0),encap(eth_type(0x0800),ipv4(src=192.168.199.210,dst=192.168.199.208,proto=6,tos=0,ttl=64,frag=no),tcp(src=50631,dst=8300),tcp_flags(0x002))

Is there a release date for CentOS 7.1?

Oops sorry wrong log (should have read the date). What is strange is the redhat package has the same git commit for its sources, and same spec file. But with this redhat package we don't have the flow_del error...

> But with this redhat package we don't have the flow_del error...

Was that on the same machine (was it RHEL or CentOS?)
i.e. you had openvswitch-2.1.2-2.el7.centos.1 installed and got flow_del errors, then _only_ upgraded to openvswitch-2.1.2-2.el7_0.1 and errors were gone?

Same machine, CentOS 7 (but we can have RHEL we are on our way to pay a subscription on Openstack platform).

So we have flow_del errors on openvswitch-2.1.2-2.el7.centos.1 running on CentOS. Then _only_ upgrading to openvswitch-2.1.2-2.el7_0.1 fix all the errors. At least it's running since a couple of days, still nothing. We don't understand why. If you have any idea?

A bit more fun, not sure if quite relevant:

Here is a dump from in order redhat srpm rebuilt in mock against centos/epel7 repo, redhat openvswitch package, and rdo package (only ovs-vswitchd):
SIZE    CHECKSUMS
1569432 0b144c4f31595d7a98c804df2b22a0d330d1bb656ea56e8263cd65dbb920a2af
1569448 f6806bf3435ca7b60de4bac2ac2acc2f671813117c505d28a661b6f41ae7ee7f
1569448 3d8c6ef31a67700760f9b92f530223e2ee3ca77c2cc50c9ac5263525094a2786

so 3 different binaries from same sources and same spec file. is it normal?

We just had this error again so definitively redhat package is not fixing the problem:

2015-03-03T01:17:33.257Z|00001|dpif(revalidator_7)|WARN|system@ovs-system: failed to flow_del (No such file or directory) skb_priority(0),in_port(6),skb_mark(0),eth(src=00:1a:64:a2:24:fe,dst=ff:ff:ff:ff:ff:ff),eth_type(0x8100),vlan(vid=4001,pcp=0),encap(eth_type(0x0806),arp(sip=10.156.1.186,tip=10.156.28.48,op=1,sha=00:1a:64:a2:24:fe,tha=00:00:00:00:00:00))

Which make more sense since it's the same sources as rdo package.

Haikel tells me ovs 2.3.1 works on EL7.0 kernel just that there might be some races. (Haikel, do you have details?)
Please try CBS build https://cbs.centos.org/koji/buildinfo?buildID=568
if that works for you, we'll push it as RDO Juno update asap.

With this package we have more warnings as soon as we start the openvswitch service:

2015-03-12T09:58:41.985Z|00005|ofproto_dpif|INFO|system@ovs-system: Datapath does not support recirculation
2015-03-12T09:58:41.985Z|00006|dpif|WARN|system@ovs-system: execute userspace(pid=0,userdata(00000000)) failed (Invalid argument) on packet metadata=0,in_port=0,vlan_tci=0x0000,dl_src=00:00:00:00:00:00,dl_dst=00:00:00:00:00:00,dl_type=0x1234
2015-03-12T09:58:41.985Z|00007|ofproto_dpif|WARN|system@ovs-system: variable-length userdata feature probe failed (Invalid argument)
2015-03-12T09:58:41.985Z|00008|dpif|WARN|system@ovs-system: failed to put[create] (Invalid argument) skb_priority(0),skb_mark(0),in_port(0),eth(src=00:00:00:00:00:00,dst=00:00:00:00:00:00),eth_type(0x8847),mpls(label=0,tc=0,ttl=0,bos=1)

We also have this, could be related to selinux:
2015-03-12T09:56:25.212Z|00001|ofproto_dpif_xlate(handler10)|WARN|/proc/sys/net/core/netdev_max_backlog: open failed (Permission denied)

Flavio Leitner says it should work and I haven't run into any issues with it so far.

"You should be able to use openvswitch with any RHEL-7 kernel, but
versions prior to the above missed one bugfix, so I'd recommend
to use at least that version."

http://openvswitch.org/pipermail/dev/2014-November/048319.html 
http://openvswitch.org/pipermail/discuss/2014-November/015511.html

Warnings from comment 10 are also present with 7.1 kernel 3.10.0-229.el7.x86_64

ovs 2.1.2 build we have in RDO repos crashes with 7.1 kernel (bug 1200918) so we need to push ovs 2.3.1 update asap.

RDO Juno update https://review.gerrithub.io/228852