Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
The FDP team is no longer accepting new bugs in Bugzilla. Please report your issues under FDP project in Jira. Thanks.

Bug 1927211

Summary: [OVN]ipv4/ipv6 flows are marked ct flags without any acl attached to ports
Product: Red Hat Enterprise Linux Fast Datapath Reporter: Haresh Khandelwal <hakhande>
Component: ovn2.13Assignee: Dumitru Ceara <dceara>
Status: CLOSED ERRATA QA Contact: Jianlin Shi <jishi>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: RHEL 8.0CC: ctrautma, dceara, jishi, mleitner, mmichels, oblaut, ralongi, yrachman
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1927230 (view as bug list) Environment:
Last Closed: 2021-03-15 14:36:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1927230    

Description Haresh Khandelwal 2021-02-10 11:15:26 UTC
Description of problem:
On RHOSP 16.1.4 which as 
RHEL: 8.2 
Kernel: 4.18.0-193.41.1.el8_2.x86_64
OVS: openvswitch2.13-2.13.0-71.el8fdp.x86_64
OVN: ovn2.13-20.09.0-17.el8fdp.x86_64

While trying ovs hw offload with ml2/ovn, we see the issue that flows with ether type ipv4/ipv6 (0x0800/0x86dd) are marked with ct flags. Since rhel 8.2 doesnt support offloading of flows with ct flags (conntrack offload is not available in rhel8.2), packet steered via tc software data path.

This breaks feature functionality. 

Look at flow below. 
ufid:f3850114-4d46-4667-aaeb-8bc83f1ff378, skb_priority(0/0),skb_mark(0/0),ct_state(0/0x23),ct_zone(0/0),ct_mark(0/0),ct_label(0/0x1),recirc_id(0),dp_hash(0/0),in_port(ens1f1_1),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/01:00:00:00:00:00,dst=f8:f2:1e:03:bf:f4),eth_type(0x86dd),ipv6(src=fe80::/ffc0::,dst=fe80::1234,label=0/0,proto=59,tclass=0/0x3,hlimit=64,frag=no), packets:44, bytes:2024, used:0.080s, dp:tc, actions:set(tunnel(tun_id=0x2,dst=152.20.0.11,ttl=64,tp_dst=6081,key6(bad key length 1, expected 0)(01)geneve({class=0x102,type=0x80,len=4,0x20003}),flags(key))),genev_sys_6081


dcera has provided fix and with that no ct flags marked and flows is offlaoded (check offloaded:yes)

ufid:ccc06bf5-bd56-4461-953e-042d0a8c6ac2, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens1f1_1),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/01:00:00:00:00:00,dst=f8:f2:1e:03:bf:f4),eth_type(0x86dd),ipv6(src=fe80::/ffc0::,dst=fe80::1234,label=0/0,proto=59,tclass=0/0x3,hlimit=64,frag=no), packets:30, bytes:3660, used:0.030s, offloaded:yes, dp:tc, 
<hakhande> actions:set(tunnel(tun_id=0x2,dst=152.20.0.11,ttl=64,tp_dst=6081,key6(bad key length 1, expected 0)(01)geneve({class=0x102,type=0x80,len=4,0x20003}),flags(key))),genev_sys_6081



Version-Release number of selected component (if applicable):
Mentioned in description

How reproducible:
Always

Steps to Reproduce:
1. Deploy 16.1.4 with hw offload
2. Make sure not attaching any security groups, port securitty on the port
3. Send traffic from VM (Ping should be fine)

Actual results:
Flows are not offloaded as they are marked with ct flags.

Expected results:
No ct flgas in rhel 8.2. 

Additional info:
Other ether types like mpls,arp dont exhibit this issue. 

ufid:b5d5e99d-adec-46f3-98ea-0dcf04377f79, skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(ens1f1_1),packet_type(ns=0/0,id=0/0),eth(src=00:00:00:00:00:00/01:00:00:00:00:00,dst=f8:f2:1e:03:bf:f3),eth_type(0x8847),mpls((bad key length 0), packets:68, bytes:8296, used:0.090s, offloaded:yes, dp:tc, actions:set(tunnel(tun_id=0x2,dst=152.20.0.11,ttl=64,tp_dst=6081,key6(bad key length 1, expected 0)(01)geneve({class=0x102,type=0x80,len=4,0x28001}),flags(key))),genev_sys_6081

Comment 1 Dumitru Ceara 2021-02-10 11:22:02 UTC
Fix posted upstream for review: http://patchwork.ozlabs.org/project/ovn/list/?series=228905&state=*

Comment 5 Jianlin Shi 2021-02-18 02:37:52 UTC
tested with following script:

systemctl start openvswitch
systemctl start ovn-northd

ovn-nbctl set-connection ptcp:6641
ovn-sbctl set-connection ptcp:6642

ovs-vsctl set open . external_ids:system-id=hv1 external_ids:ovn-remote=tcp:1.1.175.25:6642 external_ids:ovn-encap-type=geneve external_ids:ovn-encap-ip=1.1.175.25
systemctl start ovn-controller

ovn-nbctl ls-add ls1
ovn-nbctl lsp-add ls1 ls1p1
ovs-vsctl add-port br-int ls1p1 -- set interface ls1p1 type=internal external_ids:iface-id=ls1p1

ovn-nbctl acl-add ls1 from-lport 1 'icmp' allow

ovn-sbctl lflow-list ls1 | grep -e ls_in_acl_hint -e ls_out_acl_hint -e ls_in_acl -e ls_out_acl | grep 'ct\.'

result on 20.12.0-15:

[root@wsfd-advnetlab21 bz1927211]# rpm -qa | grep -E "openvswitch2.13|ovn2.13"
openvswitch2.13-2.13.0-82.el7fdp.x86_64
ovn2.13-central-20.12.0-15.el7fdp.x86_64
ovn2.13-20.12.0-15.el7fdp.x86_64
ovn2.13-host-20.12.0-15.el7fdp.x86_64

+ grep -e ls_in_acl_hint -e ls_out_acl_hint -e ls_in_acl -e ls_out_acl
+ grep 'ct\.'
  table=6 (ls_in_acl_hint     ), priority=7    , match=(ct.new && !ct.est), action=(reg0[7] = 1; reg0[9] = 1; next;)
  table=6 (ls_in_acl_hint     ), priority=6    , match=(!ct.new && ct.est && !ct.rpl && ct_label.blocked == 1), action=(reg0[7] = 1; reg0[9] = 1; next;)
  table=6 (ls_in_acl_hint     ), priority=5    , match=(!ct.trk), action=(reg0[8] = 1; reg0[9] = 1; next;)
  table=6 (ls_in_acl_hint     ), priority=4    , match=(!ct.new && ct.est && !ct.rpl && ct_label.blocked == 0), action=(reg0[8] = 1; reg0[10] = 1; next;)
  table=6 (ls_in_acl_hint     ), priority=3    , match=(!ct.est), action=(reg0[9] = 1; next;)
  table=6 (ls_in_acl_hint     ), priority=2    , match=(ct.est && ct_label.blocked == 1), action=(reg0[9] = 1; next;)
  table=6 (ls_in_acl_hint     ), priority=1    , match=(ct.est && ct_label.blocked == 0), action=(reg0[10] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=7    , match=(ct.new && !ct.est), action=(reg0[7] = 1; reg0[9] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=6    , match=(!ct.new && ct.est && !ct.rpl && ct_label.blocked == 1), action=(reg0[7] = 1; reg0[9] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=5    , match=(!ct.trk), action=(reg0[8] = 1; reg0[9] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=4    , match=(!ct.new && ct.est && !ct.rpl && ct_label.blocked == 0), action=(reg0[8] = 1; reg0[10] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=3    , match=(!ct.est), action=(reg0[9] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=2    , match=(ct.est && ct_label.blocked == 1), action=(reg0[9] = 1; next;)
  table=4 (ls_out_acl_hint    ), priority=1    , match=(ct.est && ct_label.blocked == 0), action=(reg0[10] = 1; next;)

result on 20.12.0-17:

[root@wsfd-advnetlab21 bz1927211]# rpm -qa | grep -E "openvswitch2.13|ovn2.13"
openvswitch2.13-2.13.0-82.el7fdp.x86_64
ovn2.13-central-20.12.0-17.el7fdp.x86_64
ovn2.13-20.12.0-17.el7fdp.x86_64
ovn2.13-host-20.12.0-17.el7fdp.x86_64

+ ovn-sbctl lflow-list ls1
+ grep -e ls_in_acl_hint -e ls_out_acl_hint -e ls_in_acl -e ls_out_acl
+ grep 'ct\.'

<=== no flow for ct flags

Comment 6 Jianlin Shi 2021-02-19 03:27:24 UTC
also verified on rhel8 version:

+ ovs-vsctl add-port br-int ls1p1 -- set interface ls1p1 type=internal external_ids:iface-id=ls1p1
+ ovn-nbctl acl-add ls1 from-lport 1 icmp allow
+ ovn-sbctl lflow-list ls1
+ grep 'ct\.'
+ grep -e ls_in_acl_hint -e ls_out_acl_hint -e ls_in_acl -e ls_out_acl
[root@dell-per740-12 bz1927211]# rpm -qa | grep -E "openvswitch2.13|ovn2.13"
openvswitch2.13-2.13.0-95.el8fdp.x86_64
ovn2.13-host-20.12.0-20.el8fdp.x86_64
ovn2.13-20.12.0-20.el8fdp.x86_64
ovn2.13-central-20.12.0-20.el8fdp.x86_64

Comment 8 errata-xmlrpc 2021-03-15 14:36:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ovn2.13 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0836