Bug 1970453

Summary: OVS restore flow script specifies OpenFlow14 leading error
Product: OpenShift Container Platform Reporter: Alexander Constantinescu <aconstan>
Component: NetworkingAssignee: Alexander Constantinescu <aconstan>
Networking sub component: openshift-sdn QA Contact: zhaozhanqi <zzhao>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: unspecified CC: zzhao
Version: 4.6   
Target Milestone: ---   
Target Release: 4.6.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1970452 Environment:
Last Closed: 2021-06-29 06:26:18 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1970452    
Bug Blocks:    

Description Alexander Constantinescu 2021-06-10 13:47:09 UTC 
+++ This bug was initially created as a clone of Bug #1970452 +++

Description of problem:

The ovs-save script used by the CNO to restore flows whenever OVS boots in container mode on 4.6 has a bug introduced by the bump to OVS 2.13, namely it defaults to using OpenFlow14 whereas openshift-sdn only supports OpenFlow13. This leads to un-restored flows on upgrades from 4.5 -> 4.6 causing downtime for customers on upgrades. 

Errors will look like:

2021-05-27 15:33:00 info: Loading previous flows ...
2021-05-27 15:33:00 info: Adding br0 if it doesn't exist ...
2021-05-27 15:33:00 info: Created br0, now adding flows ...
+ ovs-ofctl add-tlv-map br0 ''
2021-05-27T15:33:00Z|00001|vconn|WARN|unix:/var/run/openvswitch/br0.mgmt: version negotiation failed (we support version 0x01, peer supports version 0x04)
ovs-ofctl: br0: failed to connect to socket (Broken pipe)
+ ovs-ofctl -O OpenFlow14 add-groups br0 /var/run/openvswitch/ovs-save.WIL41g6jZP/br0.groups.dump --bundle
2021-05-27T15:33:00Z|00001|vconn|WARN|unix:/var/run/openvswitch/br0.mgmt: version negotiation failed (we support version 0x05, peer supports version 0x04)
ovs-ofctl: br0: failed to connect to socket (Broken pipe)
+ ovs-ofctl -O OpenFlow14 replace-flows br0 /var/run/openvswitch/ovs-save.WIL41g6jZP/br0.flows.dump --bundle
2021-05-27T15:33:00Z|00001|vconn|WARN|unix:/var/run/openvswitch/br0.mgmt: version negotiation failed (we support version 0x05, peer supports version 0x04)
ovs-ofctl: br0: failed to connect to socket (Broken pipe)


Version-Release number of selected component (if applicable):


How reproducible:

Upgrade from 4.5 -> 4.6 and inspect OVS logs, you will see:

+ ovs-ofctl -O OpenFlow14 replace-flows br0 /var/run/openvswitch/ovs-save.WIL41g6jZP/br0.flows.dump --bundle
2021-05-27T15:33:00Z|00001|vconn|WARN|unix:/var/run/openvswitch/br0.mgmt: version negotiation failed (we support version 0x05, peer supports version 0x04)

Steps to Reproduce:
1.
2.
3.

Actual results:

Errors similar to:

+ ovs-ofctl -O OpenFlow14 replace-flows br0 /var/run/openvswitch/ovs-save.WIL41g6jZP/br0.flows.dump --bundle
2021-05-27T15:33:00Z|00001|vconn|WARN|unix:/var/run/openvswitch/br0.mgmt: version negotiation failed (we support version 0x05, peer supports version 0x04)

Expected results:

No errors in OVS when restoring flows

Additional info:

--- Additional comment from Alexander Constantinescu on 2021-06-10 13:45:28 UTC ---

Closing this as CURRENT_RELEASE. This is not an issue on 4.7 since we launch OVS as a systemd service, it's needed however as to get the 4.6 back-port in.
Comment 1 Alexander Constantinescu 2021-06-11 13:03:32 UTC 
*** Bug 1970444 has been marked as a duplicate of this bug. ***
Comment 2 Alexander Constantinescu 2021-06-11 13:03:42 UTC 
*** Bug 1970447 has been marked as a duplicate of this bug. ***
Comment 5 zhaozhanqi 2021-06-22 02:23:25 UTC 
Verified this bug on 4.6.0-0.nightly-2021-06-19-165544



$ oc get ds ovs -n openshift-sdn -o yaml | grep -i flows.sh
                TMPDIR=/var/run/openvswitch /usr/share/openvswitch/scripts/ovs-save save-flows $bridges > /var/run/openvswitch/flows.sh
            if [[ -f /var/run/openvswitch/flows.sh ]]; then
              # There's a bug in the ovs-save script which causes flows.sh to specify
              sed -i 's/OpenFlow[0-9][0-9]/OpenFlow13/g' /var/run/openvswitch/flows.sh
              mv /var/run/openvswitch/flows.sh /var/run/openvswitch/flows-old.sh



sh-4.4# cat /var/run/openvswitch/flows.sh     
ovs-ofctl add-tlv-map br0 ''
ovs-ofctl -O OpenFlow14 add-groups br0               "/tmp/ovs-save.S8hpVLS3AN/br0.groups.dump"  --bundle
ovs-ofctl -O OpenFlow14 replace-flows br0               "/tmp/ovs-save.S8hpVLS3AN/br0.flows.dump"  --bundle
rm -rf "/tmp/ovs-save.S8hpVLS3AN"
sh-4.4# sed -i 's/OpenFlow[0-9][0-9]/OpenFlow13/g' /var/run/openvswitch/flows.sh
sh-4.4# cat /var/run/openvswitch/flows.sh 
ovs-ofctl add-tlv-map br0 ''
ovs-ofctl -O OpenFlow13 add-groups br0               "/tmp/ovs-save.S8hpVLS3AN/br0.groups.dump"  --bundle
ovs-ofctl -O OpenFlow13 replace-flows br0               "/tmp/ovs-save.S8hpVLS3AN/br0.flows.dump"  --bundle
rm -rf "/tmp/ovs-save.S8hpVLS3AN"
Comment 7 errata-xmlrpc 2021-06-29 06:26:18 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6.36 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2498