Bug 1397045

Summary: [fdProd] yum update of openvswitch 2.5 from 2.4 causes restart of openvswitch service that can disrupt network connectivity
Product: Red Hat Enterprise Linux 7 Reporter: Flavio Leitner <fleitner>
Component: openvswitchAssignee: Flavio Leitner <fleitner>
Status: CLOSED ERRATA QA Contact: Rick Alongi <ralongi>
Severity: high Docs Contact:
Priority: high    
Version: 7.3CC: amuller, atragler, fleitner, kzhang, mflusche, nyechiel, pmyers, qding, ralongi, rkhan, wchadwic
Target Milestone: rc   
Target Release: 7.3   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openvswitch-2.5.0-19.git20160727.el7fdp Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1385096 Environment:
Last Closed: 2017-01-05 20:48:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1385096    
Bug Blocks:    

Comment 5 Rick Alongi 2016-11-30 14:37:01 UTC
Fix verified.  Please see testing notes below.

Thanks
Rick

Script used for test:

[root@netqe13 home]# cat bz1397045.sh 
#!/bin/bash

# Test for BZ1397045

# Function to gather detailed OVS version information
function get-ovs-ver
{
	local ovs_version=$(rpm -qi openvswitch | grep Version)
	local ovs_release=$(rpm -qi openvswitch | grep Release)
	if [[ $ovs_version ]] && [[ $ovs_release ]]; then
		echo "OVS" $ovs_version, $ovs_release
	else
		echo "Unable to gather complete OVS version information."
		return 1
	fi
}

# Install baseline openvswitch package (must be >= 2.5.0.19 to include fix for BZ1397045)
rpm -ivh http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.5.0/19.git20160727.el7fdp/x86_64/openvswitch-2.5.0-19.git20160727.el7fdp.x86_64.rpm

echo "Baseline OVS version is: $(get-ovs-ver)"
systemctl enable openvswitch && systemctl start openvswitch
echo "Sleeping 4 minutes..."
sleep 4m
ovs_uptime1=$(systemctl status openvswitch | grep Active | awk '{print $9}' | tr -d [a-z])
echo "openvswitch service uptime is: $(systemctl status openvswitch | grep Active | awk '{print $9}')"

# Upgrade to target openvswitch package
rpm -Uvh http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.5.0/22.git20160727.el7fdp/x86_64/openvswitch-2.5.0-22.git20160727.el7fdp.x86_64.rpm

echo "Upgraded OVS version is: $(get-ovs-ver)"
echo "Sleeping 2 minutes..."
sleep 2m
ovs_uptime2=$(systemctl status openvswitch | grep Active | awk '{print $9}' | tr -d [a-z])
echo "openvswitch service uptime is now: $(systemctl status openvswitch | grep Active | awk '{print $9}')"

# Confirm that openvswitch service was not restarted as part of the upgrade
if [[ $ovs_uptime2 -ge $ovs_uptime1 ]]; then
	echo "openvswitch service did not restart during upgrade.  Test PASSED"
else
	echo "openvswitch service restarted during upgrade.  Test FAILED"
fi


Problem observed pre-fix using 2.5.0.14 FDP as baseline without the fix (openvswitch service restarted during upgrade):

[root@netqe13 home]# ./bz1397045.sh 
Retrieving http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.5.0/14.git20160727.el7fdp/x86_64/openvswitch-2.5.0-14.git20160727.el7fdp.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:openvswitch-2.5.0-14.git20160727.################################# [100%]
Baseline OVS version is: OVS Version : 2.5.0, Release : 14.git20160727.el7fdp
Created symlink from /etc/systemd/system/multi-user.target.wants/openvswitch.service to /usr/lib/systemd/system/openvswitch.service.
Sleeping 4 minutes...
openvswitch service uptime is: 4min
Retrieving http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.5.0/22.git20160727.el7fdp/x86_64/openvswitch-2.5.0-22.git20160727.el7fdp.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:openvswitch-2.5.0-22.git20160727.################################# [ 50%]
Cleaning up / removing...
   2:openvswitch-2.5.0-14.git20160727.################################# [100%]
Upgraded OVS version is: OVS Version : 2.5.0, Release : 22.git20160727.el7fdp
Sleeping 2 minutes...
openvswitch service uptime is now: 2min
openvswitch service restarted during upgrade.  Test FAILED


Fix verified using 2.5.0.19 FDP as baseline with the fix (openvswitch service does not restart during upgrade):

[root@netqe13 home]# ./bz1397045.sh 
Retrieving http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.5.0/19.git20160727.el7fdp/x86_64/openvswitch-2.5.0-19.git20160727.el7fdp.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:openvswitch-2.5.0-19.git20160727.################################# [100%]
Baseline OVS version is: OVS Version : 2.5.0, Release : 19.git20160727.el7fdp
Created symlink from /etc/systemd/system/multi-user.target.wants/openvswitch.service to /usr/lib/systemd/system/openvswitch.service.
Sleeping 4 minutes...
openvswitch service uptime is: 4min
Retrieving http://download-node-02.eng.bos.redhat.com/brewroot/packages/openvswitch/2.5.0/22.git20160727.el7fdp/x86_64/openvswitch-2.5.0-22.git20160727.el7fdp.x86_64.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:openvswitch-2.5.0-22.git20160727.################################# [ 50%]
Cleaning up / removing...
   2:openvswitch-2.5.0-19.git20160727.################################# [100%]
Upgraded OVS version is: OVS Version : 2.5.0, Release : 22.git20160727.el7fdp
Sleeping 2 minutes...
openvswitch service uptime is now: 6min
openvswitch service did not restart during upgrade.  Test PASSED

Comment 8 errata-xmlrpc 2017-01-05 20:48:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2017-0027.html

Comment 9 Matt Flusche 2017-02-07 21:43:30 UTC
*** Bug 1419159 has been marked as a duplicate of this bug. ***