Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1883786

Summary: [update][osp13] Special treatment for openvswtich is currently de-activated.
Product: Red Hat OpenStack Reporter: Sofer Athlan-Guyot <sathlang>
Component: openstack-tripleo-heat-templatesAssignee: Sofer Athlan-Guyot <sathlang>
Status: CLOSED ERRATA QA Contact: Sofer Athlan-Guyot <sathlang>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 13.0 (Queens)CC: jpretori, mburns, mgarciac
Target Milestone: z14Keywords: Regression, Triaged, ZStream
Target Release: 13.0 (Queens)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-tripleo-heat-templates-8.4.1-70.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-12-16 13:55:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Sofer Athlan-Guyot 2020-09-30 08:54:26 UTC
Description of problem:

Deploy osp13z11, run a update to 2020-09-16.1.  There is no special treatment of openvswitch triggered.

This was found during validation of https://bugzilla.redhat.com/show_bug.cgi?id=1826367

See comment 19, https://bugzilla.redhat.com/show_bug.cgi?id=1826367#c19

The main part is:

2020-09-29 14:08:00 | TASK [Get current OpenvSwitch package name] ************************************
2020-09-29 14:08:00 | Tuesday 29 September 2020  13:52:26 -0400 (0:00:00.503)       0:05:27.767 ***** 

2020-09-29 14:08:00 | changed: [controller-2] => {"changed": true, "cmd": "rpm -qa | awk -F- '/^(openvswitch[0-9]+\\.[0-9]+-[0-9]+\\.[0-9]+\\.[-0]+-|openvswitch-2)/{print $1}'", "delta": "0:00:01.258453", "end": "2020-09-29 17:52:27.655876", "rc": 0, "start": "2020-09-29    17:52:26.397423", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}

We can see that the regex matched nothing.

If we don't match the name there won't be a any special treatment.

I need to evaluate the current impact as there is a potential for blocker here as the special treatment is here to prevent any cut in Floating IP availability during update.

[stack@undercloud-0 ~]$ rpm -qa | grep tripleo-heat-temp
openstack-tripleo-heat-templates-8.4.1-67.el7ost.noarch

Comment 1 Sofer Athlan-Guyot 2020-09-30 11:38:25 UTC
Ok, I know why ci isn't failing :)

so we have deactivated layered upgrade ovs special treatment ... but non-layered special treatment kicks in and save the day:

chem@s390sx:~/tmp/DFG-upgrades-updates-13-from-z9-HA-ipv4/11$ grep '2020-09-26 18:42:32 | changed:' undercloud-0/home/stack/overcloud_update_run_Controller.log | sed -e 's/\\n/\                                                                                                                                      [14/85]
> /g'                                                                                                                                                          
2020-09-26 18:42:32 | changed: [controller-2] => {"changed": true, "cmd": "docker tag 192.168.24.1:8787/rh-osbs/rhosp13-openstack-redis:20200916.1 192.168.24.1:8787/rh-osbs/rhosp13-openstack-redis:pcmklatest", "delta": "0:00:00.044712", "end": "2020-09-26 22:41:55.634251", "rc": 0, "start": "2020-09-26 22:41:55.58953
9", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}                                                                                        
2020-09-26 18:42:32 | changed: [controller-2] => {"changed": true, "cmd": "rpm -qa | awk -F- '/^(openvswitch[0-9]+\\.[0-9]+-[0-9]+\\.[0-9]+\\.[-0]+-|openvswitch-2)/{print $1}'", "delta": "0:00:01.377361", "end": "2020-09-26 22:41:57.903441", "rc": 0, "start": "2020-09-26 22:41:56.526080", "stderr": "", "stderr_lines"
: [], "stdout": "openvswitch", "stdout_lines": ["openvswitch"]}

===> HERE: we match openvswitch, not rhosp-openvswitch, see the ["openvswitch"] in the line above.  Then we go on doing special treatment there.


2020-09-26 18:42:32 | changed: [controller-2] => {"changed": true, "cmd": "set -o pipefail
 yum info -q rhosp-openvswitch | awk '/^Version/{print $NF}'", "delta": "0:00:01.384730", "end": "2020-09-26 22:42:00.004526", "rc": 0, "start": "2020-09-26 22:41:58.619796", "stderr": "", "stderr_lines": [], "stdout": "2.11", "stdout_lines": ["2.11"]}
2020-09-26 18:42:32 | changed: [controller-2] => {"changed": true, "cmd": "rpm -qi \"openvswitch\" | awk '/^Version/{print $NF}'", "delta": "0:00:00.050173", "end": "2020-09-26 22:42:01.085754", "rc": 0, "start": "2020-09-26 22:42:01.035581", "stderr": "", "stderr_lines": [], "stdout": "2.9.0", "stdout_lines": ["2.9.
0"]}                                                                                                                                                           
2020-09-26 18:42:32 | changed: [controller-2] => {"changed": true, "cmd": "rpm -e --noscripts --nopreun --nopostun --notriggers --nodeps $(rpm -qa 'openvswitch*' | grep -v 'selinux')", "delta": "0:00:01.466809", "end": "2020-09-26 22:42:04.121166", "rc": 0, "start": "2020-09-26 22:42:02.654357", "stderr": "warning: /
etc/sysconfig/openvswitch saved as /etc/sysconfig/openvswitch.rpmsave", "stderr_lines": ["warning: /etc/sysconfig/openvswitch saved as /etc/sysconfig/openvswitch.rpmsave"], "stdout": "", "stdout_lines": []}
2020-09-26 18:42:32 | changed: [controller-2] => {"changed": true, "msg": "Warning: RPMDB altered outside of yum.
** Found 5 pre-existing rpmdb problem(s), 'yum check' output follows:                                                                                                                                                                                                                                                         
1:openstack-neutron-openvswitch-12.1.0-2.el7ost.noarch has missing requires of openvswitch
python-networking-ovn-metadata-agent-4.0.3-14.el7ost.noarch has missing requires of openvswitch >= ('0', '2.8.0', None)
python-openvswitch-2.9.0-117.bz1733374.1.el7ost.x86_64 has missing requires of libopenvswitch-2.9.so.0()(64bit)
python-openvswitch-2.9.0-117.bz1733374.1.el7ost.x86_64 has missing requires of libopenvswitch-2.9.so.0(libopenvswitch_0)(64bit)
python-openvswitch-2.9.0-117.bz1733374.1.el7ost.x86_64 has missing requires of openvswitch = ('0', '2.9.0', '117.bz1733374.1.el7ost')
", "rc": 0, "results": ["Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Resolving Dependencies
--> Running transaction check
---> Package rhosp-openvswitch.noarch 0:2.11-0.7.el7ost will be installed
--> Processing Dependency: openvswitch2.11 for package: rhosp-openvswitch-2.11-0.7.el7ost.noarch
--> Running transaction check
---> Package openvswitch2.11.x86_64 0:2.11.3-64.el7fdp will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package             Arch     Version               Repository             Size 
================================================================================
Installing:
 rhosp-openvswitch   noarch   2.11-0.7.el7ost       rhelosp-13.0-puddle   3.4 k 
Installing for dependencies:
 openvswitch2.11     x86_64   2.11.3-64.el7fdp      rhelosp-13.0-puddle    12 M 

Transaction Summary
================================================================================
Install  1 Package (+1 Dependent package)

Total download size: 12 M
Installed size: 35 M
Downloading packages:
--------------------------------------------------------------------------------
Total                                               57 MB/s |  12 MB  00:00      
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : openvswitch2.11-2.11.3-64.el7fdp.x86_64                      1/2  
  Installing : rhosp-openvswitch-2.11-0.7.el7ost.noarch                     2/2 


And so on.

ping test works, nothing break.

So we know we have something wrong but we also have an automatic workaround. The code is robust enough to survive that error.

We should fix the code, but I cannot see a case for a blocker. I'm open to discussion though.

Comment 12 errata-xmlrpc 2020-12-16 13:55:05 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 13.0 director bug fix advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5575