Bug 2061666

Summary: FFU from RHOSP 13 to RHOSP16.2 failing in undercloud upgrade at Openvswitch task
Product: Red Hat OpenStack Reporter: Aman Gupta <amagupta>
Component: tripleo-ansibleAssignee: Sofer Athlan-Guyot <sathlang>
Status: CLOSED ERRATA QA Contact: Archana Singh <arcsingh>
Severity: medium Docs Contact:
Priority: medium    
Version: 16.2 (Train)CC: arcsingh, jgrosso, jlarriba, lbezdick, mburns, sathlang
Target Milestone: z5Keywords: Triaged
Target Release: 16.2 (Train on RHEL 8.4)   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: tripleo-ansible-0.8.1-2.20230302004949.0db9f93.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-04-26 12:16:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Aman Gupta 2022-03-08 08:43:06 UTC
Description of problem:

FFU from RHOSP 13 to RHOSP16.2 failing in undercloud upgrade at Openvswitch task

Version-Release number of selected component (if applicable):
RHOSP 16.2

How reproducible:


Steps to Reproduce:

    2022-03-06 11:27:07.276183 | 00505683-3954-5ce5-1750-0000000001ed |       TASK | Special treatment for OpenvSwitch
    host: eurepbdprov01, task: TASK: Always ensure the openvswitch service is enabled and running after upgrades
    eurepbdprov01 still blocked
    host: eurepbdprov01, task: TASK: Always ensure the openvswitch service is enabled and running after upgrades
    eurepbdprov01 still blocked
    Using module file /usr/share/ansible/plugins/modules/tripleo_ovs_upgrade.py
    Pipelining is enabled.
    <172.20.62.1> ESTABLISH LOCAL CONNECTION FOR USER: root
    <172.20.62.1> EXEC /bin/sh -c '/usr/bin/python3 && sleep 0'
    host: eurepbdprov01, task: TASK: Always ensure the openvswitch service is enabled and running after upgrades
    eurepbdprov01 still blocked
    host: eurepbdprov01, task: TASK: Always ensure the openvswitch service is enabled and running after upgrades
    eurepbdprov01 still blocked
    ...
    ...
    ...
    ...
    host: eurepbdprov01, task: TASK: Always ensure the openvswitch service is enabled and running after upgrades
    eurepbdprov01 still blocked
    host: eurepbdprov01, task: TASK: Always ensure the openvswitch service is enabled and running after upgrades
    eurepbdprov01 still blocked
    The full traceback is:
    Traceback (most recent call last):
      File "<stdin>", line 102, in <module>
      File "<stdin>", line 94, in _ansiballz_main
      File "<stdin>", line 40, in invoke_module
      File "/usr/lib64/python3.6/runpy.py", line 205, in run_module
        return _run_module_code(code, init_globals, run_name, mod_spec)
      File "/usr/lib64/python3.6/runpy.py", line 96, in _run_module_code
        mod_name, mod_spec, pkg_name, script_name)
      File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py", line 267, in <module>
      File "/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py", line 256, in main
      File "/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py", line 181, in layer_product_upgrade
      File "/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py", line 125, in flatten_version
    IndexError: list index out of range
    2022-03-06 11:27:14.947283 | 00505683-3954-5ce5-1750-0000000001ed |      FATAL | Special treatment for OpenvSwitch | eurepbdprov01 | error={
        "changed": false,
        "module_stderr": "Traceback (most recent call last):\n  File \"<stdin>\", line 102, in <module>\n  File \"<stdin>\", line 94, in _ansiballz_main\n  File \"<stdin>\", line 40, in invoke_module\n  File \"/usr/lib64/python3.6/runpy.py\",
     line 205, in run_module\n    return _run_module_code(code, init_globals, run_name, mod_spec)\n  File \"/usr/lib64/python3.6/runpy.py\", line 96, in _run_module_code\n    mod_name, mod_spec, pkg_name, script_name)\n  File \"/usr/lib64/pyt
    hon3.6/runpy.py\", line 85, in _run_code\n    exec(code, run_globals)\n  File \"/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py\", line 267, in <module>\n  F
    ile \"/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py\", line 256, in main\n  File \"/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_up
    grade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py\", line 181, in layer_product_upgrade\n  File \"/tmp/ansible_tripleo_ovs_upgrade_payload_vikgfhcx/ansible_tripleo_ovs_upgrade_payload.zip/ansible/modules/tripleo_ovs_upgrade.py\", l
    ine 125, in flatten_version\nIndexError: list index out of range\n",
        "module_stdout": "",
        "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
        "rc": 1
    }

1.
2.
3.

Actual results:

FFU should complete without any issue

Expected results:


Additional info:
All logs and sos-report in Case Id:03158783 
Issue identified by lbezdick and chem in rhos-upgrade channel.

Comment 1 Sofer Athlan-Guyot 2022-03-08 08:49:31 UTC
The root cause of the issue is unsupported localization by the ansible
python module. In spanish dnf info -q rhosp-openstack returns Versión
while we expect Version.

Example of localized dnf output:

    $ LANG=es_ES.UTF-8 dnf info -q rhosp-openvswitch
    Paquetes instalados
    Nombre       : rhosp-openvswitch
    Versión      : 2.15
    Lanzamiento  : 4.el8ost.1
    Arquitectura : noarch
    Tamaño       : 0.0
    Fuente       : rhosp-openvswitch-2.15-4.el8ost.1.src.rpm
    Repositorio  : @System
    Desde repo   : rhelosp-16.2
    Resumen      : Wrapper rpm to allow installing OVS with new versioning schemes
    URL          : http://www.openvswitch.org
    Licencia     : Public domain
    Descripción  : Wrapper rpm for the base openvswitch package

Comment 3 Sofer Athlan-Guyot 2022-12-07 20:41:30 UTC
Reverting to post as we discover an issue with the original patch and we need an additional patch to fix it.

Comment 12 errata-xmlrpc 2023-04-26 12:16:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.2.5 (Train) bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:1763