Bug 1661810 - [OVN] upgrade failed ovn osp13 to osp14 on Set fact networking_ovn_metadata_agent_enabled task
Summary: [OVN] upgrade failed ovn osp13 to osp14 on Set fact networking_ovn_metadata...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 14.0 (Rocky)
Hardware: Unspecified
OS: Unspecified
medium
urgent
Target Milestone: ga
: 14.0 (Rocky)
Assignee: Daniel Alvarez Sanchez
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-12-23 14:37 UTC by Eran Kuris
Modified: 2019-09-09 14:54 UTC (History)
12 users (show)

Fixed In Version: openstack-tripleo-heat-templates-9.0.1-0.20181013060907
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-11 11:55:18 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (16.34 MB, application/x-tar)
2018-12-24 08:40 UTC, Eran Kuris
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1809645 0 None None None 2018-12-24 08:55:15 UTC
OpenStack gerrit 627163 0 None None None 2018-12-24 09:06:56 UTC
Red Hat Product Errata RHEA-2019:0045 0 None None None 2019-01-11 11:55:28 UTC

Description Eran Kuris 2018-12-23 14:37:45 UTC
Description of problem:
Processing of upgrade OVN-osp13 to osp14 failed on overcloud_upgrade_run_Compute
step in task "Set fact networking_ovn_metadata_agent_enabled"



2018-12-23 07:57:40 |  u'',
2018-12-23 07:57:40 |  u'TASK [Set fact networking_ovn_metadata_agent_enabled] **************************',
2018-12-23 07:57:40 |  u'Sunday 23 December 2018  07:57:38 -0500 (0:00:00.392)       0:01:08.108 ******* ',
2018-12-23 07:57:40 |  u'fatal: [compute-1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \'networking_ovn_metadata_agent_enabled\' is undefined\\n\\nThe error appears to have been in \'/var/lib/mistral/ac14c5f3-92da-450e-b216-8cebb4399cda/Compute/upgrade_tasks.yaml\': line 112, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n    register: networking_ovn_metadata_agent_enabled_result\\n  - name: Set fact networking_ovn_metadata_agent_enabled\\n    ^ here\\n"}',
2018-12-23 07:57:40 |  u'fatal: [compute-0]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \'networking_ovn_metadata_agent_enabled\' is undefined\\n\\nThe error appears to have been in \'/var/lib/mistral/ac14c5f3-92da-450e-b216-8cebb4399cda/Compute/upgrade_tasks.yaml\': line 112, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n    register: networking_ovn_metadata_agent_enabled_result\\n  - name: Set fact networking_ovn_metadata_agent_enabled\\n    ^ here\\n"}',
2018-12-23 07:57:40 |  u'',
2018-12-23 07:57:40 |  u'PLAY RECAP *********************************************************************',
2018-12-23 07:57:40 |  u'compute-0                  : ok=23   changed=8    unreachable=0    failed=1   ',
2018-12-23 07:57:40 |  u'compute-1                  : ok=23   changed=8    unreachable=0    failed=1   ',
2018-12-23 07:57:40 |  u'',
2018-12-23 07:57:40 |  u'Sunday 23 December 2018  07:57:38 -0500 (0:00:00.088)       0:01:08.197 ******* ',
2018-12-23 07:57:40 |  u'=============================================================================== ']
2018-12-23 07:57:40 | [u'Failed to update nodes - Compute, please see the logs.']
@                                                                                           
Version-Release number of selected component (if applicable):

from osp13 z3 to osp14 2018-12-20.4
How reproducible:

100% 
Steps to Reproduce:
1.run upgrade job.
2.
3.

Actual results:


Expected results:


Additional info:
https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/networking-ovn/job/DFG-network-networking-ovn-upgrade-13-14_director-rhel-virthost-3cont_2comp_2net-ipv4-geneve-composable/58/

Comment 1 Eran Kuris 2018-12-24 08:40:34 UTC
Created attachment 1516483 [details]
logs

Comment 2 Daniel Alvarez Sanchez 2018-12-24 09:11:35 UTC
This patch [0] merged in stable/rocky introduced the bug which wasn't caught upstream since we don't run upgrades using ML2/OVN.
I just post a fix to review [1].


[0] https://github.com/openstack/tripleo-heat-templates/commit/56bec75c02b56bab79f35bc94c9110ee93eab82f#diff-414c32eadb8359e81046e9356fead25a
[1] https://review.openstack.org/#/c/627163

Comment 19 errata-xmlrpc 2019-01-11 11:55:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2019:0045


Note You need to log in before you can comment on or make changes to this bug.