Description of problem: Processing of upgrade OVN-osp13 to osp14 failed on overcloud_upgrade_run_Compute step in task "Set fact networking_ovn_metadata_agent_enabled" 2018-12-23 07:57:40 | u'', 2018-12-23 07:57:40 | u'TASK [Set fact networking_ovn_metadata_agent_enabled] **************************', 2018-12-23 07:57:40 | u'Sunday 23 December 2018 07:57:38 -0500 (0:00:00.392) 0:01:08.108 ******* ', 2018-12-23 07:57:40 | u'fatal: [compute-1]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \'networking_ovn_metadata_agent_enabled\' is undefined\\n\\nThe error appears to have been in \'/var/lib/mistral/ac14c5f3-92da-450e-b216-8cebb4399cda/Compute/upgrade_tasks.yaml\': line 112, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n register: networking_ovn_metadata_agent_enabled_result\\n - name: Set fact networking_ovn_metadata_agent_enabled\\n ^ here\\n"}', 2018-12-23 07:57:40 | u'fatal: [compute-0]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: \'networking_ovn_metadata_agent_enabled\' is undefined\\n\\nThe error appears to have been in \'/var/lib/mistral/ac14c5f3-92da-450e-b216-8cebb4399cda/Compute/upgrade_tasks.yaml\': line 112, column 5, but may\\nbe elsewhere in the file depending on the exact syntax problem.\\n\\nThe offending line appears to be:\\n\\n register: networking_ovn_metadata_agent_enabled_result\\n - name: Set fact networking_ovn_metadata_agent_enabled\\n ^ here\\n"}', 2018-12-23 07:57:40 | u'', 2018-12-23 07:57:40 | u'PLAY RECAP *********************************************************************', 2018-12-23 07:57:40 | u'compute-0 : ok=23 changed=8 unreachable=0 failed=1 ', 2018-12-23 07:57:40 | u'compute-1 : ok=23 changed=8 unreachable=0 failed=1 ', 2018-12-23 07:57:40 | u'', 2018-12-23 07:57:40 | u'Sunday 23 December 2018 07:57:38 -0500 (0:00:00.088) 0:01:08.197 ******* ', 2018-12-23 07:57:40 | u'=============================================================================== '] 2018-12-23 07:57:40 | [u'Failed to update nodes - Compute, please see the logs.'] @ Version-Release number of selected component (if applicable): from osp13 z3 to osp14 2018-12-20.4 How reproducible: 100% Steps to Reproduce: 1.run upgrade job. 2. 3. Actual results: Expected results: Additional info: https://rhos-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/DFG/view/network/view/networking-ovn/job/DFG-network-networking-ovn-upgrade-13-14_director-rhel-virthost-3cont_2comp_2net-ipv4-geneve-composable/58/
Created attachment 1516483 [details] logs
This patch [0] merged in stable/rocky introduced the bug which wasn't caught upstream since we don't run upgrades using ML2/OVN. I just post a fix to review [1]. [0] https://github.com/openstack/tripleo-heat-templates/commit/56bec75c02b56bab79f35bc94c9110ee93eab82f#diff-414c32eadb8359e81046e9356fead25a [1] https://review.openstack.org/#/c/627163
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045