Description of problem: As part of Nuage Integration, stock openvswitch on overcloud nodes gets replaced with nuage-openvswitch. I started following steps in "Keeping Red Hat OpenStack Platform Updated" to update Red Hat packages and containers on overcloud nodes to latest and ran into below issue $ openstack overcloud update run --nodes Controller ... Truncated output ... TASK [Get current OpenvSwitch package name] ************************************ Thursday 26 December 2019 12:37:20 -0500 (0:00:00.121) 0:07:24.640 ***** changed: [overcloud-controller-0] => {"changed": true, "cmd": "rpm -qa | awk -F- '/^(openvswitch[0-9]+\\.[0-9]+-|openvswitch-2)/{print $1}'", "delta": "0:00:01.002803", "end": "2019-12-26 17:37:19.232520", "rc": 0, "start": "2019-12-26 17:37:18.229717", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [Get version from current OpenvSwitch package] **************************** Thursday 26 December 2019 12:37:21 -0500 (0:00:01.433) 0:07:26.073 ***** changed: [overcloud-controller-0] => {"changed": true, "cmd": "rpm -qi \"\" | awk '/^Version/{print $NF}'", "delta": "0:00:00.034623", "end": "2019-12-26 17:37:19.550037", "rc": 0, "start": "2019-12-26 17:37:19.515414", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [split numeric version for OpenvSwitch into parts] ************************ Thursday 26 December 2019 12:37:22 -0500 (0:00:00.340) 0:07:26.414 ***** ok: [overcloud-controller-0] => {"ansible_facts": {"ovs_version_parts": [""]}, "changed": false} TASK [get major minor version for OpenvSwitch package naming] ****************** Thursday 26 December 2019 12:37:22 -0500 (0:00:00.129) 0:07:26.544 ***** fatal: [overcloud-controller-0]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: list object has no element 1\n\nThe error appears to have been in '/var/lib/mistral/0babf1d0-a548-4fd5-b2f5-4a64cad073b0/Controller/update_tasks.yaml': line 367, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n ovs_version_parts: '{{ ovs_version_out.stdout.split(''.'') }}'\n - name: get major minor version for OpenvSwitch package naming\n ^ here\n"} PLAY RECAP ********************************************************************* overcloud-controller-0 : ok=81 changed=41 unreachable=0 failed=1 Thursday 26 December 2019 12:37:22 -0500 (0:00:00.031) 0:07:26.576 ***** =============================================================================== Version-Release number of selected component (if applicable): ansible-tripleo-ipsec.noarch 8.1.1-0.20190513184007.7eb892c.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-common.noarch 8.7.1-2.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-common-containers.noarch 8.7.1-2.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-heat-templates.noarch 8.4.1-16.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-image-elements.noarch 8.0.3-1.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-puppet-elements.noarch 8.1.1-1.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-ui.noarch 8.3.2-3.el7ost @rhel-7-server-openstack-13-rpms openstack-tripleo-validations.noarch 8.5.0-2.el7ost @rhel-7-server-openstack-13-rpms puppet-tripleo.noarch 8.5.1-3.el7ost @rhel-7-server-openstack-13-rpms python-tripleoclient.noarch 9.3.1-4.el7ost @rhel-7-server-openstack-13-rpms How reproducible: Always Steps to Reproduce: 1. Deploy Red Hat OpenStack Platform release 13.0.7 with Nuage Integration. 2. Start following "Keeping Red Hat OpenStack Platform Updated" to update overcloud nodes. Actual results: TASK [Get current OpenvSwitch package name] ************************************ Thursday 26 December 2019 12:37:20 -0500 (0:00:00.121) 0:07:24.640 ***** changed: [overcloud-controller-0] => {"changed": true, "cmd": "rpm -qa | awk -F- '/^(openvswitch[0-9]+\\.[0-9]+-|openvswitch-2)/{print $1}'", "delta": "0:00:01.002803", "end": "2019-12-26 17:37:19.232520", "rc": 0, "start": "2019-12-26 17:37:18.229717", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [Get version from current OpenvSwitch package] **************************** Thursday 26 December 2019 12:37:21 -0500 (0:00:01.433) 0:07:26.073 ***** changed: [overcloud-controller-0] => {"changed": true, "cmd": "rpm -qi \"\" | awk '/^Version/{print $NF}'", "delta": "0:00:00.034623", "end": "2019-12-26 17:37:19.550037", "rc": 0, "start": "2019-12-26 17:37:19.515414", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [split numeric version for OpenvSwitch into parts] ************************ Thursday 26 December 2019 12:37:22 -0500 (0:00:00.340) 0:07:26.414 ***** ok: [overcloud-controller-0] => {"ansible_facts": {"ovs_version_parts": [""]}, "changed": false} TASK [get major minor version for OpenvSwitch package naming] ****************** Thursday 26 December 2019 12:37:22 -0500 (0:00:00.129) 0:07:26.544 ***** fatal: [overcloud-controller-0]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: list object has no element 1\n\nThe error appears to have been in '/var/lib/mistral/0babf1d0-a548-4fd5-b2f5-4a64cad073b0/Controller/update_tasks.yaml': line 367, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n ovs_version_parts: '{{ ovs_version_out.stdout.split(''.'') }}'\n - name: get major minor version for OpenvSwitch package naming\n ^ here\n"} Expected results: Skipping the above tasks if stock openvswitch is not present. Additional info:
Additional info: Currently, these tasks are not part of a condition so they can not be disabled through configuration.
I'm working on a patch https://review.opendev.org/#/c/701383/. The approach I'm proposing is that if openvswitch or openvswitchM.N isn't installed then we bypass our special handling. The consequence is that nuage integration will have to manage updating openvswitch on it's own. Is this okay?
I tested the same logic provided in the patch https://review.opendev.org/#/c/701383/ in update_tasks of OSPD13 and it is working fine. I also added a small improvement on the patch set. Thanks, Sai
Hi, Recently we got hold of hotfix package openstack-tripleo-heat-templates-8.4.1-33.el7ost and updated existing openstack-tripleo-heat-templates-8.4.1-16.el7ost to this hotfix. Later we started testing the update of overcloud and then we ran into this new issue [1]. The Hot fix package that was provided, itself is not suffiecient to test the upgrade, there are some dependent packages changes in other packages like openstack-tripleo-common where we have definition for workflow "tripleo.nova.v1.cellv2_discovery" inside "nova_host_discovery.yaml" workbook (https://github.com/openstack/tripleo-common/blob/stable/queens/workbooks/nova_host_discovery.yaml). The current openstack-tripleo-common-8.7.1-2.el7ost does not have this workbook. So, in order for us to move forward with testing, we need all those dependent packages for openstack-tripleo-heat-templates-8.4.1-33.el7ost. Can you please provide us with all the dependent packages for openstack-tripleo-heat-templates-8.4.1-33.el7ost. Thanks, Sai [1] 2020-02-26 17:36:05Z [AllNodesDeploySteps]: UPDATE_IN_PROGRESS state changed 2020-02-26 17:36:08Z [AllNodesDeploySteps]: UPDATE_FAILED resources.AllNodesDeploySteps: Property error: resources.WorkflowTasks_Step5.properties.tasks[0].workflow: Error validating value 'tripleo.nova.v1.cellv2_discovery': The Workflow (tripleo.nova.v1.cellv2_discovery) could not be found. 2020-02-26 17:36:08Z [overcloud]: UPDATE_FAILED Resource UPDATE failed: resources.AllNodesDeploySteps: Property error: resources.WorkflowTasks_Step5.properties.tasks[0].workflow: Error validating value 'tripleo .nova.v1.cellv2_discovery': The Workflow (tripleo.nova.v1.cellv2_discovery) could not be foun Stack overcloud UPDATE_FAILED START with options: ['stack', 'failures', 'list', u'overcloud'] command: stack failures list -> heatclient.osc.v1.stack_failures.ListStackFailures (auth=True) Using auth plugin: password overcloud.AllNodesDeploySteps: resource_type: OS::TripleO::PostDeploySteps physical_resource_id: 63157a18-1ff1-4d36-8b43-6f81fc605fe4 status: UPDATE_FAILED status_reason: | resources.AllNodesDeploySteps: Property error: resources.WorkflowTasks_Step5.properties.tasks[0].workflow: Error validating value 'tripleo.nova.v1.cellv2_discovery': The Workflow (tripleo.nova.v1.cellv2_disc overy) could not be found.
According to our records, this should be resolved by openstack-tripleo-heat-templates-8.4.1-42.el7ost. This build is available now.