Description of problem: A FFU from OSP 13 to OSP 16.1 was being executed. While running "openstack overcloud upgrade prepare" for the first time, there were some errors in templates. After fixing all the template errors, "openstack overcloud upgrade prepare" wasn't proceeding and the error was: [..] Removing the current plan files Uploading new plan files The action raised an exception [action_ex_id=6c652ff6-d261-4e71-be35-eda798b3ceb5, msg='argument of type 'NoneType' is not iterable', action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{'container': 'overcloud'}'] 2020-12-29 11:21:13.676 1041087 ERROR openstack [-] Exception updating plan: The action raised an exception [action_ex_id=6c652ff6-d261-4e71-be35-eda798b3ceb5, msg='argument of type 'NoneType' is not iterable', action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{'container': 'overcloud'}']: tripleoclient.exceptions.WorkflowServiceError: Exception updating plan: The action raised an exception [action_ex_id=6c652ff6-d261-4e71-be35-eda798b3ceb5, msg='argument of type 'NoneType' is not iterable', action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{'container': 'overcloud'}'] 2020-12-29 11:21:13.680 1041087 INFO osc_lib.shell [-] END return value: 1 [..] The solution was to revert to undercloud backup (ReaR image in OSP 13), upgrade again the undercloud to OSP 16.1 and continue with overcloud upgrade. Version-Release number of selected component (if applicable): openstack-tripleo-common-11.4.1-1.20200917023445.el8ost.noarch openstack-tripleo-heat-templates-11.3.2-1.20200914170177.el8ost.noarch python3-tripleoclient-12.3.2-1.20200914164930.el8ost.noarch Steps to Reproduce: 1. Follow documented steps to upgrade undercloud from OSP 13 to OSP 16.1 2. Run "openstack overcloud upgrade prepare" with at least the following errors in templates: a) have OS::TripleO::Services::NovaPlacement instead of OS::TripleO::Services::PlacementApi in roles_data.yaml b) Use TLS for public endpoints and have NovaPlacement{Admin,Internal,Public} in EndpointMap instead of Placement{Admin,Internal,Public} 3. Fix aforementioned errors and run again "openstack overcloud upgrade prepare" with the same set of environment files. At this poing, even using "openstack overcloud upgrade prepare" without providing any set of environment files will result in the same error. Actual results: "openstack overcloud upgrade prepare" fails with: Removing the current plan files Uploading new plan files The action raised an exception [action_ex_id=6c652ff6-d261-4e71-be35-eda798b3ceb5, msg='argument of type 'NoneType' is not iterable', action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{'container': 'overcloud'}'] 2020-12-29 11:21:13.676 1041087 ERROR openstack [-] Exception updating plan: The action raised an exception [action_ex_id=6c652ff6-d261-4e71-be35-eda798b3ceb5, msg='argument of type 'NoneType' is not iterable', action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{'container': 'overcloud'}']: tripleoclient.exceptions.WorkflowServiceError: Exception updating plan: The action raised an exception [action_ex_id=6c652ff6-d261-4e71-be35-eda798b3ceb5, msg='argument of type 'NoneType' is not iterable', action_cls='<class 'mistral.actions.action_factory.GeneratePasswordsAction'>', attributes='{}', params='{'container': 'overcloud'}'] 2020-12-29 11:21:13.680 1041087 INFO osc_lib.shell [-] END return value: 1 Expected results: "openstack overcloud upgrade prepare" should work if the templates are fixed.
Missed section in documentation was https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/framework_for_upgrades_13_to_16.1/index#updating-composable-services-and-parameters But I do agree that we should add additional protection into python-tripleoclient.
Created attachment 1751997 [details] Instructions to apply the hotfix in the Undercloud The attached text file contains all the steps required to apply the hotfix manually, as well as via an automated Ansible playbook. The playbook has been tested in a test environment, but the conditions might differ from the customer's environment.
Created attachment 1751998 [details] Signed RPMs including the fix for BZ 1911653 The attached tar file includes all the RPMs needed to apply the hotfix: openstack-tripleo-common openstack-tripleo-common-container-base openstack-tripleo-common-containers openstack-tripleo-common-devtools python3-tripleo-common Follow the steps provided in the attached guide to build the container image and redeploy the faulty container.
Created attachment 1752000 [details] Ansible playbook to automate the hotfix installation Attaching the Ansible playbook with the tasks that automate the hotfix installation. The playbook is also available in a public Github gist: https://gist.githubusercontent.com/jfrancoa/2cf943e925b6061f4df0eda0e1fb14dc/raw/f20f063bc2d19941a9c8c1bfc248b8e97db9bef4/hotfix_apply_undercloud.yaml
Moving to 16.1.5
*** Bug 1921745 has been marked as a duplicate of this bug. ***
Verified by specifying OS::TripleO::Services::NovaPlacement in ~/roles_data.yaml and running openstack overcloud upgrade prepare \ ... -r ~/roles_data.yaml \ During the process a deprecation message appeared and "upgrade prepare" didn't fail.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0817