Description of problem: OSP8->OSP9 upgrade fails on upgrading pacemaker, with a nova db error When deploying with tripleo-quickstart, with virtualized environment, i'm getting the following error: Notice: /Stage[main]/Main/Exec[nova-db-online-data-migrations]/returns: usage: nova-manage db [-h]\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[nova-db-online-data-migrations]/returns: \u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[nova-db-online-data-migrations]/returns: {archive_deleted_rows,contract,expand,migrate,null_instance_uuid_scan,sync,version}\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[nova-db-online-data-migrations]/returns: ...\u001b[0m\n\u001b[mNotice: /Stage[main]/Main/Exec[nova-db-online-data-migrations]/returns: nova-manage db: error: argument action: invalid choice: 'online_data_migrations' (choose from 'archive_deleted_rows', 'contract', 'expand', 'migrate', 'null_instance_uuid_scan', 'sync', 'version') That happens when executing the following command on a quickstart undercloud: openstack overcloud deploy --templates tripleo-heat-templates -e tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e tripleo-heat-templates/environments/network-isolation.yaml -e tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e ~/network-environment.yaml -e tripleo-heat-templates/environments/puppet-pacemaker.yaml -e tripleo-heat-templates/environments/major-upgrade-pacemaker.yaml Undercloud has been successfully upgraded, and previous step of the overcloud (aodh, keystone), worked as well.
Going on comment #0 - looks like the nova-db-online-data-migrations exec is missing in puppet-nova on the environment. I went looking at stable/mitaka https://github.com/openstack/puppet-nova/tree/stable/mitaka/manifests/db and indeed it isn't there. So either it wasn't backported or we had to do it downstream only as Mitaka-->EOL. Leaving untriaged for now to be discussed later today - imo this is more likely DFG:Compute or DFG:DF to investigate & fixup the backport if necessary. Also as with a similar BZ for 8-9 we should get some jobs going to see if we can repro these issues will ask QE flks on the call today.
I tested the 8 to 9 upgrade process on a 3 controller, 2 computes, 3 ceph nodes environment and I wasn't able to reproduce the issue. Could you please let us know what version of openstack-puppet-modules was installed on the nodes at failure time?
I have that before starting the upgrade: Name : openstack-puppet-modules Arch : noarch Epoch : 1 Version : 7.1.5 Release : 1.el7ost Size : 7.1 M Repo : installed From repo : iso Summary : Collection of Puppet modules for OpenStack deployment URL : https://github.com/redhat-openstack License : ASL 2.0 and GPLv2 and GPLv3 Description : A collection of Puppet modules which are required to install and configure : OpenStack via installers using Puppet configuration tool.
Having done the upgrade to latest OSP8, then upgrading undercloud to OSP9, i'm not hitting this error any more.
Thanks Yolanda for the update. You might have some issues with the setting of the OSP repos perhaps ?