Description of problem:
Following an upgrade to OSP 10, instances that were created before the upgrade produce an error when attempting to resize.
Resize() command failed for VM with UUID: aa8d6f33-4b62-4ea1-aa15-21266eaaf50d. Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. <class 'RequestSpecNotFound_Remote'> (HTTP 500) (Request-ID: req-c1af65e1-2072-4168-b0f4-71e0c36e6ee5)
This is due to 'nova-manage db online_data_migrations' not being run as part of the upgrade per https://bugs.launchpad.net/nova/+bug/1707934
After running this manually, an error is produced and the command does not complete:
# nova-manage db online_data_migrations
Option "rpc_backend" from group "DEFAULT" is deprecated for removal. Its value may be silently ignored in the future.
Option "notification_driver" from group "DEFAULT" is deprecated. Use option "driver" from group "oslo_messaging_notifications".
Option "notification_topics" from group "DEFAULT" is deprecated. Use option "topics" from group "oslo_messaging_notifications".
Running batches of 50 until complete
Error attempting to run <function migrate_instances_add_request_spec at 0x5c13758>
After adding a raise to the exception block in /nova/cmd/manage.py a mysql error was observed:
error: (pymysql.err.DataError) (1406, u"Data too long for column 'spec' at row 1") [SQL: u'INSERT INTO request_specs
This is due to the RequestSpec.instance_group.members being massive. After manually skipping the affected instances, the nova-manage command was able to complete and the resize operation began working as expected.
Version-Release number of selected component (if applicable):
Every time with this deployment
Steps to Reproduce:
1. Upgrade from OSP 8 -> 9 -> 10
2. Attempt to migrate an instance that was created before the upgrade
An error is generated, but the resize is actually successful and can be confirmed. Error is not generated after implementing workaround to allow 'nova-manage db online_data_migrations' to complete.
No error is produced
Upstream bug has been opened to address the mysql error https://bugs.launchpad.net/nova/+bug/1738094
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.