We have a solution in testing for OSP 12/13, we should implement one for OSP 14 too, it's going to be slightly different due to the differences in upgrade framework since 14+. Currently online data migrations in 14 still happen during controller upgrade.
Verified with openstack-tripleo-heat-templates-9.0.1-0.20181013060906.el7ost.noarch Ran next command before upgrade converge. openstack overcloud external-upgrade run \ --stack qe-Cloud-0 \ --tags online_upgrade 2>&1 ...2018-12-17 08:21:25 | u'TASK [Online data migration for Cinder] ****************************************', 2018-12-17 08:21:25 | u'Monday 17 December 2018 08:21:17 -0500 (0:00:00.098) 0:00:32.496 ******* ', 2018-12-17 08:21:25 | u'changed: [undercloud -> 192.168.24.8] => {"changed": true, "cmd": ["docker", "exec", "cinder_api", "cinder-manage", "db", "online_data_migrations"], "delta": "0:00:02.313713", "end": "2018-12-17 13:21:19.709889", "rc": 0, "start": "2018-12-17 13:21:17.396176", "stderr": "Deprecated: Option \\"logdir\\" from group \\"DEFAULT\\" is deprecated. Use option \\"log-dir\\" from group \\"DEFAULT\\".", "stderr_lines": ["Deprecated: Option \\"logdir\\" from group \\"DEFAULT\\" is deprecated. Use option \\"log-dir\\" from group \\"DEFAULT\\"."], "stdout": "Running batches of 50 until complete.\\n+--------------------------------------------+--------------+-----------+\\n| Migration | Total Needed | Completed |\\n+--------------------------------------------+--------------+-----------+\\n| attachment_specs_online_data_migration | 0 | 0 |\\n| backup_service_online_migration | 0 | 0 |\\n| service_uuids_online_data_migration | 0 | 0 |\\n| shared_targets_online_data_migration | 0 | 0 |\\n| volume_service_uuids_online_data_migration | 0 | 0 |\\n+--------------------------------------------+--------------+-----------+", "stdout_lines": ["Running batches of 50 until complete.", "+--------------------------------------------+--------------+-----------+", "| Migration | Total Needed | Completed |", "+--------------------------------------------+--------------+-----------+", "| attachment_specs_online_data_migration | 0 | 0 |", "| backup_service_online_migration | 0 | 0 |", "| service_uuids_online_data_migration | 0 | 0 |", "| shared_targets_online_data_migration | 0 | 0 |", "| volume_service_uuids_online_data_migration | 0 | 0 |", "+--------------------------------------------+--------------+-----------+"]}', 2018-12-17 08:21:25 | u'', 2018-12-17 08:21:25 | u'TASK [Online data migration for Nova] ******************************************', 2018-12-17 08:21:25 | u'Monday 17 December 2018 08:21:19 -0500 (0:00:02.767) 0:00:35.263 ******* ', 2018-12-17 08:21:26 | u'changed: [undercloud -> 192.168.24.8] => {"changed": true, "cmd": ["docker", "exec", "nova_api", "nova-manage", "db", "online_data_migrations"], "delta": "0:00:04.313646", "end": "2018-12-17 13:21:24.344519", "rc": 0, "start": "2018-12-17 13:21:20.030873", "stderr": "", "stderr_lines": [], "stdout": "Running batches of 50 until complete\\n+---------------------------------------------+--------------+-----------+\\n| Migration | Total Needed | Completed |\\n+---------------------------------------------+--------------+-----------+\\n| create_incomplete_consumers | 0 | 0 |\\n| delete_build_requests_with_no_instance_uuid | 0 | 0 |\\n| migrate_instances_add_request_spec | 0 | 0 |\\n| migrate_keypairs_to_api_db | 0 | 0 |\\n| migrate_quota_classes_to_api_db | 0 | 0 |\\n| migrate_quota_limits_to_api_db | 0 | 0 |\\n| migration_migrate_to_uuid | 0 | 0 |\\n| populate_missing_availability_zones | 0 | 0 |\\n| populate_queued_for_delete | 0 | 0 |\\n| populate_uuids | 0 | 0 |\\n| service_uuids_online_data_migration | 0 | 0 |\\n+---------------------------------------------+--------------+-----------+", "stdout_lines": ["Running batches of 50 until complete", "+---------------------------------------------+--------------+-----------+", "| Migration | Total Needed | Completed |", "+---------------------------------------------+--------------+-----------+", "| create_incomplete_consumers | 0 | 0 |", "| delete_build_requests_with_no_instance_uuid | 0 | 0 |", "| migrate_instances_add_request_spec | 0 | 0 |", "| migrate_keypairs_to_api_db | 0 | 0 |", "| migrate_quota_classes_to_api_db | 0 | 0 |", "| migrate_quota_limits_to_api_db | 0 | 0 |", "| migration_migrate_to_uuid | 0 | 0 |", "| populate_missing_availability_zones | 0 | 0 |", "| populate_queued_for_delete | 0 | 0 |", "| populate_uuids | 0 | 0 |", "| service_uuids_online_data_migration | 0 | 0 |", "+---------------------------------------------+--------------+-----------+"]}', 2018-12-17 08:21:26 | u'', ...
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045