Bug 1884556

Summary:	[OSP16.0 to OSP16.1] Minor update procedure not triggering online DB migrations in "openstack overcloud external-update run --tags online-upgrade"
Product:	Red Hat OpenStack	Reporter:	Jose Luis Franco <jfrancoa>
Component:	openstack-tripleo-heat-templates	Assignee:	Jose Luis Franco <jfrancoa>
Status:	CLOSED ERRATA	QA Contact:	Sofer Athlan-Guyot <sathlang>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	16.1 (Train)	CC:	igallagh, jpretori, jstransk, mbollo, mburns, mgarciac, sathlang, vgrosu
Target Milestone:	z4	Keywords:	Triaged
Target Release:	16.1 (Train on RHEL 8.2)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-heat-templates-11.3.2-1.20201114031850.el8ost	Doc Type:	Bug Fix
Doc Text:	Before this update, you were required to use the `openstack overcloud external-upgrade run --tags online_upgrade` command to perform online database updates when upgrading from RHOSP 15 to RHOSP 16.1. With this update, you can now use the `openstack overcloud external-update run --tags online_upgrade` command.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-03-17 15:32:20 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jose Luis Franco 2020-10-02 10:02:50 UTC

Description of problem:

The documentation steps to update the environment form OSP16.0 to OSP16.1 state that the "openstack overcloud external-update run --tags online_upgrade" should be called to perform the online DB migrations:

https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/keeping_red_hat_openstack_platform_updated/index#performing-online-database-updates

However, there are no external_update_tasks to handle the online DB migrations, but external_UPGRADE_tasks:

https://github.com/openstack/tripleo-heat-templates/search?q=online

Therefore, the right command in order to get these tasks executed would be:

openstack overcloud external-upgrade run --tags online_upgrade


It would be interesting to verify if the same is documented for old releases and modify it there too.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:

Change section 4.6 from documentation (https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/keeping_red_hat_openstack_platform_updated/index#performing-online-database-updates) to use "openstack overcloud external-upgrade run --tags online_upgrade"
instead.

Expected results:


Additional info:

Comment 1 Jose Luis Franco 2020-10-13 16:53:12 UTC

There's been an interesting discussion in https://review.opendev.org/#/c/755751/1 ending up on dealing with this issue in the code, instead of modifying the downstream documentation.

Therefore, I am moving this into DFG:Upgrades and changing the component.

Comment 2 Jiri Stransky 2020-10-19 13:20:29 UTC

*** Bug 1885900 has been marked as a duplicate of this bug. ***

Comment 9 Sofer Athlan-Guyot 2021-02-02 16:17:03 UTC

We can see them running now.

Note that the ironic tasks are not run there are the service isn't installed on the overcloud.

excerpt from http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/DFG-upgrades-updates-16.1-from-ga-composable-ipv6/72/undercloud-0/home/stack/overcloud_update_data_migration.log.gz

2021-01-22 18:27:33 | PLAY [External update step 0] **************************************************
2021-01-22 18:27:33 | 
2021-01-22 18:27:33 | TASK [Online data migration for Cinder] ****************************************
2021-01-22 18:27:33 | Friday 22 January 2021  18:27:29 +0000 (0:00:00.621)       0:00:08.541 ******** 
2021-01-22 18:27:33 | skipping: [undercloud] => {"changed": false, "skip_reason": "Conditional result was False"}
2021-01-22 18:27:33 | 
2021-01-22 18:27:33 | TASK [Online data migration for Nova] ******************************************
2021-01-22 18:27:33 | Friday 22 January 2021  18:27:29 +0000 (0:00:00.096)       0:00:08.638 ******** 
2021-01-22 18:27:33 | skipping: [undercloud] => {"changed": false, "skip_reason": "Conditional result was False"}
2021-01-22 18:27:33 | 
2021-01-22 18:27:33 | PLAY [External update step 1] **************************************************
2021-01-22 18:27:33 | 
2021-01-22 18:27:33 | TASK [Online data migration for Cinder] ****************************************
2021-01-22 18:27:33 | Friday 22 January 2021  18:27:29 +0000 (0:00:00.100)       0:00:08.739 ******** 
2021-01-22 18:27:33 | changed: [undercloud -> 192.168.24.27] => {"changed": true, "cmd": ["podman", "exec", "cinder_api", "cinder-manage", "db", "online_data_migrations"], "delta": "0:00:03.475221", "end": "2021-01-22 18:27:33.458936", "rc": 0, "start": "2021-01-22 18:27:29.983715", "stderr": "Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\".", "stderr_lines": ["Deprecated: Option \"logdir\" from group \"DEFAULT\" is deprecated. Use option \"log-dir\" from group \"DEFAULT\"."], "stdout": "Running batches of 50 until complete.\n+-----------------------------------------+--------------+-----------+\n|                Migration                | Total Needed | Completed |\n+-----------------------------------------+--------------+-----------+\n| untyped_snapshots_online_data_migration |      0       |     0     |\n|  untyped_volumes_online_data_migration  |      0       |     0     |\n+-----------------------------------------+--------------+-----------+", "stdout_lines": ["Running batches of 50 until complete.", "+-----------------------------------------+--------------+-----------+", "|                Migration                | Total Needed | Completed |", "+-----------------------------------------+--------------+-----------+", "| untyped_snapshots_online_data_migration |      0       |     0     |", "|  untyped_volumes_online_data_migration  |      0       |     0     |", "+-----------------------------------------+--------------+-----------+"]}
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | TASK [Online data migration for Nova] ******************************************
2021-01-22 18:27:40 | Friday 22 January 2021  18:27:33 +0000 (0:00:04.273)       0:00:13.012 ******** 
2021-01-22 18:27:40 | changed: [undercloud -> 192.168.24.27] => {"changed": true, "cmd": ["podman", "exec", "nova_conductor", "nova-manage", "db", "online_data_migrations"], "delta": "0:00:04.889869", "end": "2021-01-22 18:27:38.945328", "rc": 0, "start": "2021-01-22 18:27:34.055459", "stderr": "", "stderr_lines": [], "stdout": "Running batches of 50 until complete\n1 rows matched query fill_virtual_interface_list, 0 migrated\n+-------------------------------------+--------------+-----------+\n|              Migration              | Total Needed | Completed |\n+-------------------------------------+--------------+-----------+\n|     fill_virtual_interface_list     |      1       |     0     |\n|         migrate_empty_ratio         |      0       |     0     |\n|   migrate_quota_classes_to_api_db   |      0       |     0     |\n|    migrate_quota_limits_to_api_db   |      0       |     0     |\n|      migration_migrate_to_uuid      |      0       |     0     |\n| populate_missing_availability_zones |      0       |     0     |\n|      populate_queued_for_delete     |      0       |     0     |\n|           populate_user_id          |      0       |     0     |\n|            populate_uuids           |      0       |     0     |\n| service_uuids_online_data_migration |      0       |     0     |\n+-------------------------------------+--------------+-----------+", "stdout_lines": ["Running batches of 50 until complete", "1 rows matched query fill_virtual_interface_list, 0 migrated", "+-------------------------------------+--------------+-----------+", "|              Migration              | Total Needed | Completed |", "+-------------------------------------+--------------+-----------+", "|     fill_virtual_interface_list     |      1       |     0     |", "|         migrate_empty_ratio         |      0       |     0     |", "|   migrate_quota_classes_to_api_db   |      0       |     0     |", "|    migrate_quota_limits_to_api_db   |      0       |     0     |", "|      migration_migrate_to_uuid      |      0       |     0     |", "| populate_missing_availability_zones |      0       |     0     |", "|      populate_queued_for_delete     |      0       |     0     |", "|           populate_user_id          |      0       |     0     |", "|            populate_uuids           |      0       |     0     |", "| service_uuids_online_data_migration |      0       |     0     |", "+-------------------------------------+--------------+-----------+"]}
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | PLAY [External deploy step 1] **************************************************
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | PLAY [External deploy step 2] **************************************************
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | PLAY [External deploy step 3] **************************************************
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | PLAY [External deploy step 4] **************************************************
2021-01-22 18:27:40 | 
2021-01-22 18:27:40 | PLAY [External deploy step 5] **************************************************

Comment 15 errata-xmlrpc 2021-03-17 15:32:20 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1.4 director bug fix advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:0817