Bug 1293440

Summary: [RFE][cinder] allow RBD in-use volumes to be (live) migrated (different Ceph pools)
Product: Red Hat OpenStack Reporter: Sean Cohen <scohen>
Component: openstack-cinderAssignee: Jon Bernard <jobernar>
Status: CLOSED ERRATA QA Contact: Tzach Shefi <tshefi>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.0 (Kilo)CC: aiyengar, bkopilov, cpaquin, dhill, egallen, eharney, flucifre, gcharot, gfidente, gkadam, jamsmith, jobernar, kchamart, lmarsh, ltoscano, lyarwood, marjones, nchandek, nwolf, pablo.iranzo, pgrist, rajini.karthik, rszmigie, srevivo, tshefi
Target Milestone: z2Keywords: FutureFeature, TestOnly, Triaged
Target Release: 16.1 (Train on RHEL 8.2)Flags: tshefi: automate_bug?
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
This update enables you to migrate or retype RBD in-use cinder volumes from one Ceph pool to another within the same Ceph cluster. For more information, see link:https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/storage_guide/index#section-volumes_basic[Basic volume usage and configuration] in the link:https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.1/html-single/storage_guide/index[Storage Guide]
Story Points: ---
Clone Of:
: 1888670 1888672 1888674 (view as bug list) Environment:
Last Closed: 2020-10-28 15:36:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 905125, 1306562, 1306569, 1623877, 1780119    
Bug Blocks: 1434362, 1543156, 1601807, 1728334, 1728337, 1888670, 1888672, 1888674    

Description Sean Cohen 2015-12-21 18:50:34 UTC
Description
=========

Migrating volumes between different storage backends or different storage clusters while the volume is in use in Cinder.

Currently, you can migrate an offline volume between different storage clusters, or technologies, like LVM and Ceph, but the Volume needs to be not in use.

User Stories
========
- As an operator, I want to migrate images between storage clusters without regard for their active or idle status.

Comment 2 Sean Cohen 2015-12-21 18:52:59 UTC
Current libvirt https://bugzilla.redhat.com/show_bug.cgi?id=905125 prevents this from working, once it's fixed we should be able to move forward.

Comment 4 Sean Cohen 2016-07-07 21:38:58 UTC
*** Bug 1303183 has been marked as a duplicate of this bug. ***

Comment 6 Sean Cohen 2017-05-17 13:38:10 UTC
*** Bug 1434362 has been marked as a duplicate of this bug. ***

Comment 7 Sean Cohen 2017-06-28 13:14:57 UTC
*** Bug 1462798 has been marked as a duplicate of this bug. ***

Comment 9 Sean Cohen 2017-11-01 13:25:35 UTC
*** Bug 1507636 has been marked as a duplicate of this bug. ***

Comment 16 Alan Bishop 2018-07-10 22:49:45 UTC
*** Bug 1587875 has been marked as a duplicate of this bug. ***

Comment 17 Alan Bishop 2018-07-10 22:50:27 UTC
*** Bug 1598513 has been marked as a duplicate of this bug. ***

Comment 18 Gregory Charot 2019-06-07 13:08:26 UTC
Pushing to 17 since the nova part has been postponed.

Comment 31 bkopilov 2020-10-04 11:38:28 UTC
Hi , 
Starting to test migration when volume is attached to instance .

hostgroup@netapp#rhos_cinder
hostgroup@tripleo_ceph#tripleo_ceph 


# Migrate multi-attached volume (attached to two instances) ------->  Fails due to:

/var/log/containers/cinder/cinder-volume.log:86:2020-10-04 10:19:35.002 46 ERROR oslo_messaging.rpc.server novaclient.exceptions.BadRequest: Swapping multi-attach volumes with more than one read/write attachment is not supported. (HTTP 400) (Request-ID: req-c5cd87a2-a6a4-4a42-b0cd-fc6342194808)


(overcloud) [stack@undercloud-0 ~]$ cinder show 3052ff7f-79e0-4b94-9845-6e810b7ecc92
+--------------------------------+----------------------------------------------------------------------------------+
| Property                       | Value                                                                            |
+--------------------------------+----------------------------------------------------------------------------------+
| attached_servers               | ['a3ce73f0-b169-4f4c-8c50-b1db7257fd10', '781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['a8bb65cc-c6de-44b0-8fd6-d339da08bb2b', 'f07ca15b-41f6-41c1-9414-eb91000ead73'] |
| availability_zone              | nova                                                                             |
| bootable                       | false                                                                            |
| consistencygroup_id            | None                                                                             |
| created_at                     | 2020-10-04T08:05:40.000000                                                       |
| description                    | None                                                                             |
| encrypted                      | False                                                                            |
| id                             | 3052ff7f-79e0-4b94-9845-6e810b7ecc92                                             |
| metadata                       |                                                                                  |
| migration_status               | None                                                                             |
| multiattach                    | True                                                                             |
| name                           | None                                                                             |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph                                              |
| os-vol-mig-status-attr:migstat | None                                                                             |
| os-vol-mig-status-attr:name_id | None                                                                             |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b                                                 |
| replication_status             | None                                                                             |
| size                           | 1                                                                                |
| snapshot_id                    | None                                                                             |
| source_volid                   | None                                                                             |
| status                         | in-use                                                                           |
| updated_at                     | 2020-10-04T08:06:45.000000                                                       |
| user_id                        | 708dc8c91fbd467196b518f9535ad130                                                 |
| volume_type                    | multiattach                                                                      |
+--------------------------------+----------------------------------------------------------------------------------+


#  Trying to migrate encrypted volume attached to instance  -------->  Fails

/var/log/containers/nova/nova-compute.log:65:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server     raise NotImplementedError(_("Swap volume is not supported for "
/var/log/containers/nova/nova-compute.log:66:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server NotImplementedError: Swap volume is not supported for encrypted volumes when native LUKS decryption is enabled.
/var/log/containers/nova/nova-compute.log:67:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server 


(overcloud) [stack@undercloud-0 ~]$ cinder show d5bf96a0-d770-4b3e-a296-eee5650e5f8e
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['d8d32f80-501b-449d-8b50-d9cd056fdb1d'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T08:01:14.000000               |
| description                    | None                                     |
| encrypted                      | True                                     |
| id                             | d5bf96a0-d770-4b3e-a296-eee5650e5f8e     |
| metadata                       |                                          |
| migration_status               | None                                     |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph      |
| os-vol-mig-status-attr:migstat | None                                     |
| os-vol-mig-status-attr:name_id | None                                     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T08:02:01.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | LUKS                                     |
+--------------------------------+------------------------------------------+



# Migrate attached volume from ceph to netapp (ISCSI) -> PASSED

(overcloud) [stack@undercloud-0 ~]$ cinder show 5b8c015d-8475-4320-a777-c494719adb7a
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['e3649d0c-025d-4a4a-b452-3a71cd83b86f'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T07:55:42.000000               |
| description                    | None                                     |
| encrypted                      | False                                    |
| id                             | 5b8c015d-8475-4320-a777-c494719adb7a     |
| metadata                       |                                          |
| migration_status               | None                                     |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph      |
| os-vol-mig-status-attr:migstat | None                                     |
| os-vol-mig-status-attr:name_id | None                                     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T07:59:03.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | tripleo                                  |
+--------------------------------+------------------------------------------+


cinder migrate 5b8c015d-8475-4320-a777-c494719adb7a  hostgroup@netapp#rhos_cinder


#  Migrate attached volume from netapp ISCSI to ceph  ---> Failed not supported on netapp

(overcloud) [stack@undercloud-0 ~]$ cinder show 7f69439a-6efd-4073-8dd0-0a12718d239c
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['c301cc4c-0840-434e-a2d1-5d8920a87dcf'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T07:56:26.000000               |
| description                    | None                                     |
| encrypted                      | False                                    |
| id                             | 7f69439a-6efd-4073-8dd0-0a12718d239c     |
| metadata                       |                                          |
| migration_status               | success                                  |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@netapp#rhos_cinder             |
| os-vol-mig-status-attr:migstat | success                                  |
| os-vol-mig-status-attr:name_id | fefbfc41-20ad-4541-8f6b-78f45aa805b2     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T07:59:38.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | tripleo                                  |
+--------------------------------+------------------------------------------+
(overcloud) [stack@undercloud-0 ~]$ 

cinder migrate 7f69439a-6efd-4073-8dd0-0a12718d239c   hostgroup@netapp#rhos_cinder


overcloud) [stack@undercloud-0 ~]$ cinder migrate 7f69439a-6efd-4073-8dd0-0a12718d239c   hostgroup@netapp#rhos_cinder 
Migration for volume 7f69439a-6efd-4073-8dd0-0a12718d239c failed: Invalid volume: Volume 7f69439a-6efd-4073-8dd0-0a12718d239c status must be available or in-use, must not be migrating, have snapshots, be replicated, be part of a group and destination host/cluster must be different than the current one (HTTP 400) (Request-ID: req-16808347-c0e5-4dab-bb70-dac05f5d88c9).



#  detach volumes after migration and attach to another instance:
Passed

Comment 32 bkopilov 2020-10-04 11:38:42 UTC
Hi , 
Starting to test migration when volume is attached to instance .

hostgroup@netapp#rhos_cinder
hostgroup@tripleo_ceph#tripleo_ceph 


# Migrate multi-attached volume (attached to two instances) ------->  Fails due to:

/var/log/containers/cinder/cinder-volume.log:86:2020-10-04 10:19:35.002 46 ERROR oslo_messaging.rpc.server novaclient.exceptions.BadRequest: Swapping multi-attach volumes with more than one read/write attachment is not supported. (HTTP 400) (Request-ID: req-c5cd87a2-a6a4-4a42-b0cd-fc6342194808)


(overcloud) [stack@undercloud-0 ~]$ cinder show 3052ff7f-79e0-4b94-9845-6e810b7ecc92
+--------------------------------+----------------------------------------------------------------------------------+
| Property                       | Value                                                                            |
+--------------------------------+----------------------------------------------------------------------------------+
| attached_servers               | ['a3ce73f0-b169-4f4c-8c50-b1db7257fd10', '781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['a8bb65cc-c6de-44b0-8fd6-d339da08bb2b', 'f07ca15b-41f6-41c1-9414-eb91000ead73'] |
| availability_zone              | nova                                                                             |
| bootable                       | false                                                                            |
| consistencygroup_id            | None                                                                             |
| created_at                     | 2020-10-04T08:05:40.000000                                                       |
| description                    | None                                                                             |
| encrypted                      | False                                                                            |
| id                             | 3052ff7f-79e0-4b94-9845-6e810b7ecc92                                             |
| metadata                       |                                                                                  |
| migration_status               | None                                                                             |
| multiattach                    | True                                                                             |
| name                           | None                                                                             |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph                                              |
| os-vol-mig-status-attr:migstat | None                                                                             |
| os-vol-mig-status-attr:name_id | None                                                                             |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b                                                 |
| replication_status             | None                                                                             |
| size                           | 1                                                                                |
| snapshot_id                    | None                                                                             |
| source_volid                   | None                                                                             |
| status                         | in-use                                                                           |
| updated_at                     | 2020-10-04T08:06:45.000000                                                       |
| user_id                        | 708dc8c91fbd467196b518f9535ad130                                                 |
| volume_type                    | multiattach                                                                      |
+--------------------------------+----------------------------------------------------------------------------------+


#  Trying to migrate encrypted volume attached to instance  -------->  Fails

/var/log/containers/nova/nova-compute.log:65:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server     raise NotImplementedError(_("Swap volume is not supported for "
/var/log/containers/nova/nova-compute.log:66:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server NotImplementedError: Swap volume is not supported for encrypted volumes when native LUKS decryption is enabled.
/var/log/containers/nova/nova-compute.log:67:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server 


(overcloud) [stack@undercloud-0 ~]$ cinder show d5bf96a0-d770-4b3e-a296-eee5650e5f8e
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['d8d32f80-501b-449d-8b50-d9cd056fdb1d'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T08:01:14.000000               |
| description                    | None                                     |
| encrypted                      | True                                     |
| id                             | d5bf96a0-d770-4b3e-a296-eee5650e5f8e     |
| metadata                       |                                          |
| migration_status               | None                                     |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph      |
| os-vol-mig-status-attr:migstat | None                                     |
| os-vol-mig-status-attr:name_id | None                                     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T08:02:01.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | LUKS                                     |
+--------------------------------+------------------------------------------+



# Migrate attached volume from ceph to netapp (ISCSI) -> PASSED

(overcloud) [stack@undercloud-0 ~]$ cinder show 5b8c015d-8475-4320-a777-c494719adb7a
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['e3649d0c-025d-4a4a-b452-3a71cd83b86f'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T07:55:42.000000               |
| description                    | None                                     |
| encrypted                      | False                                    |
| id                             | 5b8c015d-8475-4320-a777-c494719adb7a     |
| metadata                       |                                          |
| migration_status               | None                                     |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph      |
| os-vol-mig-status-attr:migstat | None                                     |
| os-vol-mig-status-attr:name_id | None                                     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T07:59:03.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | tripleo                                  |
+--------------------------------+------------------------------------------+


cinder migrate 5b8c015d-8475-4320-a777-c494719adb7a  hostgroup@netapp#rhos_cinder


#  Migrate attached volume from netapp ISCSI to ceph  ---> Failed not supported on netapp

(overcloud) [stack@undercloud-0 ~]$ cinder show 7f69439a-6efd-4073-8dd0-0a12718d239c
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['c301cc4c-0840-434e-a2d1-5d8920a87dcf'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T07:56:26.000000               |
| description                    | None                                     |
| encrypted                      | False                                    |
| id                             | 7f69439a-6efd-4073-8dd0-0a12718d239c     |
| metadata                       |                                          |
| migration_status               | success                                  |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@netapp#rhos_cinder             |
| os-vol-mig-status-attr:migstat | success                                  |
| os-vol-mig-status-attr:name_id | fefbfc41-20ad-4541-8f6b-78f45aa805b2     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T07:59:38.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | tripleo                                  |
+--------------------------------+------------------------------------------+
(overcloud) [stack@undercloud-0 ~]$ 

cinder migrate 7f69439a-6efd-4073-8dd0-0a12718d239c   hostgroup@netapp#rhos_cinder


overcloud) [stack@undercloud-0 ~]$ cinder migrate 7f69439a-6efd-4073-8dd0-0a12718d239c   hostgroup@netapp#rhos_cinder 
Migration for volume 7f69439a-6efd-4073-8dd0-0a12718d239c failed: Invalid volume: Volume 7f69439a-6efd-4073-8dd0-0a12718d239c status must be available or in-use, must not be migrating, have snapshots, be replicated, be part of a group and destination host/cluster must be different than the current one (HTTP 400) (Request-ID: req-16808347-c0e5-4dab-bb70-dac05f5d88c9).



#  detach volumes after migration and attach to another instance:
Passed

Comment 33 bkopilov 2020-10-04 11:38:59 UTC
Hi , 
Starting to test migration when volume is attached to instance .

hostgroup@netapp#rhos_cinder
hostgroup@tripleo_ceph#tripleo_ceph 


# Migrate multi-attached volume (attached to two instances) ------->  Fails due to:

/var/log/containers/cinder/cinder-volume.log:86:2020-10-04 10:19:35.002 46 ERROR oslo_messaging.rpc.server novaclient.exceptions.BadRequest: Swapping multi-attach volumes with more than one read/write attachment is not supported. (HTTP 400) (Request-ID: req-c5cd87a2-a6a4-4a42-b0cd-fc6342194808)


(overcloud) [stack@undercloud-0 ~]$ cinder show 3052ff7f-79e0-4b94-9845-6e810b7ecc92
+--------------------------------+----------------------------------------------------------------------------------+
| Property                       | Value                                                                            |
+--------------------------------+----------------------------------------------------------------------------------+
| attached_servers               | ['a3ce73f0-b169-4f4c-8c50-b1db7257fd10', '781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['a8bb65cc-c6de-44b0-8fd6-d339da08bb2b', 'f07ca15b-41f6-41c1-9414-eb91000ead73'] |
| availability_zone              | nova                                                                             |
| bootable                       | false                                                                            |
| consistencygroup_id            | None                                                                             |
| created_at                     | 2020-10-04T08:05:40.000000                                                       |
| description                    | None                                                                             |
| encrypted                      | False                                                                            |
| id                             | 3052ff7f-79e0-4b94-9845-6e810b7ecc92                                             |
| metadata                       |                                                                                  |
| migration_status               | None                                                                             |
| multiattach                    | True                                                                             |
| name                           | None                                                                             |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph                                              |
| os-vol-mig-status-attr:migstat | None                                                                             |
| os-vol-mig-status-attr:name_id | None                                                                             |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b                                                 |
| replication_status             | None                                                                             |
| size                           | 1                                                                                |
| snapshot_id                    | None                                                                             |
| source_volid                   | None                                                                             |
| status                         | in-use                                                                           |
| updated_at                     | 2020-10-04T08:06:45.000000                                                       |
| user_id                        | 708dc8c91fbd467196b518f9535ad130                                                 |
| volume_type                    | multiattach                                                                      |
+--------------------------------+----------------------------------------------------------------------------------+


#  Trying to migrate encrypted volume attached to instance  -------->  Fails

/var/log/containers/nova/nova-compute.log:65:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server     raise NotImplementedError(_("Swap volume is not supported for "
/var/log/containers/nova/nova-compute.log:66:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server NotImplementedError: Swap volume is not supported for encrypted volumes when native LUKS decryption is enabled.
/var/log/containers/nova/nova-compute.log:67:2020-10-04 10:22:38.074 7 ERROR oslo_messaging.rpc.server 


(overcloud) [stack@undercloud-0 ~]$ cinder show d5bf96a0-d770-4b3e-a296-eee5650e5f8e
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['d8d32f80-501b-449d-8b50-d9cd056fdb1d'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T08:01:14.000000               |
| description                    | None                                     |
| encrypted                      | True                                     |
| id                             | d5bf96a0-d770-4b3e-a296-eee5650e5f8e     |
| metadata                       |                                          |
| migration_status               | None                                     |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph      |
| os-vol-mig-status-attr:migstat | None                                     |
| os-vol-mig-status-attr:name_id | None                                     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T08:02:01.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | LUKS                                     |
+--------------------------------+------------------------------------------+



# Migrate attached volume from ceph to netapp (ISCSI) -> PASSED

(overcloud) [stack@undercloud-0 ~]$ cinder show 5b8c015d-8475-4320-a777-c494719adb7a
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['e3649d0c-025d-4a4a-b452-3a71cd83b86f'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T07:55:42.000000               |
| description                    | None                                     |
| encrypted                      | False                                    |
| id                             | 5b8c015d-8475-4320-a777-c494719adb7a     |
| metadata                       |                                          |
| migration_status               | None                                     |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@tripleo_ceph#tripleo_ceph      |
| os-vol-mig-status-attr:migstat | None                                     |
| os-vol-mig-status-attr:name_id | None                                     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T07:59:03.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | tripleo                                  |
+--------------------------------+------------------------------------------+


cinder migrate 5b8c015d-8475-4320-a777-c494719adb7a  hostgroup@netapp#rhos_cinder


#  Migrate attached volume from netapp ISCSI to ceph  ---> Failed not supported on netapp

(overcloud) [stack@undercloud-0 ~]$ cinder show 7f69439a-6efd-4073-8dd0-0a12718d239c
+--------------------------------+------------------------------------------+
| Property                       | Value                                    |
+--------------------------------+------------------------------------------+
| attached_servers               | ['781195f9-6683-49c6-a7b5-7524814b21a6'] |
| attachment_ids                 | ['c301cc4c-0840-434e-a2d1-5d8920a87dcf'] |
| availability_zone              | nova                                     |
| bootable                       | false                                    |
| consistencygroup_id            | None                                     |
| created_at                     | 2020-10-04T07:56:26.000000               |
| description                    | None                                     |
| encrypted                      | False                                    |
| id                             | 7f69439a-6efd-4073-8dd0-0a12718d239c     |
| metadata                       |                                          |
| migration_status               | success                                  |
| multiattach                    | False                                    |
| name                           | None                                     |
| os-vol-host-attr:host          | hostgroup@netapp#rhos_cinder             |
| os-vol-mig-status-attr:migstat | success                                  |
| os-vol-mig-status-attr:name_id | fefbfc41-20ad-4541-8f6b-78f45aa805b2     |
| os-vol-tenant-attr:tenant_id   | d213f5e7c68e4d609bb5212c8c40963b         |
| replication_status             | None                                     |
| size                           | 1                                        |
| snapshot_id                    | None                                     |
| source_volid                   | None                                     |
| status                         | in-use                                   |
| updated_at                     | 2020-10-04T07:59:38.000000               |
| user_id                        | 708dc8c91fbd467196b518f9535ad130         |
| volume_type                    | tripleo                                  |
+--------------------------------+------------------------------------------+
(overcloud) [stack@undercloud-0 ~]$ 

cinder migrate 7f69439a-6efd-4073-8dd0-0a12718d239c   hostgroup@netapp#rhos_cinder


overcloud) [stack@undercloud-0 ~]$ cinder migrate 7f69439a-6efd-4073-8dd0-0a12718d239c   hostgroup@netapp#rhos_cinder 
Migration for volume 7f69439a-6efd-4073-8dd0-0a12718d239c failed: Invalid volume: Volume 7f69439a-6efd-4073-8dd0-0a12718d239c status must be available or in-use, must not be migrating, have snapshots, be replicated, be part of a group and destination host/cluster must be different than the current one (HTTP 400) (Request-ID: req-16808347-c0e5-4dab-bb70-dac05f5d88c9).



#  detach volumes after migration and attach to another instance:
Passed

Comment 35 Luigi Toscano 2020-10-15 13:27:09 UTC
Now the bug has more specialized clones that take care of various use case. This bug itself is about the feature itself and its implementation between different Ceph pools.

Comment 37 Luigi Toscano 2020-10-15 18:30:02 UTC
Deployed cinder with two backends, each one backed by a different Ceph pool.

Both backends have been configured to be used by tempest, and the following test successfully passed:
tempest.scenario.test_volume_migrate_attached.TestVolumeMigrateRetypeAttached.test_volume_retype_attached

from openstack-tempest-23.0.0-0.20200609093435.d432237


On the controllers: openstack-cinder-15.3.1-1.20200914173114.00ac80b

Comment 43 errata-xmlrpc 2020-10-28 15:36:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Red Hat OpenStack Platform 16.1 bug fix and enhancement advisory), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2020:4284