+++ This bug was initially created as a clone of Bug #1729755 +++ We backported MA RBD to OSP13, hit same issue on OSP13. Description of problem: We currently rightfully don't allow/support creation of encrypted multi-attach volume: $ cinder create 1 --volume-type enc_ma_lvm --name enc_ma_vol ERROR: Invalid volume: Multiattach cannot be used with encrypted volumes. (HTTP 400) (Request-ID: req-8859dbc6-646f-4cb0-8a0d-419a019ffae1) However if you create an encrypted (none MA) volume and retype it to a none supported multi-attached encrypted volume it would ideally fail/warn. Yet operation is executed despite the none supported resulting volume, this IMHO should be blocked. Yes I know retype doesn't validate/check backends before it accepts what looks like a valid request, which explains why no error is reported and operation is preformed. Call it wishful QE thinking I'd like to see such operations validated and blocked if end result isn't supported. Version-Release number of selected component (if applicable): puppet-cinder-14.4.1-0.20190420083336.1cf0604.el8ost.noarch openstack-cinder-14.0.1-0.20190607000407.23d1a72.el8ost.noarch python3-cinderclient-4.2.0-0.20190520060354.953243d.el8ost.noarch python3-cinder-14.0.1-0.20190607000407.23d1a72.el8ost.noarch How reproducible: Every time Steps to Reproduce: 1. Create both backends | 04c5d0b5-3ed4-4ba1-b74d-7bcd8b2b4540 | enc_lvm | {'volume_backend_name': 'tripleo_iscsi'} cinder encryption-type-create --cipher aes-xts-plain64 --key_size 256 --control_location front-end enc_lvm nova.volume.encryptors.luks.LuksEncryptor | 50c3fb05-1c34-45e6-b40c-44296b3b90ad | enc_ma_lvm | {'volume_backend_name': 'tripleo_iscsi', 'multiattach': '<is> True'} | cinder encryption-type-create --cipher aes-xts-plain64 --key_size 256 --control_location front-end enc_ma_lvm nova.volume.encryptors.luks.LuksEncryptor 2. Create an enc lvm volume: (overcloud) [stack@undercloud-0 ~]$ cinder create 1 --volume-type enc_lvm --name Enc-vol +--------------------------------+---------------------------------------+ | Property | Value | +--------------------------------+---------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2019-07-14T09:16:16.000000 | | description | None | | encrypted | True | | id | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | | metadata | {} | | migration_status | None | | multiattach | False | | name | Enc-vol | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 5850830f52774e0dab7b7b8e508b6a56 | | replication_status | None | | size | 1 | | snapshot_id | None | | source_volid | None | | status | available | | updated_at | 2019-07-14T09:16:17.000000 | | user_id | 8fe58395864a43158a847d1a9ffd4e9d | | volume_type | enc_lvm | +--------------------------------+---------------------------------------+ Volume before migration (overcloud) [stack@undercloud-0 ~]$ cinder list | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | available | Enc-vol | 1 | enc_lvm | false | | 3. Retype the volume to none supported enc_ma_lvm type: (overcloud) [stack@undercloud-0 ~]$ cinder retype 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 enc_ma_lvm --migration-policy on-demand !error returned :( (overcloud) [stack@undercloud-0 ~]$ cinder show 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 +--------------------------------+---------------------------------------+ | Property | Value | +--------------------------------+---------------------------------------+ | attached_servers | [] | | attachment_ids | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2019-07-14T09:16:16.000000 | | description | None | | encrypted | True | | id | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | | metadata | | | migration_status | success | | multiattach | False | | name | Enc-vol | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | success | | os-vol-mig-status-attr:name_id | 558b1fa0-21fb-4d80-afff-f411d5717bcb | | os-vol-tenant-attr:tenant_id | 5850830f52774e0dab7b7b8e508b6a56 | | replication_status | None | | size | 1 | | snapshot_id | None | | source_volid | None | | status | available | | updated_at | 2019-07-14T09:18:32.000000 | | user_id | 8fe58395864a43158a847d1a9ffd4e9d | | volume_type | enc_ma_lvm | +--------------------------------+---------------------------------------+ Notice volume is available (bad), don't want to know what state data was left in. Volume type changed from enc_lvm to -> enc_ma_lvm Actual results: Volume is migrated despite not allowing such a resulting type Expected results: Please block such operations, before a customer opens a bug about lost/bad data. Additional info: No logs were add as there is no error to be found in logs. --- Additional comment from Tzach Shefi on 2019-07-14 09:41:19 UTC --- FYI might be a case for another bug, when we try to attach the resulting (none supported) Enc MA volume (overcloud) [stack@undercloud-0 ~]$ nova volume-attach inst1 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 auto +----------+--------------------------------------+ | Property | Value | +----------+--------------------------------------+ | device | /dev/vdb | | id | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | | serverId | a4946e59-08e2-41a9-b00a-d052a6a33d5e | | tag | - | | volumeId | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | +----------+--------------------------------------+ We get stuck with | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | attaching | Enc-vol | 1 | enc_ma_lvm | false | | Attaching state till we timeout is reached, then return to available state. | 723431a2-38c3-4c9d-b87a-8dc4dd9f9da0 | available | Enc-vol | 1 | enc_ma_lvm | false | | Would we not allow retype this wouldn't be an issue, if search for attach problems I'm sure I'll find them. Yet the common user won't understand why attach is failing.
@Jon, assigning to you as you have the OSP-15 version of this bz. Feel free to ask for help getting the fix backported to the earlier release.
Tested on: openstack-cinder-12.0.10-2.el7ost.noarch Created two backends: (overcloud) [stack@puma51 ~]$ cinder extra-specs-list +--------------------------------------+---------+----------------------------------------------------------------------+ | ID | Name | extra_specs | +--------------------------------------+---------+----------------------------------------------------------------------+ | 5d5caec0-738b-421e-aa6b-5f9df061d3cb | enc_ma | {'volume_backend_name': 'tripleo_iscsi', 'multiattach': '<is> True'} | | 6b187af5-fc0f-4eb6-8f9f-fea97ede359b | enc | {'volume_backend_name': 'tripleo_iscsi'} | | ab130a7e-bf02-40b0-b551-c84aee7f40dc | tripleo | {} | Create an encrypted volume from an image (overcloud) [stack@puma51 ~]$ cinder create 5 --volume-type enc --name EncVol1 (overcloud) [stack@puma51 ~]$ cinder show EncVol1 +--------------------------------+---------------------------------------+ | Property | Value | +--------------------------------+---------------------------------------+ | attached_servers | [] | | attachment_ids | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2020-03-09T09:45:39.000000 | | description | None | | encrypted | True | | id | 42b2ba66-6938-4c2d-a464-a35d534160a3 | | metadata | | | migration_status | None | | multiattach | False | | name | EncVol1 | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | c1479695d5cf43b1a854f600226dad39 | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | available | | updated_at | 2020-03-09T09:45:42.000000 | | user_id | cb29148baa654f739c78ed4f92cb6e2b | | volume_type | enc | +--------------------------------+---------------------------------------+ Now lets try to retype this to enc_ma it should fail (overcloud) [stack@puma51 ~]$ cinder retype 42b2ba66-6938-4c2d-a464-a35d534160a3 enc_ma --migration-policy on-demand Retype still happens when it should be blocked (overcloud) [stack@puma51 ~]$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | 42b2ba66-6938-4c2d-a464-a35d534160a3 | retyping | EncVol1 | 5 | enc | false | | | d97dce16-d2e1-4a00-b1ee-24c2c8dd2073 | available | EncVol1 | 5 | enc_ma | false | | (overcloud) [stack@puma51 ~]$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | 42b2ba66-6938-4c2d-a464-a35d534160a3 | available | EncVol1 | 5 | enc_ma | false | | | d97dce16-d2e1-4a00-b1ee-24c2c8dd2073 | deleting | EncVol1 | 5 | enc_ma | false | | (overcloud) [stack@puma51 ~]$ cinder list +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+-----------+------+-------------+----------+--------------------------------------+ | 42b2ba66-6938-4c2d-a464-a35d534160a3 | available | EncVol1 | 5 | enc_ma | false | | Failed to verify, vol shouldn't migrate from enc to none supported enc_ma (overcloud) [stack@puma51 ~]$ cinder show 42b2ba66-6938-4c2d-a464-a35d534160a3 +--------------------------------+---------------------------------------+ | Property | Value | +--------------------------------+---------------------------------------+ | attached_servers | [] | | attachment_ids | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2020-03-09T09:45:39.000000 | | description | None | | encrypted | True | So volume is encrypted, but look down multipath is false. | id | 42b2ba66-6938-4c2d-a464-a35d534160a3 | | metadata | | | migration_status | success | -> migrated "successfully" when it should have been blocked. | multiattach | False | | name | EncVol1 | | os-vol-host-attr:host | hostgroup@tripleo_iscsi#tripleo_iscsi | | os-vol-mig-status-attr:migstat | success | | os-vol-mig-status-attr:name_id | d97dce16-d2e1-4a00-b1ee-24c2c8dd2073 | | os-vol-tenant-attr:tenant_id | c1479695d5cf43b1a854f600226dad39 | | replication_status | None | | size | 5 | | snapshot_id | None | | source_volid | None | | status | available | -> it might very well be but not sure what's inside if still valid data. | updated_at | 2020-03-09T10:09:57.000000 | | user_id | cb29148baa654f739c78ed4f92cb6e2b | | volume_type | enc_ma | -> not good changed from enc to enc_ma which is supported. +--------------------------------+---------------------------------------+ Just for reference here I wanted to create/show a enc_ma volume just to check that ma = true. Things got interesting, I was blocked from creating such a volume which is great: (overcloud) [stack@puma51 ~]$ cinder create 1 --volume-type enc_ma --name enc_maVol1 ERROR: Invalid volume: Multiattach cannot be used with encrypted volumes. (HTTP 400) (Request-ID: req-ee8a1c1c-4a2d-4890-baaf-bf5f4c0cb995) However again I wasn't blocked from migrating a supported enc volume to this none supported enc_ma.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0764