Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1588360

Summary: Barbican - Unable to delete encryption key for volume
Product: Red Hat OpenStack Reporter: bkopilov <bkopilov>
Component: openstack-cinderAssignee: Cinder Bugs List <cinder-bugs>
Status: CLOSED NOTABUG QA Contact: Avi Avraham <aavraham>
Severity: unspecified Docs Contact: Kim Nylander <knylande>
Priority: urgent    
Version: 13.0 (Queens)CC: aavraham, abishop, cschwede, srevivo, tshefi
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-20 13:34:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
cinder.api.logs none

Description bkopilov 2018-06-07 07:05:11 UTC 
Description of problem:
rhos13 , barbican + cinder LVM backend.
virsh setup,
1 controller , 1 compute 

When we create an encrypted volume , the volume created successfully.
Once we try to delete this volume with admin tenant , barbican complains about 
PolicyNotAuthorized: secret:delete is disallowed by policy


Version-Release number of selected component (if applicable):


How reproducible:
On evrey try

Steps to Reproduce:
1.
2.
3.

Actual results:

(overcloud) [stack@undercloud-0 ~]$ openstack project list
+----------------------------------+----------------------------------------------+
| ID                               | Name                                         |
+----------------------------------+----------------------------------------------+
| 12b130a6476a43c7933086b17a181645 | service                                      |
| 88170cc9a2664818824c3990472fc878 | alt_demo                                     |
| 992616fda462495e933953196728fa64 | tempest-TestEncryptedCinderVolumes-86019268  |
| b58a24b5f90246fa9061c99802b0b0d3 | admin                                        |
| bad914955daa4db19bae03937bf68f5a | tempest-TestEncryptedCinderVolumes-549998142 |
| d4c81807a629460185db248189ebde37 | demo                                         |
+----------------------------------+----------------------------------------------+



(overcloud) [stack@undercloud-0 ~]$ cinder type-show 614c5b5b-5396-4f58-a809-cf63fb31f0a5
+---------------------------------+----------------------------------------------------------+
| Property                        | Value                                                    |
+---------------------------------+----------------------------------------------------------+
| description                     | None                                                     |
| extra_specs                     |                                                          |
| id                              | 614c5b5b-5396-4f58-a809-cf63fb31f0a5                     |
| is_public                       | True                                                     |
| name                            | tempest-volume-typeTestEncryptedCinderVolumes-1472171707 |
| os-volume-type-access:is_public | True                                                     |
| qos_specs_id                    | None                                                     |
+---------------------------------+----------------------------------------------------------+


Try 'cinder help ' for more information.
(overcloud) [stack@undercloud-0 ~]$ cinder encryption-type-show 614c5b5b-5396-4f58-a809-cf63fb31f0a5
+--------------------------------------+-------------------------------------------+-----------------+----------+------------------+
| Volume Type ID                       | Provider                                  | Cipher          | Key Size | Control Location |
+--------------------------------------+-------------------------------------------+-----------------+----------+------------------+
| 614c5b5b-5396-4f58-a809-cf63fb31f0a5 | nova.volume.encryptors.luks.LuksEncryptor | aes-xts-plain64 | 256      | front-end        |
+--------------------------------------+-------------------------------------------+-----------------+----------+------------------+
(overcloud) [stack@undercloud-0 ~]$ 



(overcloud) [stack@undercloud-0 ~]$ cinder show 8b14c928-d291-4dea-9a18-9a9871e8679b
+--------------------------------+----------------------------------------------------------+
| Property                       | Value                                                    |
+--------------------------------+----------------------------------------------------------+
| attached_servers               | []                                                       |
| attachment_ids                 | []                                                       |
| availability_zone              | nova                                                     |
| bootable                       | false                                                    |
| consistencygroup_id            | None                                                     |
| created_at                     | 2018-06-07T06:49:09.000000                               |
| description                    | None                                                     |
| encrypted                      | True                                                     |
| id                             | 8b14c928-d291-4dea-9a18-9a9871e8679b                     |
| metadata                       |                                                          |
| migration_status               | None                                                     |
| multiattach                    | False                                                    |
| name                           | tempest-TestEncryptedCinderVolumes-volume-1807924162     |
| os-vol-host-attr:host          | hostgroup@tripleo_iscsi#tripleo_iscsi                    |
| os-vol-mig-status-attr:migstat | None                                                     |
| os-vol-mig-status-attr:name_id | None                                                     |
| os-vol-tenant-attr:tenant_id   | bad914955daa4db19bae03937bf68f5a                         |
| replication_status             | None                                                     |
| size                           | 1                                                        |
| snapshot_id                    | None                                                     |
| source_volid                   | None                                                     |
| status                         | error_deleting                                           |
| updated_at                     | 2018-06-07T06:50:00.000000                               |
| user_id                        | 22ab4006677d4f9fa71fcc5610163640                         |
| volume_type                    | tempest-volume-typeTestEncryptedCinderVolumes-1472171707 |
+--------------------------------+----------------------------------------------------------+



Expected results:


Additional info:
Comment 1 bkopilov 2018-06-07 07:14:21 UTC 
Created attachment 1448623 [details]
cinder.api.logs
Comment 2 Tzach Shefi 2018-06-07 10:17:18 UTC 
Adding version info from system

openstack-cinder-12.0.1-0.20180418194613.c476898.el7ost.noarch
openstack-barbican-common-6.0.1-0.20180421143301.b10e100.el7ost.noarch
openstack-barbican-api-6.0.1-0.20180421143301.b10e100.el7ost.noarch
openstack-tripleo-common-8.6.1-18.el7ost.noarch
openstack-tripleo-ui-8.3.1-2.el7ost.noarch
python-tripleoclient-9.2.1-12.el7ost.noarch
puppet-tripleo-8.3.2-6.el7ost.noarch
openstack-tripleo-common-containers-8.6.1-18.el7ost.noarch
openstack-tripleo-heat-templates-8.0.2-29.el7ost.noarch
openstack-tripleo-puppet-elements-8.0.0-2.el7ost.noarch
ansible-tripleo-ipsec-8.1.1-0.20180308133440.8f5369a.el7ost.noarch
openstack-tripleo-validations-8.4.1-5.el7ost.noarch
openstack-tripleo-image-elements-8.0.1-1.el7ost.noarch
Comment 3 Alan Bishop 2018-06-14 13:09:08 UTC 
I did some experimenting, confirmed my understanding of what's happening, and
determined the steps necessary for the admin user to delete the volumes.

Tempest is creating new projects (tenants) for these tests, and the issue is
the admin user is not automatically granted the admin role in the tempest
projects. When cinder tries to delete the barbican secret, barbican checks
whether the secret is owned by that user, or if the user is has the admin role
*in that project*.

It's this last part that fails: the admin (user) is not an admin (the role)
in the tempest project. You see the role assignments using this command:

  openstack role assignment list --names

Fortunately, the admin user has the ability to grant themself the admin role
in any project.

  openstack role add --project <project> --user admin admin

Now they should be able to delete the volume. But first, the volume state
will need to be reset if its status is error_deleting.

So, using the data in the BZ description, this should work:

  openstack role add \
    --project tempest-TestEncryptedCinderVolumes-549998142 --user admin admin
  openstack volume set --state=available \
    tempest-TestEncryptedCinderVolumes-volume-1807924162
  openstack volume delete tempest-TestEncryptedCinderVolumes-volume-1807924162
Comment 4 Alan Bishop 2018-06-14 13:13:55 UTC 
Whoops, sorry, one more thing. For the delete to work, the command must specify the tempest project.

  OS_PROJECT_NAME=tempest-TestEncryptedCinderVolumes-549998142 \
  OS_TENANT_NAME=tempest-TestEncryptedCinderVolumes-549998142 \
    openstack volume delete tempest-TestEncryptedCinderVolumes-volume-1807924162
Comment 5 Christian Schwede (cschwede) 2018-06-20 13:34:25 UTC 
Benny, we just discussed this in our bugscrub meeting and agreed to close this. Please feel free to re-open if this is still an issue. Thanks!