Bug 1804848 - [OSP 16.0.1] Volume encryption keys deleted when snapshotting instances created from images with cinder_encryption_key_id set
Summary: [OSP 16.0.1] Volume encryption keys deleted when snapshotting instances creat...
Keywords:
Status: CLOSED DUPLICATE of bug 1801255
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 16.0 (Train)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: z1
: ---
Assignee: Lee Yarwood
QA Contact: OSP DFG:Compute
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-02-19 18:19 UTC by Brian Rosmaita
Modified: 2023-03-21 19:27 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: Known Issue
Doc Text:
There is a known issue when all of the following conditions exist: (0) You are using the OpenStack Train release (or code from master (Ussuri development)) (1) cinder_encryption_key_id and cinder_encryption_key_deletion_policy are not included in the non_inheritable_image_properties setting in nova.conf. These properties are not included by default. (2) A user has created a volume of an encrypted volume-type in the Block Storage service (cinder). For example, Volume-1. (3) Using the Block Storage service, the user has uploaded the encrypted volume as an image to the Image service (glance). For example, Image-1. (4) Using the Compute service (nova), the user has attempted to boot a server from the image directly. Note: this is an unsupported action, the supported workflow is to use the image to boot-from-volume. (5) Although an unsupported action, if a user does (4), it currently results in a server in status ACTIVE but which is unusable because the operating system cannot be found. (6) Using the Compute service, the user requests the createImage action on the unusable server, resulting in the creation of Image-2. (7) Using the Image service, the user deletes Image-2 which has inherited the cinder_encryption_key_* properties from Image-1 and the encryption key is deleted. As a result, Image-1 is rendered non-decryptable so that it can no longer be used in the normal boot-from-volume workflow. The workaround for this issue is to add the cinder_encryption_key_id,cinder_encryption_key_deletion_policy properties to the non_inheritable_image_properties option in the [DEFAULT] section of nova.conf. Image-2 can be deleted and the encryption key used by Image-1 remains available.
Clone Of:
Environment:
Last Closed: 2020-02-21 10:17:46 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1852106 0 None None None 2020-02-19 18:21:39 UTC

Description Brian Rosmaita 2020-02-19 18:19:30 UTC
This bug was initially created as a copy of Bug #1801255

I am copying this bug because:

The fix is scheduled for 16.0.2, but we want to get a "known issues" about it into the release notes for 16.0.1.



We have to take this seriously because of possible data loss, but the end user would have to boot an instance that is 'active' but unusable, and then do the image-create action on that instance.

Brief backstory: when a Cinder encrypted volume is uploaded as an image to Glance, a secret in barbican specific to that image is created.  Beginning with Train, the image metadata "cinder_encryption_key_deletion_policy": "on_image_deletion" is also put on such an image.  On image deletion, if Glance finds such metadata on an image, it will try to delete the barbican secret for the image.

The problem: direct boot of an instance from an image I-1 created from an encrypted volume is *not supported* by Nova.  (The workflow is that you use the image to boot an instance from a volume.)  However, if you do try a direct boot of I-1, Nova creates an instance S-1 in 'active' status (though it is unusable).  If a user does the Nova image-create action on S-1, a new image I-2 will be created *and* it will inherit the image properties from I-1, including the cinder_encryption_key_id and cinder_encryption_key_deletion_policy metadata.  Since I-2 has a reference to the barbican secret for I-1, when I-2 is deleted, the secret for I-1 will be deleted, thereby making it impossible to decrypt I-1.


Version-Release number of selected component (if applicable): 16 (Train)
This does not occur in earlier versions.


How reproducible: always


Steps to Reproduce:
1. in cinder: create volume V-1 of an encrypted volume-type
2. in cinder: use the upload-volume-to-image action to create I-1 in glance
3. in nova: boot an instance S-1 from I-1
4. in nova: do the image-create action on S-1, yielding image I-2 in glance
5. in glance: delete image I-2


Actual results:
The barbican secret whose uuid is contained in the cinder_encryption_key_id image property of image I-1 is deleted.


Expected results:
The barbican secret for image I-1 should *not* be deleted.


Additional info:
Nova has a non_inheritable_image_properties configuration option.  The 'cinder_encryption_key_*' properties should be added to this list, and then this situation will not occur.

Comment 2 Brian Rosmaita 2020-02-19 18:30:21 UTC
See http://lists.openstack.org/pipermail/openstack-discuss/2020-February/012641.html
for the complete write-up sent to the openstack mailing list.

Comment 3 Lee Yarwood 2020-02-21 10:17:46 UTC

*** This bug has been marked as a duplicate of bug 1801255 ***


Note You need to log in before you can comment on or make changes to this bug.