Description of problem:

$subject, by default admins do not have access to user created barbican secrets. As a result admins cannot hard reboot or stop/start instances as this deletes local libvirt secrets, refetches secrets from Barbican and recreates the local secrets.

However this initial attempt by an admin will destroy the local secrets *before* failing to access anything in Barbican.

As a result any request by the owner of the instance to hard reboot or stop/start the instance can fail as the _detach_encryptor logic fails to find any local secret and assumes that native LUKSv1 encryption isn't being used. This causes the os-brick encryptors to be loaded that can fail if the underlying volume type isn't supported, such as rbd.

This was initially seen in https://bugzilla.redhat.com/show_bug.cgi?id=1933268 and results in a trace like the following:

2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server [req-fe304872-e35f-4cb3-8760-4fd1eed745bc fef8c04ca63ab77e9a37b9d79367fd49747d2016352759f6faa8475fbf6f63c1 4127275f099844f28fde120064aa4753 - 1d485afd913b4c489730f79d83044080 1d485afd913b4c489730f79d83044080] Exception during message handling: os_brick.exception.VolumeEncryptionNotSupported: Volume encryption is not supported for rbd volume d9817c6a-9c84-472a-8fc8-58ad73b389aa.
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server Traceback (most recent call last):
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 79, in wrapped
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     function_name, call_dict, binary, tb)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     raise value
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 69, in wrapped
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     return f(self, context, *args, **kw)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 191, in decorated_function
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     "Error: %s", e, instance=instance)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     raise value
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 161, in decorated_function
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1372, in decorated_function
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 219, in decorated_function
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     kwargs['instance'], e, sys.exc_info())
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     raise value
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 207, in decorated_function
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     return function(self, context, *args, **kwargs)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 3140, in start_instance
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self._power_on(context, instance)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 3110, in _power_on
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     block_device_info)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3459, in power_on
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self._hard_reboot(context, instance, network_info, block_device_info)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3306, in _hard_reboot
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     block_device_info=block_device_info)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1316, in destroy
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     destroy_disks)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1389, in cleanup
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     cleanup_instance_disks=cleanup_instance_disks)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1474, in _cleanup
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     instance=instance)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self.force_reraise()
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     six.reraise(self.type_, self.value, self.tb)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     raise value
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1461, in _cleanup
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self._disconnect_volume(context, connection_info, instance)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1687, in _disconnect_volume
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     self._detach_encryptor(context, connection_info, encryption=encryption)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1813, in _detach_encryptor
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     encryption)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 1730, in _get_volume_encryptor
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     **encryption)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/os_brick/encryptors/__init__.py", line 93, in get_volume_encryptor
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     **kwargs)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/oslo_utils/importutils.py", line 44, in import_object
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     return import_class(import_str)(*args, **kwargs)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/os_brick/encryptors/luks.py", line 61, in __init__
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     *args, **kwargs)
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.6/site-packages/os_brick/encryptors/cryptsetup.py", line 55, in __init__
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server     volume_type=connection_info['driver_volume_type'])
2021-02-23 17:07:50.453 7 ERROR oslo_messaging.rpc.server os_brick.exception.VolumeEncryptionNotSupported: Volume encryption is not supported for rbd volume d9817c6a-9c84-472a-8fc8-58ad73b389aa.

Version-Release number of selected component (if applicable):
13, 16.{0,1,2} and 17

How reproducible:
Always

Steps to Reproduce:
1. As an non-admin user create an instance with encrypted rbd volumes attached
2. Attempt to hard reboot or stop/start the instance as an admin
3. Attempt to hard reboot or stop/start the instance as the owner

Actual results:
The request by the admin to hard reboot or stop/start the instance fails.
The request by the owner to hard reboot or stop/start the instance fails due to os_brick.exception.VolumeEncryptionNotSupported being raised.

Expected results:
The request by the admin to hard reboot or stop/start the instance fails.
The request by the owner to hard reboot or stop/start the instance succeeds.

Additional info: