+++ This bug was initially created as a clone of Bug #1795507 +++ Description of problem: Hi, OpenStack cloud with backend - 3parfc Encrypted volume attached to an instance, When extending the volume, cinder reports that action was done successfully and the size changed BUT inside the VM the lsblk reports that the size was not changed. We found a traceback from nova about failures in resize: var/log/containers/nova/nova-compute.log:71:2020-01-28 07:14:17.394 7 ERROR nova.virt.libvirt.driver [instance: f93e235e-30e9-4bdc-ad48-8e972552bbb5] /var/log/containers/nova/nova-compute.log:73:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server [req-ef88174f-6762-4842-a64c-76074a848ac7 b753e5c55ba94950b5463ae41bb623ab 08d5a55b6cad4413910abc863b4a2b15 - default default] Exception during message handling: libvirt.libvirtError: internal error: unable to execute QEMU command 'block_resize': Cannot grow device files /var/log/containers/nova/nova-compute.log:74:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server Traceback (most recent call last): /var/log/containers/nova/nova-compute.log:75:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming /var/log/containers/nova/nova-compute.log:76:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server res = self.dispatcher.dispatch(message) /var/log/containers/nova/nova-compute.log:77:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 274, in dispatch /var/log/containers/nova/nova-compute.log:78:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server return self._do_dispatch(endpoint, method, ctxt, args) /var/log/containers/nova/nova-compute.log:79:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_messaging/rpc/dispatcher.py", line 194, in _do_dispatch /var/log/containers/nova/nova-compute.log:80:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server result = func(ctxt, **new_args) /var/log/containers/nova/nova-compute.log:81:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 79, in wrapped /var/log/containers/nova/nova-compute.log:82:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server function_name, call_dict, binary, tb) /var/log/containers/nova/nova-compute.log:83:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ /var/log/containers/nova/nova-compute.log:84:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server self.force_reraise() /var/log/containers/nova/nova-compute.log:85:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise /var/log/containers/nova/nova-compute.log:86:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) /var/log/containers/nova/nova-compute.log:87:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise /var/log/containers/nova/nova-compute.log:88:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server raise value /var/log/containers/nova/nova-compute.log:89:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/exception_wrapper.py", line 69, in wrapped /var/log/containers/nova/nova-compute.log:90:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server return f(self, context, *args, **kw) /var/log/containers/nova/nova-compute.log:91:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 9223, in external_instance_event /var/log/containers/nova/nova-compute.log:92:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server self.extend_volume(context, instance, event.tag) /var/log/containers/nova/nova-compute.log:93:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/utils.py", line 1372, in decorated_function /var/log/containers/nova/nova-compute.log:94:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) /var/log/containers/nova/nova-compute.log:95:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 219, in decorated_function /var/log/containers/nova/nova-compute.log:96:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server kwargs['instance'], e, sys.exc_info()) /var/log/containers/nova/nova-compute.log:97:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ /var/log/containers/nova/nova-compute.log:98:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server self.force_reraise() /var/log/containers/nova/nova-compute.log:99:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise /var/log/containers/nova/nova-compute.log:100:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) /var/log/containers/nova/nova-compute.log:101:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise /var/log/containers/nova/nova-compute.log:102:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server raise value /var/log/containers/nova/nova-compute.log:103:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 207, in decorated_function /var/log/containers/nova/nova-compute.log:104:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server return function(self, context, *args, **kwargs) /var/log/containers/nova/nova-compute.log:105:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/compute/manager.py", line 9081, in extend_volume /var/log/containers/nova/nova-compute.log:106:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server bdm.volume_size * units.Gi) /var/log/containers/nova/nova-compute.log:107:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 2048, in extend_volume /var/log/containers/nova/nova-compute.log:108:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server instance=instance) /var/log/containers/nova/nova-compute.log:109:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 220, in __exit__ /var/log/containers/nova/nova-compute.log:110:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server self.force_reraise() /var/log/containers/nova/nova-compute.log:111:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 196, in force_reraise /var/log/containers/nova/nova-compute.log:112:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server six.reraise(self.type_, self.value, self.tb) /var/log/containers/nova/nova-compute.log:113:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise /var/log/containers/nova/nova-compute.log:114:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server raise value /var/log/containers/nova/nova-compute.log:115:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 2037, in extend_volume /var/log/containers/nova/nova-compute.log:116:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server dev.resize(new_size // units.Ki) /var/log/containers/nova/nova-compute.log:117:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/nova/virt/libvirt/guest.py", line 809, in resize /var/log/containers/nova/nova-compute.log:118:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server self._guest._domain.blockResize(self._disk, size_kb) /var/log/containers/nova/nova-compute.log:119:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit /var/log/containers/nova/nova-compute.log:120:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server result = proxy_call(self._autowrap, f, *args, **kwargs) /var/log/containers/nova/nova-compute.log:121:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call /var/log/containers/nova/nova-compute.log:122:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server rv = execute(f, *args, **kwargs) /var/log/containers/nova/nova-compute.log:123:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute /var/log/containers/nova/nova-compute.log:124:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server six.reraise(c, e, tb) /var/log/containers/nova/nova-compute.log:125:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/six.py", line 693, in reraise /var/log/containers/nova/nova-compute.log:126:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server raise value /var/log/containers/nova/nova-compute.log:127:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib/python3.6/site-packages/eventlet/tpool.py", line 83, in tworker /var/log/containers/nova/nova-compute.log:128:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server rv = meth(*args, **kwargs) /var/log/containers/nova/nova-compute.log:129:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server File "/usr/lib64/python3.6/site-packages/libvirt.py", line 1006, in blockResize /var/log/containers/nova/nova-compute.log:130:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server if ret == -1: raise libvirtError ('virDomainBlockResize() failed', dom=self) /var/log/containers/nova/nova-compute.log:131:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server libvirt.libvirtError: internal error: unable to execute QEMU command 'block_resize': Cannot grow device files /var/log/containers/nova/nova-compute.log:132:2020-01-28 07:14:17.778 7 ERROR oslo_messaging.rpc.server /var/log/containers/libvirt/libvirtd.log:2:2020-01-28 07:14:17.392+0000: 4425: error : qemuMonitorJSONCheckError:418 : internal error: unable to execute QEMU command 'block_resize': Cannot grow device files Thanks, Benny --- Additional comment from Tzach Shefi on 2020-01-28 07:33:51 UTC --- Version: openstack-nova-compute-20.0.2-0.20191230035951.27bfd0b.el8ost.noarch openstack-nova-migration-20.0.2-0.20191230035951.27bfd0b.el8ost.noarch python3-novaclient-15.1.0-0.20190919143437.cd396b8.el8ost.noarch python3-nova-20.0.2-0.20191230035951.27bfd0b.el8ost.noarch puppet-nova-15.4.1-0.20191126042922.b1bb388.el8ost.noarch openstack-nova-common-20.0.2-0.20191230035951.27bfd0b.el8ost.noarch If this isn't supported, than this BZ should turn into a doc bug no support for extending an attached encrypted volume. --- Additional comment from Tzach Shefi on 2020-01-28 08:29:00 UTC --- FYI retested, This time encrypted volume was detached before extending and reattached after. Extending worked fine in this case, lsblk reported correct extended size. This we know extending an attached volume doesn't support encrypted volumes, Workaround detach the volume extend and reattach it.
If this bug requires doc text for errata release, please set the 'Doc Type' and provide draft text according to the template in the 'Doc Text' field. The documentation team will review, edit, and approve the text. If this bug does not require doc text, please set the 'requires_doc_text' flag to '-'.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:3148