Description of problem:
when we try to migrate or resize an instance (not a live migration), the instance status is switched to "Error" even if the resize was done successfully.
In the nova log file, we have this :
~~~
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
executor_callback))
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
executor_callback)
File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
result = func(ctxt, **new_args)
File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
payload)
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__
six.reraise(self.type_, self.value, self.tb)
File "/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
return f(self, context, *args, **kw)
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 400, in decorated_function
return function(self, context, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 378, in decorated_function
kwargs['instance'], e, sys.exc_info())
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__
six.reraise(self.type_, self.value, self.tb)
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 366, in decorated_function
return function(self, context, *args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3428, in confirm_resize
do_confirm_resize(context, instance, migration.id)
File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 254, in inner
return f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3426, in do_confirm_resize
migration=migration)
File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3453, in _confirm_resize
network_info)
File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7199, in confirm_migration
self._cleanup_resize(instance, network_info)
File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1050, in _cleanup_resize
ignore_errors=True)
File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 885, in remove_snap
return self.driver.remove_snap(self.rbd_name, name, ignore_errors)
File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 391, in remove_snap
with RBDVolumeProxy(self, volume) as vol:
File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 65, in __init__
driver._disconnect_from_rados(client, ioctx)
File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__
six.reraise(self.type_, self.value, self.tb)
File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 61, in __init__
read_only=read_only)
File "/usr/lib/python2.7/site-packages/rbd.py", line 374, in __init__
raise make_ex(ret, 'error opening image %s at snapshot %s' % (name, snapshot))
ImageNotFound: error opening image dda93e33-f1a3-450c-bfaf-52cc2747fb10_disk at snapshot None
~~~
Version-Release number of selected component (if applicable):
RHEL OSP 8
openstack-nova-compute-12.0.3-1.el7ost.noarch
How reproducible:
Everytime for Cu
Steps to Reproduce:
- ceph is configured as cinder backend.
- You have instance booted from rbd volume [boot from volume]
- Resize the instance.
Actual results:
Instance is ending with error state.
Expected results:
It should get completed successfully.
Additional info:
Issue is already reported upstream [1]
Got a confirmation from Cu that issue is fixed after applying the patch [2]
[1] https://bugs.launchpad.net/nova/+bug/1580625
[2] https://git.openstack.org/cgit/openstack/nova/commit/?id=743d5efccaa99e3b4873831a8f43c216a31c7113
I checked the changelog of the recent openstack-nova version till the version which Cu is using but I didn't find the bug fixed in that version.
Description of problem: when we try to migrate or resize an instance (not a live migration), the instance status is switched to "Error" even if the resize was done successfully. In the nova log file, we have this : ~~~ Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply executor_callback)) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch executor_callback) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch result = func(ctxt, **new_args) File "/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped payload) File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped return f(self, context, *args, **kw) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 400, in decorated_function return function(self, context, *args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 378, in decorated_function kwargs['instance'], e, sys.exc_info()) File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 366, in decorated_function return function(self, context, *args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3428, in confirm_resize do_confirm_resize(context, instance, migration.id) File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 254, in inner return f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3426, in do_confirm_resize migration=migration) File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 3453, in _confirm_resize network_info) File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 7199, in confirm_migration self._cleanup_resize(instance, network_info) File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 1050, in _cleanup_resize ignore_errors=True) File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 885, in remove_snap return self.driver.remove_snap(self.rbd_name, name, ignore_errors) File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 391, in remove_snap with RBDVolumeProxy(self, volume) as vol: File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 65, in __init__ driver._disconnect_from_rados(client, ioctx) File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 204, in __exit__ six.reraise(self.type_, self.value, self.tb) File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 61, in __init__ read_only=read_only) File "/usr/lib/python2.7/site-packages/rbd.py", line 374, in __init__ raise make_ex(ret, 'error opening image %s at snapshot %s' % (name, snapshot)) ImageNotFound: error opening image dda93e33-f1a3-450c-bfaf-52cc2747fb10_disk at snapshot None ~~~ Version-Release number of selected component (if applicable): RHEL OSP 8 openstack-nova-compute-12.0.3-1.el7ost.noarch How reproducible: Everytime for Cu Steps to Reproduce: - ceph is configured as cinder backend. - You have instance booted from rbd volume [boot from volume] - Resize the instance. Actual results: Instance is ending with error state. Expected results: It should get completed successfully. Additional info: Issue is already reported upstream [1] Got a confirmation from Cu that issue is fixed after applying the patch [2] [1] https://bugs.launchpad.net/nova/+bug/1580625 [2] https://git.openstack.org/cgit/openstack/nova/commit/?id=743d5efccaa99e3b4873831a8f43c216a31c7113 I checked the changelog of the recent openstack-nova version till the version which Cu is using but I didn't find the bug fixed in that version.