| Summary: | cinder: deleting several snapshots at the same time from horizon cause snapshots to get stuck in 'error_deleting' state | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Dafna Ron <dron> | ||||
| Component: | openstack-cinder | Assignee: | Eric Harney <eharney> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Tzach Shefi <tshefi> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 4.0 | CC: | abaron, ddomingo, eharney, hateya, yeylon | ||||
| Target Milestone: | rc | ||||||
| Target Release: | 4.0 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | storage | ||||||
| Fixed In Version: | openstack-cinder-2013.2-3.el6ost | Doc Type: | Bug Fix | ||||
| Doc Text: |
GlusterFS driver operations in the Block Storage service now use locking when performing multiple snapshot operations. Locking ensures that multiple operations interacting with qcow2 data in the Block Storage service's GlusterFS snapshot driver do not happen concurrently.
In previous releases, the lack of locking sometimes resulted in failed operations or even corruption of the snapshot state whenever multiple snapshot operations were issued.
|
Story Points: | --- | ||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2013-12-20 00:30:40 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Bug Depends On: | 1030659 | ||||||
| Bug Blocks: | |||||||
| Attachments: |
|
||||||
This is bug 1030659. *** Bug 1030659 has been marked as a duplicate of this bug. *** verified on python-django-horizon-2013.2-8.el6ost.noarch Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2013-1859.html |
Created attachment 814966 [details] logs Description of problem: working with glusterfs as cinder backend I created a volume and 2 snapshots from horizon I selected both snapshots and delete. we fail to delete one of the snapshots and it gets stuck in error_deleting state Version-Release number of selected component (if applicable): openstack-cinder-2013.2-0.11.rc1.el6ost.noarch How reproducible: 100% Steps to Reproduce: 1. create a volume 2. create 2 snapshots 3. from horizon, select both snapshots and delete them. Actual results: 1 of the snapshots fails to be deleted and get stuck in error_deleting state Expected results: we should be able to delete all snapshots + in case we fail we should move snapshot to error Additional info: 2013-10-22 14:53:14.796 27438 WARNING cinder.quota [req-53fcd16f-6721-40f1-aa42-2cf433dca3a1 a660044c9b074450aaa45fba0d641fcc e27aae2598b94dca88cd0408406e0848] Deprecated: Default quota for resource: snapshots is set by the default quota flag: quota_snapshots, it is now deprecated. Please use the the default quota class for default quota. 2013-10-22 14:53:15.454 27438 ERROR cinder.openstack.common.rpc.amqp [req-e45c2a25-35ac-492f-a7fa-9c695f2d63be a660044c9b074450aaa45fba0d641fcc e27aae2598b94dca88cd0408406e0848] Exception during message handling 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp Traceback (most recent call last): 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/amqp.py", line 441, in _process_data 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp **args) 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/dispatcher.py", line 148, in dispatch 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp return getattr(proxyobj, method)(ctxt, **kwargs) 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/utils.py", line 808, in wrapper 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp return func(self, *args, **kwargs) 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 405, in delete_snapshot 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp {'status': 'error_deleting'}) 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__ 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp self.gen.next() 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 393, in delete_snapshot 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp self.driver.delete_snapshot(snapshot_ref) 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/glusterfs.py", line 650, in delete_snapshot 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp raise exception.GlusterfsException(msg) 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp GlusterfsException: No snap found with volume-f9d6b98f-8394-4a01-9424-f23897382d87.1493f904-4994-44ed-be1a-8270f5070919 as backing file. 2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp 2013-10-22 14:55:05.935 27438 WARNING cinder.quota [req-1f7f3a14-b8e9-47bf-b33c-748e4f4ab830 a6