Bug 1021966 - cinder: deleting several snapshots at the same time from horizon cause snapshots to get stuck in 'error_deleting' state
Summary: cinder: deleting several snapshots at the same time from horizon cause snapsh...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 4.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: 4.0
Assignee: Eric Harney
QA Contact: Tzach Shefi
URL:
Whiteboard: storage
: 1030659 (view as bug list)
Depends On: 1030659
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-22 12:27 UTC by Dafna Ron
Modified: 2016-04-26 15:19 UTC (History)
5 users (show)

Fixed In Version: openstack-cinder-2013.2-3.el6ost
Doc Type: Bug Fix
Doc Text:
GlusterFS driver operations in the Block Storage service now use locking when performing multiple snapshot operations. Locking ensures that multiple operations interacting with qcow2 data in the Block Storage service's GlusterFS snapshot driver do not happen concurrently. In previous releases, the lack of locking sometimes resulted in failed operations or even corruption of the snapshot state whenever multiple snapshot operations were issued.
Clone Of:
Environment:
Last Closed: 2013-12-20 00:30:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (3.18 KB, application/x-gzip)
2013-10-22 12:27 UTC, Dafna Ron
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1243170 0 None None None Never
Red Hat Bugzilla 1021966 0 unspecified CLOSED cinder: deleting several snapshots at the same time from horizon cause snapshots to get stuck in 'error_deleting' state 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHEA-2013:1859 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform Enhancement Advisory 2013-12-21 00:01:48 UTC

Internal Links: 1021966 1045428

Description Dafna Ron 2013-10-22 12:27:49 UTC
Created attachment 814966 [details]
logs

Description of problem:

working with glusterfs as cinder backend I created a volume and 2 snapshots
from horizon I selected both snapshots and delete. 
we fail to delete one of the snapshots and it gets stuck in error_deleting state


Version-Release number of selected component (if applicable):

openstack-cinder-2013.2-0.11.rc1.el6ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. create a volume 
2. create 2 snapshots
3. from horizon, select both snapshots and delete them. 


Actual results:

1 of the snapshots fails to be deleted and get stuck in error_deleting state

Expected results:

we should be able to delete all snapshots + in case we fail we should move snapshot to error 

Additional info:

2013-10-22 14:53:14.796 27438 WARNING cinder.quota [req-53fcd16f-6721-40f1-aa42-2cf433dca3a1 a660044c9b074450aaa45fba0d641fcc e27aae2598b94dca88cd0408406e0848] Deprecated: Default quota for resource: snapshots is set by the default quota flag: quota_snapshots, it is now deprecated. Please use the the default quota class for default quota.
2013-10-22 14:53:15.454 27438 ERROR cinder.openstack.common.rpc.amqp [req-e45c2a25-35ac-492f-a7fa-9c695f2d63be a660044c9b074450aaa45fba0d641fcc e27aae2598b94dca88cd0408406e0848] Exception during message handling
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp Traceback (most recent call last):
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/amqp.py", line 441, in _process_data
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     **args)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/dispatcher.py", line 148, in dispatch
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     return getattr(proxyobj, method)(ctxt, **kwargs)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/utils.py", line 808, in wrapper
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     return func(self, *args, **kwargs)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 405, in delete_snapshot
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     {'status': 'error_deleting'})
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     self.gen.next()
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 393, in delete_snapshot
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     self.driver.delete_snapshot(snapshot_ref)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/glusterfs.py", line 650, in delete_snapshot
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     raise exception.GlusterfsException(msg)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp GlusterfsException: No snap found with volume-f9d6b98f-8394-4a01-9424-f23897382d87.1493f904-4994-44ed-be1a-8270f5070919 as backing file.
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp 
2013-10-22 14:55:05.935 27438 WARNING cinder.quota [req-1f7f3a14-b8e9-47bf-b33c-748e4f4ab830 a6

Comment 1 Eric Harney 2013-11-15 17:30:32 UTC
This is bug 1030659.

Comment 2 Eric Harney 2013-11-22 15:23:34 UTC
*** Bug 1030659 has been marked as a duplicate of this bug. ***

Comment 4 Dafna Ron 2013-12-12 12:56:09 UTC
verified on python-django-horizon-2013.2-8.el6ost.noarch

Comment 6 errata-xmlrpc 2013-12-20 00:30:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2013-1859.html


Note You need to log in before you can comment on or make changes to this bug.