Bug 1021966 - cinder: deleting several snapshots at the same time from horizon cause snapshots to get stuck in 'error_deleting' state
cinder: deleting several snapshots at the same time from horizon cause snapsh...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder (Show other bugs)
4.0
x86_64 Linux
unspecified Severity high
: rc
: 4.0
Assigned To: Eric Harney
Tzach Shefi
storage
:
: 1030659 (view as bug list)
Depends On: 1030659
Blocks:
  Show dependency treegraph
 
Reported: 2013-10-22 08:27 EDT by Dafna Ron
Modified: 2016-04-26 11:19 EDT (History)
5 users (show)

See Also:
Fixed In Version: openstack-cinder-2013.2-3.el6ost
Doc Type: Bug Fix
Doc Text:
GlusterFS driver operations in the Block Storage service now use locking when performing multiple snapshot operations. Locking ensures that multiple operations interacting with qcow2 data in the Block Storage service's GlusterFS snapshot driver do not happen concurrently. In previous releases, the lack of locking sometimes resulted in failed operations or even corruption of the snapshot state whenever multiple snapshot operations were issued.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-19 19:30:40 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
logs (3.18 KB, application/x-gzip)
2013-10-22 08:27 EDT, Dafna Ron
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Bugzilla 1021966 None None None Never
Launchpad 1243170 None None None Never

  None (edit)
Description Dafna Ron 2013-10-22 08:27:49 EDT
Created attachment 814966 [details]
logs

Description of problem:

working with glusterfs as cinder backend I created a volume and 2 snapshots
from horizon I selected both snapshots and delete. 
we fail to delete one of the snapshots and it gets stuck in error_deleting state


Version-Release number of selected component (if applicable):

openstack-cinder-2013.2-0.11.rc1.el6ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. create a volume 
2. create 2 snapshots
3. from horizon, select both snapshots and delete them. 


Actual results:

1 of the snapshots fails to be deleted and get stuck in error_deleting state

Expected results:

we should be able to delete all snapshots + in case we fail we should move snapshot to error 

Additional info:

2013-10-22 14:53:14.796 27438 WARNING cinder.quota [req-53fcd16f-6721-40f1-aa42-2cf433dca3a1 a660044c9b074450aaa45fba0d641fcc e27aae2598b94dca88cd0408406e0848] Deprecated: Default quota for resource: snapshots is set by the default quota flag: quota_snapshots, it is now deprecated. Please use the the default quota class for default quota.
2013-10-22 14:53:15.454 27438 ERROR cinder.openstack.common.rpc.amqp [req-e45c2a25-35ac-492f-a7fa-9c695f2d63be a660044c9b074450aaa45fba0d641fcc e27aae2598b94dca88cd0408406e0848] Exception during message handling
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp Traceback (most recent call last):
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/amqp.py", line 441, in _process_data
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     **args)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/openstack/common/rpc/dispatcher.py", line 148, in dispatch
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     return getattr(proxyobj, method)(ctxt, **kwargs)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/utils.py", line 808, in wrapper
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     return func(self, *args, **kwargs)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 405, in delete_snapshot
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     {'status': 'error_deleting'})
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib64/python2.6/contextlib.py", line 23, in __exit__
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     self.gen.next()
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/volume/manager.py", line 393, in delete_snapshot
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     self.driver.delete_snapshot(snapshot_ref)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp   File "/usr/lib/python2.6/site-packages/cinder/volume/drivers/glusterfs.py", line 650, in delete_snapshot
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp     raise exception.GlusterfsException(msg)
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp GlusterfsException: No snap found with volume-f9d6b98f-8394-4a01-9424-f23897382d87.1493f904-4994-44ed-be1a-8270f5070919 as backing file.
2013-10-22 14:53:15.454 27438 TRACE cinder.openstack.common.rpc.amqp 
2013-10-22 14:55:05.935 27438 WARNING cinder.quota [req-1f7f3a14-b8e9-47bf-b33c-748e4f4ab830 a6
Comment 1 Eric Harney 2013-11-15 12:30:32 EST
This is bug 1030659.
Comment 2 Eric Harney 2013-11-22 10:23:34 EST
*** Bug 1030659 has been marked as a duplicate of this bug. ***
Comment 4 Dafna Ron 2013-12-12 07:56:09 EST
verified on python-django-horizon-2013.2-8.el6ost.noarch
Comment 6 errata-xmlrpc 2013-12-19 19:30:40 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHEA-2013-1859.html

Note You need to log in before you can comment on or make changes to this bug.