Created attachment 808973 [details]
logs

Description of problem:

we had an outage and after all servers were restarted I tried sending a delete snapshot command. 
since there was a problem in gluster created by the outage cinder still failed to mount the storage and was down. 
once problem in storage was fixed, the snapshots are still stuck in deleting status. 

Version-Release number of selected component (if applicable):

[root@cougar06 ~(keystone_admin)]# rpm -qa |grep cinder 
python-cinder-2013.2-0.9.b3.el6ost.noarch
python-cinderclient-1.0.5-1.el6ost.noarch
openstack-cinder-2013.2-0.9.b3.el6ost.noarch

How reproducible:


Steps to Reproduce:
1. configure cinder to work with gluster
2. create a volume and a snapshot
3. shut down gluster 
4. send a delete snapshot command
5. restart gluster and mount

Actual results:

snapshots are stuck in deleting forever 

Expected results:

snapshots should be deleted once the storage is back or failed right away by scheduler.
  
Additional info:

[root@cougar06 ~(keystone_admin)]# cinder snapshot-list 
+--------------------------------------+--------------------------------------+----------+--------------+------+
|                  ID                  |              Volume ID               |  Status  | Display Name | Size |
+--------------------------------------+--------------------------------------+----------+--------------+------+
| 4a6597fc-d33c-49b0-88e4-3dc0a6e9855c | 1560fa00-752b-4d7b-a747-3ef9bf483692 | deleting |     None     |  1   |
| 4ac236e1-61f1-47dc-a0fa-d96162622121 | 1560fa00-752b-4d7b-a747-3ef9bf483692 | deleting |     None     |  1   |
+--------------------------------------+--------------------------------------+----------+--------------+------+



https://bugs.launchpad.net/cinder/+bug/1236506