Bug 1024391 - snapshot is stuck in error_deleting if we try to delete it while volume is attached to instance
Summary: snapshot is stuck in error_deleting if we try to delete it while volume is at...
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 4.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 5.0 (RHEL 7)
Assignee: Eric Harney
QA Contact: Dafna Ron
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-10-29 14:59 UTC by Dafna Ron
Modified: 2016-04-26 23:21 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-05-01 16:40:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
logs (826.40 KB, application/x-gzip)
2013-10-29 14:59 UTC, Dafna Ron
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1223189 0 None None None Never
Launchpad 1245910 0 None None None Never

Description Dafna Ron 2013-10-29 14:59:10 UTC
Created attachment 817110 [details]
logs

Description of problem:

I am running with gluster backend to cinder 
when I boot an instance from a volume and take a snapshot of the instance we take a volume snapshot. 
if we try to delete the snapshot while the instance is still running the snapshot is stuck in error_deleting. 

Version-Release number of selected component (if applicable):

openstack-cinder-2013.2-1.el6ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. boot an instance from a volume 
2. take a snapshot of the instance 
3. try to delete the volume snapshot created 

Actual results:

snapshot are stuck in status error_deleting

Expected results:

if we cannot delete the snapshot while the volume is attached we need to fail action. 

Additional info:

Comment 1 Eric Harney 2013-10-30 22:00:08 UTC
> if we cannot delete the snapshot while the volume is attached we need to fail action.

Not sure what you mean, error_deleting is the result of a failed delete action.


This failure appears to be a further result of the fact that the Cinder user can't write to the GlusterFS share.  (create_snapshot failed to write the info file, meaning the snapshot metadata is inconsistent.)

Comment 2 Dafna Ron 2013-10-31 08:44:06 UTC
(In reply to Eric Harney from comment #1)
> > if we cannot delete the snapshot while the volume is attached we need to fail action.
> 
> Not sure what you mean, error_deleting is the result of a failed delete
> action.
> 

the status of the image should not be error_deleting since we cannot try to re-delete the snapshot once that happens because this status is illegal for delete... 

like all other objects in the system we should move it to status error (unless you would like to add status "error_deleting" to the list of allowed statuses for delete - that would be a good solution as well). 

[root@cougar06 ~(keystone_admin)]# cinder snapshot-list 
+--------------------------------------+--------------------------------------+----------------+-------------------+------+
|                  ID                  |              Volume ID               |     Status     |    Display Name   | Size |
+--------------------------------------+--------------------------------------+----------------+-------------------+------+
| 9f351882-6f3b-4e6f-aabc-da565bd3bf00 | 6de84c80-1247-4b9d-9614-62e5c7a725cc | error_deleting | snapshot for boo1 |  10  |
| aa98b95a-764a-4134-b7f9-b5c0e3920657 | 6de84c80-1247-4b9d-9614-62e5c7a725cc | error_deleting |  snapshot for boo |  10  |
| ec9b5c1d-ec65-4683-b29e-1afd98af1fde | 6de84c80-1247-4b9d-9614-62e5c7a725cc | error_deleting | snapshot for boo2 |  10  |
+--------------------------------------+--------------------------------------+----------------+-------------------+------+
[root@cougar06 ~(keystone_admin)]# 


> 
> This failure appears to be a further result of the fact that the Cinder user
> can't write to the GlusterFS share.  (create_snapshot failed to write the
> info file, meaning the snapshot metadata is inconsistent.)

Comment 3 Eric Harney 2013-10-31 15:07:27 UTC
(In reply to Dafna Ron from comment #2)
> (In reply to Eric Harney from comment #1)
> > > if we cannot delete the snapshot while the volume is attached we need to fail action.
> > 
> > Not sure what you mean, error_deleting is the result of a failed delete
> > action.
> > 
> 
> the status of the image should not be error_deleting since we cannot try to
> re-delete the snapshot once that happens because this status is illegal for
> delete... 
> 
> like all other objects in the system we should move it to status error
> (unless you would like to add status "error_deleting" to the list of allowed
> statuses for delete - that would be a good solution as well). 

IMO you should be able to trigger another delete operation from the "error_deleting" state, but it is correct that you can't currently.  (I've considered this change before, myself.)

This situation can be fixed by doing:
# cinder snapshot-reset-state --state error <snapshot>
# cinder snapshot-delete <snapshot>

Comment 4 Eric Harney 2013-11-20 16:49:12 UTC
Related case upstream: LP 1223189.


Note You need to log in before you can comment on or make changes to this bug.