Bug 1024391

Summary: snapshot is stuck in error_deleting if we try to delete it while volume is attached to instance
Product: Red Hat OpenStack Reporter: Dafna Ron <dron>
Component: openstack-cinderAssignee: Eric Harney <eharney>
Status: CLOSED UPSTREAM QA Contact: Dafna Ron <dron>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 4.0CC: eharney, sgotliv, yeylon
Target Milestone: ---   
Target Release: 5.0 (RHEL 7)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-05-01 16:40:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logs none

Description Dafna Ron 2013-10-29 14:59:10 UTC
Created attachment 817110 [details]
logs

Description of problem:

I am running with gluster backend to cinder 
when I boot an instance from a volume and take a snapshot of the instance we take a volume snapshot. 
if we try to delete the snapshot while the instance is still running the snapshot is stuck in error_deleting. 

Version-Release number of selected component (if applicable):

openstack-cinder-2013.2-1.el6ost.noarch

How reproducible:

100%

Steps to Reproduce:
1. boot an instance from a volume 
2. take a snapshot of the instance 
3. try to delete the volume snapshot created 

Actual results:

snapshot are stuck in status error_deleting

Expected results:

if we cannot delete the snapshot while the volume is attached we need to fail action. 

Additional info:

Comment 1 Eric Harney 2013-10-30 22:00:08 UTC
> if we cannot delete the snapshot while the volume is attached we need to fail action.

Not sure what you mean, error_deleting is the result of a failed delete action.


This failure appears to be a further result of the fact that the Cinder user can't write to the GlusterFS share.  (create_snapshot failed to write the info file, meaning the snapshot metadata is inconsistent.)

Comment 2 Dafna Ron 2013-10-31 08:44:06 UTC
(In reply to Eric Harney from comment #1)
> > if we cannot delete the snapshot while the volume is attached we need to fail action.
> 
> Not sure what you mean, error_deleting is the result of a failed delete
> action.
> 

the status of the image should not be error_deleting since we cannot try to re-delete the snapshot once that happens because this status is illegal for delete... 

like all other objects in the system we should move it to status error (unless you would like to add status "error_deleting" to the list of allowed statuses for delete - that would be a good solution as well). 

[root@cougar06 ~(keystone_admin)]# cinder snapshot-list 
+--------------------------------------+--------------------------------------+----------------+-------------------+------+
|                  ID                  |              Volume ID               |     Status     |    Display Name   | Size |
+--------------------------------------+--------------------------------------+----------------+-------------------+------+
| 9f351882-6f3b-4e6f-aabc-da565bd3bf00 | 6de84c80-1247-4b9d-9614-62e5c7a725cc | error_deleting | snapshot for boo1 |  10  |
| aa98b95a-764a-4134-b7f9-b5c0e3920657 | 6de84c80-1247-4b9d-9614-62e5c7a725cc | error_deleting |  snapshot for boo |  10  |
| ec9b5c1d-ec65-4683-b29e-1afd98af1fde | 6de84c80-1247-4b9d-9614-62e5c7a725cc | error_deleting | snapshot for boo2 |  10  |
+--------------------------------------+--------------------------------------+----------------+-------------------+------+
[root@cougar06 ~(keystone_admin)]# 


> 
> This failure appears to be a further result of the fact that the Cinder user
> can't write to the GlusterFS share.  (create_snapshot failed to write the
> info file, meaning the snapshot metadata is inconsistent.)

Comment 3 Eric Harney 2013-10-31 15:07:27 UTC
(In reply to Dafna Ron from comment #2)
> (In reply to Eric Harney from comment #1)
> > > if we cannot delete the snapshot while the volume is attached we need to fail action.
> > 
> > Not sure what you mean, error_deleting is the result of a failed delete
> > action.
> > 
> 
> the status of the image should not be error_deleting since we cannot try to
> re-delete the snapshot once that happens because this status is illegal for
> delete... 
> 
> like all other objects in the system we should move it to status error
> (unless you would like to add status "error_deleting" to the list of allowed
> statuses for delete - that would be a good solution as well). 

IMO you should be able to trigger another delete operation from the "error_deleting" state, but it is correct that you can't currently.  (I've considered this change before, myself.)

This situation can be fixed by doing:
# cinder snapshot-reset-state --state error <snapshot>
# cinder snapshot-delete <snapshot>

Comment 4 Eric Harney 2013-11-20 16:49:12 UTC
Related case upstream: LP 1223189.