Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1257304

Summary: glusterFS_native_driver: snapshot delete doesn't delete snapshot entries that are in error state
Product: Red Hat OpenStack Reporter: krishnaram Karthick <kramdoss>
Component: openstack-manilaAssignee: Csaba Henk <csaba>
Status: CLOSED NEXTRELEASE QA Contact: nlevinki <nlevinki>
Severity: unspecified Docs Contact: Don Domingo <ddomingo>
Priority: unspecified    
Version: 7.0 (Kilo)CC: scohen, sgotliv
Target Milestone: ---Keywords: ZStream
Target Release: 7.0 (Kilo)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
With the the File Share Service, when an attempt to create a snapshot of a provisioned share fails, an entry for the snapshot will still be created. However, this entry will be in an 'error' state, and any attempts to delete it will fail. To prevent this, avoid creating share snapshots if the back end volume, service, or host is down.
 Story Points: ---
Clone Of:
: 1263541 (view as bug list) Environment:
Last Closed: 2015-09-16 06:53:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1263541    

Description krishnaram Karthick 2015-08-26 17:53:23 UTC 
Description of problem:

In openstack-manila running kilo release, if snapshot creation fails for some reason, an entry for snapshot is still created with 'error' state. Attempting to delete this invalid entry of snapshot doesn't succeed.

Possible scenarios which could create a snapshot in error state are when,

 - gluster daemon is down in the backend gluster node
 - gluster node is down
 - gluster brick process is down
 - gluster volume is down
 - snapshot created by manila is deleted from backend gluster

While it is okay to create a snapshot with error state, attempting to delete or force-delete should clear the entry from database. Administrator has no means to remove these snapshots in error state.

Error msg from the logs:

GlusterfsException: Failed to identify backing GlusterFS object for snapshot 0a6e89a3-cbc3-4322-bcb7-05282b3b9816 of share 78fea54d-78f6-4d18-b478-4b40ad2b7c30: a single candidate was expected, 0 was found.

CLI output:

# manila snapshot-list
+--------------------------------------+--------------------------------------+--------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+--------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | error | snap-01 | 1 |
+--------------------------------------+--------------------------------------+--------+---------+------------+
# manila snapshot-delete fac9835a-9689-46a0-8ee7-d2f2d12ad486
# manila snapshot-list
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | error_deleting | snap-01 | 1 |
+--------------------------------------+--------------------------------------+----------------+---------+------------+

# manila snapshot-reset-state fac9835a-9689-46a0-8ee7-d2f2d12ad486
# manila snapshot-list
+--------------------------------------+--------------------------------------+-----------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+-----------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | available | snap-01 | 1 |
+--------------------------------------+--------------------------------------+-----------+---------+------------+

# manila snapshot-force-delete fac9835a-9689-46a0-8ee7-d2f2d12ad486
# manila snapshot-list
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | error_deleting | snap-01 | 1 |
+--------------------------------------+--------------------------------------+----------------+---------+------------+


Version-Release number of selected component (if applicable):

openstack-manila-2015.1.0-2

How reproducible:

Always

Steps to Reproduce:

1) Try to create a snapshot when volume is down (one of the possible ways to create a snapshot that would end up in error state)
2) A snapshot in error state is created
3) Attempt to delete or force-delete the snapshot

Actual results:

snapshot entry in error state is not deleted

Expected results:

snapshot entry in error state should be deleted

Additional info:
Comment 3 krishnaram Karthick 2015-08-28 05:07:11 UTC 
Additionally, the underlying share cannot be deleted too if there are any snapshots in error state.
Comment 4 Csaba Henk 2015-09-16 06:53:35 UTC 
No fix shall be provided for upstream Kilo and thus it's not fixed in RHEL OSP7 either.
It's added to the OSP7 Errata, see
https://access.redhat.com/documentation/en/red-hat-enterprise-linux-openstack-platform/version-7/release-notes#idm140465730614816

The bug is handled in upstream Liberty. To track the downstream port of that fix, please see the variant of this bug for OSP8:

BZ #1263541 https://bugzilla.redhat.com/show_bug.cgi?id=1263541