Bug 1257304 - glusterFS_native_driver: snapshot delete doesn't delete snapshot entries that are in error state
glusterFS_native_driver: snapshot delete doesn't delete snapshot entries that...
Status: CLOSED NEXTRELEASE
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-manila (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 7.0 (Kilo)
Assigned To: Csaba Henk
nlevinki
Don Domingo
: ZStream
Depends On:
Blocks: 1263541
  Show dependency treegraph
 
Reported: 2015-08-26 13:53 EDT by krishnaram Karthick
Modified: 2015-11-10 10:12 EST (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
With the the File Share Service, when an attempt to create a snapshot of a provisioned share fails, an entry for the snapshot will still be created. However, this entry will be in an 'error' state, and any attempts to delete it will fail. To prevent this, avoid creating share snapshots if the back end volume, service, or host is down.
Story Points: ---
Clone Of:
: 1263541 (view as bug list)
Environment:
Last Closed: 2015-09-16 02:53:35 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Launchpad 1489097 None None None Never

  None (edit)
Description krishnaram Karthick 2015-08-26 13:53:23 EDT
Description of problem:

In openstack-manila running kilo release, if snapshot creation fails for some reason, an entry for snapshot is still created with 'error' state. Attempting to delete this invalid entry of snapshot doesn't succeed.

Possible scenarios which could create a snapshot in error state are when,

 - gluster daemon is down in the backend gluster node
 - gluster node is down
 - gluster brick process is down
 - gluster volume is down
 - snapshot created by manila is deleted from backend gluster

While it is okay to create a snapshot with error state, attempting to delete or force-delete should clear the entry from database. Administrator has no means to remove these snapshots in error state.

Error msg from the logs:

GlusterfsException: Failed to identify backing GlusterFS object for snapshot 0a6e89a3-cbc3-4322-bcb7-05282b3b9816 of share 78fea54d-78f6-4d18-b478-4b40ad2b7c30: a single candidate was expected, 0 was found.

CLI output:

# manila snapshot-list
+--------------------------------------+--------------------------------------+--------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+--------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | error | snap-01 | 1 |
+--------------------------------------+--------------------------------------+--------+---------+------------+
# manila snapshot-delete fac9835a-9689-46a0-8ee7-d2f2d12ad486
# manila snapshot-list
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | error_deleting | snap-01 | 1 |
+--------------------------------------+--------------------------------------+----------------+---------+------------+

# manila snapshot-reset-state fac9835a-9689-46a0-8ee7-d2f2d12ad486
# manila snapshot-list
+--------------------------------------+--------------------------------------+-----------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+-----------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | available | snap-01 | 1 |
+--------------------------------------+--------------------------------------+-----------+---------+------------+

# manila snapshot-force-delete fac9835a-9689-46a0-8ee7-d2f2d12ad486
# manila snapshot-list
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| ID | Share ID | Status | Name | Share Size |
+--------------------------------------+--------------------------------------+----------------+---------+------------+
| fac9835a-9689-46a0-8ee7-d2f2d12ad486 | 68c37e73-9710-430a-8f39-7d79ad6af669 | error_deleting | snap-01 | 1 |
+--------------------------------------+--------------------------------------+----------------+---------+------------+


Version-Release number of selected component (if applicable):

openstack-manila-2015.1.0-2

How reproducible:

Always

Steps to Reproduce:

1) Try to create a snapshot when volume is down (one of the possible ways to create a snapshot that would end up in error state)
2) A snapshot in error state is created
3) Attempt to delete or force-delete the snapshot

Actual results:

snapshot entry in error state is not deleted

Expected results:

snapshot entry in error state should be deleted

Additional info:
Comment 3 krishnaram Karthick 2015-08-28 01:07:11 EDT
Additionally, the underlying share cannot be deleted too if there are any snapshots in error state.
Comment 4 Csaba Henk 2015-09-16 02:53:35 EDT
No fix shall be provided for upstream Kilo and thus it's not fixed in RHEL OSP7 either.
It's added to the OSP7 Errata, see
https://access.redhat.com/documentation/en/red-hat-enterprise-linux-openstack-platform/version-7/release-notes#idm140465730614816

The bug is handled in upstream Liberty. To track the downstream port of that fix, please see the variant of this bug for OSP8:

BZ #1263541 https://bugzilla.redhat.com/show_bug.cgi?id=1263541

Note You need to log in before you can comment on or make changes to this bug.