Bug 980977 - [RHS-RHOS] If storage becomes unavailable volume actions fail (error_deleting)
Summary: [RHS-RHOS] If storage becomes unavailable volume actions fail (error_deleting)
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder
Version: 3.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 5.0 (RHEL 7)
Assignee: Sergey Gotliv
QA Contact: Dafna Ron
Whiteboard: storage
Depends On:
TreeView+ depends on / blocked
Reported: 2013-07-03 17:50 UTC by Ben Turner
Modified: 2016-04-26 13:33 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
virt rhos cinder integration
Last Closed: 2014-05-29 15:40:47 UTC
Target Upstream Version:

Attachments (Terms of Use)

Description Ben Turner 2013-07-03 17:50:31 UTC
Description of problem:

If the storage back end becomes available during volume actions the status in the webUI gets stuck:

test-vol-BZ "stopping gluster volume while creating."	10GB	Error_Deleting

I had to go into the database and manually remove the entries to fix the web UI:

mysql --host=localhost --user=root --password=blah
show tables;
select volumes;
select * from volumes;
select id,host,display_name from volumes
delete from volumes where display_name='test-vol-BZ';

Version-Release number of selected component (if applicable):


How reproducible:

I have hit this a couple of different ways so far:

-Power off all gluster nodes during a volume create
-Stop mounted gluster volume, then delete the cinder volume

Steps to Reproduce:
1.  Create a new volume on a glusterfs backend
2.  On the gluster server run:

# gluster volume stop testvol

3.  Attempt to delete the volume from the webUI while the gluster volume is stopped.

4.  Restart the gluster volume:

# gluster volume start testvol

Actual results:

The volume status hangs at error_deleting.  I get an error message in the webUI saying:

You do not have permission to delete volume: test-vol-BZ

Expected results:

I am able to delete the volume after storage comes back up.

Additional info:

I can see this being problematic when there are connectivity issues and/or if there are split brain issues on the gluster back end.

Comment 1 Eric Harney 2013-07-03 17:54:16 UTC
I believe "cinder force-delete" will allow you to remove the volume rather than manipulating the database.

Comment 5 Ayal Baron 2013-08-06 09:32:07 UTC
(In reply to Eric Harney from comment #1)
> I believe "cinder force-delete" will allow you to remove the volume rather
> than manipulating the database.

Eric, regardless of a workaround of using force-delete, in the least the error message is incorrect.
In addition, what happens if the user issues delete again? would that work?

Note You need to log in before you can comment on or make changes to this bug.