Bug 980977 - [RHS-RHOS] If storage becomes unavailable volume actions fail (error_deleting)
[RHS-RHOS] If storage becomes unavailable volume actions fail (error_deleting)
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-cinder (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 5.0 (RHEL 7)
Assigned To: Sergey Gotliv
Dafna Ron
Depends On:
  Show dependency treegraph
Reported: 2013-07-03 13:50 EDT by Ben Turner
Modified: 2016-04-26 09:33 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
virt rhos cinder integration
Last Closed: 2014-05-29 11:40:47 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Ben Turner 2013-07-03 13:50:31 EDT
Description of problem:

If the storage back end becomes available during volume actions the status in the webUI gets stuck:

test-vol-BZ "stopping gluster volume while creating."	10GB	Error_Deleting

I had to go into the database and manually remove the entries to fix the web UI:

mysql --host=localhost --user=root --password=blah
show tables;
select volumes;
select * from volumes;
select id,host,display_name from volumes
delete from volumes where display_name='test-vol-BZ';

Version-Release number of selected component (if applicable):


How reproducible:

I have hit this a couple of different ways so far:

-Power off all gluster nodes during a volume create
-Stop mounted gluster volume, then delete the cinder volume

Steps to Reproduce:
1.  Create a new volume on a glusterfs backend
2.  On the gluster server run:

# gluster volume stop testvol

3.  Attempt to delete the volume from the webUI while the gluster volume is stopped.

4.  Restart the gluster volume:

# gluster volume start testvol

Actual results:

The volume status hangs at error_deleting.  I get an error message in the webUI saying:

You do not have permission to delete volume: test-vol-BZ

Expected results:

I am able to delete the volume after storage comes back up.

Additional info:

I can see this being problematic when there are connectivity issues and/or if there are split brain issues on the gluster back end.
Comment 1 Eric Harney 2013-07-03 13:54:16 EDT
I believe "cinder force-delete" will allow you to remove the volume rather than manipulating the database.
Comment 5 Ayal Baron 2013-08-06 05:32:07 EDT
(In reply to Eric Harney from comment #1)
> I believe "cinder force-delete" will allow you to remove the volume rather
> than manipulating the database.

Eric, regardless of a workaround of using force-delete, in the least the error message is incorrect.
In addition, what happens if the user issues delete again? would that work?

Note You need to log in before you can comment on or make changes to this bug.