Bug 1278397 - When volume creation fails, gluster volume and brick lvs are not deleted
When volume creation fails, gluster volume and brick lvs are not deleted
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: heketi (Show other bugs)
unspecified
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Luis Pabón
Anush Shetty
: Reopened, ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-11-05 06:51 EST by Anush Shetty
Modified: 2016-11-08 17:25 EST (History)
3 users (show)

See Also:
Fixed In Version: heketi-1.0.2-1.fc23 heketi-1.0.2-1.fc22 heketi-1.0.2-1.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1292205 (view as bug list)
Environment:
Last Closed: 2015-12-16 12:41:37 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Anush Shetty 2015-11-05 06:51:43 EST
Description of problem: When heketi volume creation fails (because of gluster volumes failing to start), heketi fails to clean up the logical volumes cit attempts to clean up the lvs without deleting the gluster volume. 


Version-Release number of selected component (if applicable): heketi-1.0.1-1.el7rhgs.x86_64


How reproducible: Always when volume creation fails.

Actual results:

Excerpt from the logs:

[sshexec] ERROR 2015/11/05 16:00:54 /builddir/build/BUILD/heketi-f9273193371ab92456a6b042e60fc883f5b67d8b/src/github.com/heketi/heketi/executors/sshexec/brick.go:125:
 Process exited with: 32. Reason was:  ()
[sshexec] ERROR 2015/11/05 16:00:54 /builddir/build/BUILD/heketi-f9273193371ab92456a6b042e60fc883f5b67d8b/src/github.com/heketi/heketi/utils/ssh/ssh.go:155: Failed to
 run command [sudo lvremove -f vg_f9188e0f1334126cdc51a5619de2e1c5/tp_813ffa9b7a442ff8234210bb648f8578] on rhshdp12.lab.eng.blr.redhat.com:22: Err[Process exited with
: 5. Reason was:  ()]: Stdout []: Stderr [  Logical volume vg_f9188e0f1334126cdc51a5619de2e1c5/brick_813ffa9b7a442ff8234210bb648f8578 contains a filesystem in use.

Expected results:

gluster volumes should be deleted before attempting to delete the logical volumes. 

Additional info:
Comment 2 Luis Pabón 2015-11-05 16:03:03 EST
https://github.com/heketi/heketi/issues/247
Comment 3 Luis Pabón 2015-11-05 17:20:52 EST
Please try out different max brick values using the Heketi version from https://github.com/heketi/heketi/issues/248 .
Comment 4 Luis Pabón 2015-11-06 15:14:32 EST
https://github.com/heketi/heketi/issues/247 has been updated with a patch.  I created a functional test to reproduce the issue.  I noticed that having a large number of maximum bricks per volume did not produce reliable results from GlusterFS volume create.  I have limited the maximum number of bricks per volume to 100.
Comment 5 Fedora Update System 2015-12-03 19:42:23 EST
heketi-1.0.2-1.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-34d6d5296a
Comment 6 Fedora Update System 2015-12-04 15:43:59 EST
heketi-1.0.2-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-6b376a19a9
Comment 7 Fedora Update System 2015-12-04 15:44:35 EST
heketi-1.0.2-1.fc22 has been submitted as an update to Fedora 22. https://bodhi.fedoraproject.org/updates/FEDORA-2015-d0abd928f1
Comment 8 Fedora Update System 2015-12-05 15:51:12 EST
heketi-1.0.2-1.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'yum --enablerepo=epel-testing update heketi'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2015-34d6d5296a
Comment 9 Fedora Update System 2015-12-06 00:22:19 EST
heketi-1.0.2-1.fc22 has been pushed to the Fedora 22 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update heketi'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-d0abd928f1
Comment 10 Fedora Update System 2015-12-06 12:20:56 EST
heketi-1.0.2-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update heketi'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-6b376a19a9
Comment 12 Anush Shetty 2015-12-09 05:16:43 EST
Verified with heketi-1.0.2-1.el7rhgs.x86_64
Comment 13 Fedora Update System 2015-12-14 05:22:03 EST
heketi-1.0.2-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 14 Fedora Update System 2015-12-14 06:51:13 EST
heketi-1.0.2-1.fc22 has been pushed to the Fedora 22 stable repository. If problems still persist, please make note of it in this bug report.
Comment 16 Fedora Update System 2015-12-30 13:04:14 EST
heketi-1.0.2-1.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.