Bug 1430678

Summary:	heketi device delete command seems to be hung forever
Product:	[Red Hat Storage] Red Hat Gluster Storage	Reporter:	krishnaram Karthick <kramdoss>
Component:	heketi	Assignee:	Raghavendra Talur <rtalur>
Status:	CLOSED ERRATA	QA Contact:	krishnaram Karthick <kramdoss>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	cns-3.5	CC:	hchiramm, mliyazud, pprakash, rhs-bugs, rtalur, sselvan, storage-qa-internal
Target Milestone:	---
Target Release:	CNS 3.5
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	heketi-4.0.0-2.el7rhgs	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-04-20 18:37:49 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1415600

Description krishnaram Karthick 2017-03-09 09:50:22 UTC

Description of problem:
When a device is removed and later deleted, delete command seems to be hung forever.

# heketi-cli device remove 75671a6256cac8d63fe5a19a0380f291
Device 75671a6256cac8d63fe5a19a0380f291 Removed
# heketi-cli device delete 75671a6256cac8d63fe5a19a0380f291

snippet of heketi logs:
=========================

[negroni] Started DELETE /devices/75671a6256cac8d63fe5a19a0380f291
[heketi] INFO 2017/03/09 09:18:12 Deleting device 75671a6256cac8d63fe5a19a0380f291 on node c9846359ff29408e41020dd68847f25b
[asynchttp] INFO 2017/03/09 09:18:12 Started job 268f8b73eeb6ca56871c1e53ffdccc4a
[heketi] INFO 2017/03/09 09:18:12 Backup successful
[negroni] Completed 202 Accepted in 382.40539ms
[negroni] Started GET /queue/268f8b73eeb6ca56871c1e53ffdccc4a
[negroni] Completed 200 OK in 97.918µs
[negroni] Started GET /queue/268f8b73eeb6ca56871c1e53ffdccc4a
[negroni] Completed 200 OK in 97.61µs
[negroni] Started GET /queue/268f8b73eeb6ca56871c1e53ffdccc4a
[negroni] Completed 200 OK in 107.691µs
[negroni] Started GET /queue/268f8b73eeb6ca56871c1e53ffdccc4a
[negroni] Completed 200 OK in 80.766µs
[negroni] Started GET /queue/268f8b73eeb6ca56871c1e53ffdccc4a
[negroni] Completed 200 OK in 104.9µs

Version-Release number of selected component (if applicable):
[root@dhcp46-202 ~]# rpm -qa | grep 'heketi'
heketi-client-4.0.0-1.el7rhgs.x86_64

How reproducible:
Always

Steps to Reproduce:
1. create few volumes by submitting pvc requests and allow volumes to be created  in the backend
2. remove device and allow remove device to complete
3. delete the device on which remove command was run

Actual results:
From the cli, delete device commands seems to be hung

Expected results:
delte success or unsuccess message needs to be returned at the cli

Additional info:
heketi logs shall be provided

Comment 2 krishnaram Karthick 2017-03-09 09:56:22 UTC

sosreports are available here --> http://rhsqe-repo.lab.eng.blr.redhat.com/sosreports/1430678/

Comment 3 Raghavendra Talur 2017-03-13 08:33:59 UTC

Patch in review https://github.com/heketi/heketi/pull/710

Comment 7 krishnaram Karthick 2017-03-22 04:38:45 UTC

This issue is not seen with the latest heketi build - heketi-client-4.0.0-3.el7rhgs.x86_64.

delete device works as expected. i.e., removes the device from heketi's device list and cleans up the vgs created on top of the device.

Moving the bug to verified.

Comment 9 errata-xmlrpc 2017-04-20 18:37:49 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1111