Bug 1888760

Summary: Panic seen during brick evict op when volume does not exist in gluster but does in heketi db
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Raghavendra Talur <rtalur>
Component: heketiAssignee: John Mulligan <jmulligan>
Status: CLOSED ERRATA QA Contact: susgupta
Severity: high Docs Contact:
Priority: unspecified    
Version: ocs-3.11CC: hchiramm, madam, nigoyal, puebele, rhs-bugs, storage-qa-internal
Target Milestone: ---Keywords: ZStream
Target Release: OCS 3.11.z Batch Update 6   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: heketi-9.0.0-13.el7rhgs Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-12-17 04:31:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Raghavendra Talur 2020-10-15 17:22:45 UTC
Description of problem:
Heketi crashes during a brick evict operation if a volume exists in heketi db but not in gluster.

Version-Release number of selected component (if applicable):
9.0.0-9.5

How reproducible:
Always

Steps to Reproduce:
1. Create a volume using heketi
2. Delete the volume using gluster cli
3. Run "heketi-cli brick evict <brick-id-of-any-brick-of-the-volume>

Actual results:
Heketi crashes

Expected results:
Heketi should not crash

Additional info:
[cmdexec] DEBUG 2020/10/15 14:45:07 heketi/executors/cmdexec/volume.go:287:cmdexec.(*CmdExecutor).VolumeInfo: {OpRet:-1 OpErrno:30806 OpErrStr:Volume does not exist VolInfo:{XMLName:{Space: Local:volInfo} Volumes:{XMLName:{Space: Local:volumes} Count:0 VolumeList:[]}}}
panic: runtime error: index out of range

goroutine 11786 [running]:
github.com/heketi/heketi/executors/cmdexec.(*CmdExecutor).VolumeInfo(0xc42026ea20, 0xc42133c810, 0x2e, 0xc42133cde0, 0x24, 0x0, 0x0, 0x0)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/executors/cmdexec/volume.go:288 +0x627
github.com/heketi/heketi/apps/glusterfs.(*VolumeEntry).getBrickSetForBrickId(0xc421035a20, 0x19560a0, 0xc4203d8960, 0x1980980, 0xc42026ea20, 0xc421032440, 0x20, 0xc42133c810, 0x2e, 0x2460600, ...)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/apps/glusterfs/volume_entry_allocate.go:98 +0xa6
github.com/heketi/heketi/apps/glusterfs.(*BrickEvictOperation).execGetReplacmentInfo(0xc42019bb00, 0x1980980, 0xc42026ea20, 0x3, 0x3)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/apps/glusterfs/operations_device.go:514 +0x1ad
github.com/heketi/heketi/apps/glusterfs.(*BrickEvictOperation).Exec(0xc42019bb00, 0x1980980, 0xc42026ea20, 0xc420643e50, 0x3)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/apps/glusterfs/operations_device.go:591 +0x43
github.com/heketi/heketi/apps/glusterfs.runOperationAfterBuild(0x1972260, 0xc42019bb00, 0x1980980, 0xc42026ea20, 0x1, 0x1)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/apps/glusterfs/operations_manage.go:161 +0x1f7
github.com/heketi/heketi/apps/glusterfs.AsyncHttpOperation.func1(0x0, 0x0, 0x0, 0x0)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/apps/glusterfs/operations_manage.go:232 +0x16a
github.com/heketi/heketi/server/rest.(*AsyncHttpHandler).handle.func1(0xc421328a40, 0xc421321260)
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/server/rest/asynchttp.go:286 +0xe9
created by github.com/heketi/heketi/server/rest.(*AsyncHttpHandler).handle
	/builddir/build/BUILD/heketi-9.0.0/src/github.com/heketi/heketi/server/rest/asynchttp.go:282 +0x49

Comment 9 errata-xmlrpc 2020-12-17 04:31:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Storage 3.11.z bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:5602