This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1295431 - Heketi on RHEL6: Device add operation crashes heketi server
Heketi on RHEL6: Device add operation crashes heketi server
Status: CLOSED NOTABUG
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: heketi (Show other bugs)
3.1
Unspecified Unspecified
unspecified Severity high
: ---
: ---
Assigned To: Luis Pabón
Anush Shetty
: ZStream
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-04 08:22 EST by Anush Shetty
Modified: 2016-11-08 17:25 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-07 04:28:14 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Anush Shetty 2016-01-04 08:22:46 EST
Description of problem: While running Heketi on RHEL6, adding a device to a Heketi cluster crashes the Heketi server. 


Version-Release number of selected component (if applicable): heketi-1.0.2-1.el6rhs.x86_64


How reproducible: Always


Steps to Reproduce:
1. Create a cluster, add a node, and then try add a device to the node
2.
3.

Actual results:

# heketi-cli device add -name=/dev/vdb -node=78cc8391d136fbaafc45e8c0c15563d6
Error: Get http://localhost:8081/queue/b7964d57ce86a1b68c2fb662ff54fb18: read tcp 127.0.0.1:8081: connection reset by peer

Additional info:

From heketi log:

[heketi] INFO 2016/01/05 02:52:56 Adding device /dev/vdb to node 78cc8391d136fbaafc45e8c0c15563d6
[negroni] Completed 202 Accepted in 1.867598ms
[asynchttp] INFO 2016/01/05 02:52:56 Started job b7964d57ce86a1b68c2fb662ff54fb18
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x5ba2e0]

goroutine 8 [running]:
github.com/heketi/heketi/executors/sshexec.(*SshExecutor).accessConnection(0x0, 0xc208096ea0, 0x1f)
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/src/github.com/heketi/heketi/executors/sshexec/sshexec.go:97 +0x150
github.com/heketi/heketi/executors/sshexec.(*SshExecutor).sshExec(0x0, 0xc208096ea0, 0x1f, 0xc2080afd40, 0x2, 0x2, 0x5, 0x0, 0x0, 0x0, ...)
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/src/github.com/heketi/heketi/executors/sshexec/sshexec.go:118 +0x89
github.com/heketi/heketi/executors/sshexec.(*SshExecutor).DeviceSetup(0x0, 0xc208096ea0, 0x1f, 0xc2080a4a70, 0x8, 0xc208097320, 0x20, 0x0, 0x0, 0x0)
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/src/github.com/heketi/heketi/executors/sshexec/device.go:48 +0x470
github.com/heketi/heketi/apps/glusterfs.func·015(0x0, 0x0, 0x0, 0x0)
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/src/github.com/heketi/heketi/apps/glusterfs/app_device.go:72 +0x159
github.com/heketi/heketi/rest.func·001()
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/src/github.com/heketi/heketi/rest/asynchttp.go:128 +0x157
created by github.com/heketi/heketi/rest.(*AsyncHttpManager).AsyncHttpRedirectFunc
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/src/github.com/heketi/heketi/rest/asynchttp.go:138 +0xfb

goroutine 1 [select]:
main.main()
        /builddir/build/BUILD/heketi-6563551111f7178b679866e85a0682325929d037/main.go:153 +0x1463
Comment 2 Luis Pabón 2016-01-05 00:07:19 EST
I'll take a look.
Comment 3 Luis Pabón 2016-01-05 01:21:34 EST
This is really weird.  The panic points to https://github.com/heketi/heketi/blob/master/executors/sshexec/sshexec.go#L97 which cannot be null.  It really is starting to feel like a compiler bug, but before I continue, I will try to reproduce.  It looks like rhel6 is using golang 1.3.3.
Comment 13 Anush Shetty 2016-01-07 02:11:49 EST
I reinstalled my RHEL6 VM, and now I am unable to reproduce it either. Will keep this bug open for a while until I try it for a few more times.
Comment 14 Anush Shetty 2016-01-07 04:28:14 EST
I haven't been able to reproduce the issue on a fresh RHEL6 machine. Closing this bug.

Note You need to log in before you can comment on or make changes to this bug.