Bug 831630 - GlusterFS: Volumes sometimes fail to start
GlusterFS: Volumes sometimes fail to start
Status: CLOSED NOTABUG
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterfs (Show other bugs)
unspecified
Unspecified Unspecified
low Severity high
: ---
: ---
Assigned To: Shireesh
Sudhir D
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-06-13 09:24 EDT by Daniel Paikov
Modified: 2013-07-04 03:57 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-11-16 04:24:12 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
glusterfs log (9.87 KB, application/x-compressed-tar)
2012-06-13 09:24 EDT, Daniel Paikov
no flags Details
vdsm.log (329.07 KB, application/x-compressed-tar)
2012-06-13 09:25 EDT, Daniel Paikov
no flags Details

  None (edit)
Description Daniel Paikov 2012-06-13 09:24:58 EDT
Created attachment 591499 [details]
glusterfs log

When starting several volumes one after the other, one of the volumes failed to start. The same volume could be successfully started later on.

From vdsm.log:
MainProcess|Thread-102376::DEBUG::2012-06-13 12:34:04,463::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/sbin/gluster --mode=script volume start vol3-distrep' (cwd None)
MainProcess|Thread-102376::DEBUG::2012-06-13 12:34:04,575::__init__::1164::Storage.Misc.excCmd::(_log) FAILED: <err> = 'operation failed\n'; <rc> = 255
Thread-102376::DEBUG::2012-06-13 12:34:04,577::BindingXMLRPC::880::vds::(wrapper) return volumeStart with {'status': {'message': 'Volume start failed\nerror: operation failed\nreturn code: 255', 'code': 4125, 'rc': 255, 'err': ['operation failed'], 'out': []}}
MainProcess|Thread-102375::DEBUG::2012-06-13 12:34:04,948::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
Thread-102375::DEBUG::2012-06-13 12:34:04,949::BindingXMLRPC::880::vds::(wrapper) return volumeStart with {'status': {'message': 'Done', 'code': 0}}

From gluster log:
[2012-06-13 12:34:04.566422] I [glusterd-volume-ops.c:290:glusterd_handle_cli_start_volume] 0-glusterd: Received start vol reqfor volume vol3-distrep
[2012-06-13 12:34:04.566505] E [glusterd-utils.c:277:glusterd_lock] 0-glusterd: Unable to get lock for uuid: 876aef78-4863-470c-b9d1-ac912993c392, lock held by: 876aef78-4863-470c-b9d1-ac912993c392
[2012-06-13 12:34:04.566525] E [glusterd-handler.c:453:glusterd_op_txn_begin] 0-management: Unable to acquire local lock, ret: -1
[2012-06-13 12:34:04.944327] I [glusterd-rpc-ops.c:1316:glusterd3_1_commit_op_cbk] 0-glusterd: Received ACC from uuid: 9433c12e-2342-4663-9571-5587fa55300e
[2012-06-13 12:34:04.944397] I [glusterd-op-sm.c:2254:glusterd_op_modify_op_ctx] 0-management: op_ctx modification not required
[2012-06-13 12:34:04.946053] I [glusterd-rpc-ops.c:606:glusterd3_1_cluster_unlock_cbk] 0-glusterd: Received ACC from uuid: 9433c12e-2342-4663-9571-5587fa55300e
[2012-06-13 12:34:04.946088] I [glusterd-op-sm.c:2653:glusterd_op_txn_complete] 0-glusterd: Cleared local lock
Comment 1 Daniel Paikov 2012-06-13 09:25:20 EDT
Created attachment 591500 [details]
vdsm.log
Comment 3 Selvasundaram 2012-06-15 02:34:21 EDT
It seems to be GlusterFS bug.
Comment 4 Amar Tumballi 2012-09-18 01:04:12 EDT
handling multiple commands with gluster CLI is not supported at the moment, hence starting multiple volumes simultaneously failed.
Comment 5 Shireesh 2012-10-08 05:17:26 EDT
Daniel, Amar's statement conflicts with your statement that you were trying to start the volumes 'one after the other'. Can you please confirm whether you had tried to "select multiple volumes in the tab and start all of them at once" or "start one volume at a time, in a sequence"?
Comment 6 Daniel Paikov 2012-10-09 04:05:27 EDT
(In reply to comment #5)
> Daniel, Amar's statement conflicts with your statement that you were trying
> to start the volumes 'one after the other'. Can you please confirm whether
> you had tried to "select multiple volumes in the tab and start all of them
> at once" or "start one volume at a time, in a sequence"?

If I remember correctly, I started the volumes one volume at a time, via the GUI.
Comment 7 Shireesh 2012-10-09 04:26:48 EDT
Amar, in that case, it looks like a different issue, as the commands were fired one after the other?
Comment 8 Shireesh 2012-11-16 04:24:12 EST
I'm closing this, assuming that it could be a corner case bug/limitation in glusterfs. RHS-C invokes the volume start verb on vdsm, which ultimately invokes the gluster cli command 'gluster volume start <volname>'. We can see clearly in the log that the command was invoked correctly, and the error came up from gluster cli. So it is not an issue with RHS-C. I feel gluster cli should provide more details on the error message than just 'operation failed'.

Note You need to log in before you can comment on or make changes to this bug.