Bug 913662 - Need to increase throughput of mgmt operations using synctask framework
Need to increase throughput of mgmt operations using synctask framework
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: glusterd (Show other bugs)
mainline
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: krishnan parthasarathi
:
Depends On:
Blocks: 895528
  Show dependency treegraph
 
Reported: 2013-02-21 13:21 EST by krishnan parthasarathi
Modified: 2015-11-03 18:05 EST (History)
5 users (show)

See Also:
Fixed In Version: glusterfs-3.4.0
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-07-24 13:18:18 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description krishnan parthasarathi 2013-02-21 13:21:14 EST
Description of problem:
Volume operations using synctask framework issue management rpc ops in a serial manner. This approach will not scale with increasing no. of peers in the cluster.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:
Comment 1 Vijay Bellur 2013-02-22 01:57:19 EST
CHANGE: http://review.gluster.org/4558 (synctask: support for (assymetric) counted barriers) merged in master by Anand Avati (avati@redhat.com)
Comment 2 Vijay Bellur 2013-02-26 12:04:49 EST
CHANGE: http://review.gluster.org/4580 (volgen: Use bind-address option for bricks when option set on glusterd) merged in master by Vijay Bellur (vbellur@redhat.com)
Comment 3 Vijay Bellur 2013-02-26 12:06:27 EST
CHANGE: http://review.gluster.org/4570 (glusterd: Increasing throughput of synctask based mgmt ops.) merged in master by Vijay Bellur (vbellur@redhat.com)
Comment 4 Ben Turner 2013-02-28 16:06:46 EST
I am seeing volume commands fail occasionally on my 6 node 3x2 setups.  Here is what I am seeing:

# gluster volume heal healtest info split-brain
operation failed
# echo $?
255

This happened at just about 20:32:07.  In the etc-glusterfs-glusterd.vol.log I see:

[2013-02-27 20:32:08.081243] E [glusterd-utils.c:278:glusterd_lock] 0-glusterd: Unable to get lock for uuid: c3538fdf-c16e-4eaf-9650-6cf0caf7478b, lock held by: 647b8ff3-8110-4eab-956b-df78235fb192
[2013-02-27 20:32:08.081254] E [glusterd-handler.c:470:glusterd_op_txn_begin] 0-management: Unable to acquire local lock, ret: -1

Is this a symptom of what this bug was opened on or is this a different issue?
Comment 5 krishnan parthasarathi 2013-02-28 23:45:16 EST
Ben,
This bug was opened to track the code changes for how (internal rpc) requests were sent to peers in cluster, during a volume operation.
The above observation is not a symptom of this bug. While you open a new bug to track what you observe, please provide the output of "gluster system:: fsm log". Run the command on the machine whose uuid is (still) holding the lock.
Comment 6 Vijay Bellur 2013-03-07 00:32:54 EST
CHANGE: http://review.gluster.org/4636 (synctask: support for (assymetric) counted barriers) merged in release-3.4 by Anand Avati (avati@redhat.com)
Comment 7 Vijay Bellur 2013-03-07 01:05:54 EST
CHANGE: http://review.gluster.org/4637 (volgen: Use bind-address option for bricks when option set on glusterd) merged in release-3.4 by Anand Avati (avati@redhat.com)

Note You need to log in before you can comment on or make changes to this bug.