Bug 1753857 - geo-rep: performance improvement while syncing heavy renames with existing destination
Summary: geo-rep: performance improvement while syncing heavy renames with existing de...
Keywords:
Status: CLOSED NEXTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: geo-replication
Version: mainline
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
Assignee: Sunny Kumar
QA Contact:
URL:
Whiteboard:
Depends On: 1726000
Blocks: 1696809 1755212
TreeView+ depends on / blocked
 
Reported: 2019-09-20 05:16 UTC by Sunny Kumar
Modified: 2019-10-22 06:42 UTC (History)
9 users (show)

Fixed In Version:
Clone Of: 1726000
: 1755212 (view as bug list)
Environment:
Last Closed: 2019-09-23 12:07:40 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Gluster.org Gerrit 23459 0 None Merged geo-rep: performance improvement while syncing renames with existing gfid 2019-09-23 12:07:39 UTC

Description Sunny Kumar 2019-09-20 05:16:54 UTC
+++ This bug was initially created as a clone of Bug #1726000 +++

Description of problem:

The bz#1694820
addresses issue for RENAME issue when handling RENAME with existing destination. This fix requires some performance tuning considering this issue occurs in heavy rename workload.
 
Version-Release number of selected component (if applicable):
master

How reproducible:


Steps to Reproduce:

1. Run frequent RENAME on master mount and check for sync in slave.

Ex - while true; do uuid="`uuidgen`"; echo "some data" > "test$uuid"; mv "test$uuid" "test" -f; done 

Actual results:


Expected results:


Additional info:

Comment 1 Worker Ant 2019-09-20 05:30:34 UTC
REVIEW: https://review.gluster.org/23459 (geo-rep: performance improvement while syncing renames with existing gfid) posted (#1) for review on master by Sunny Kumar

Comment 2 Worker Ant 2019-09-23 12:07:40 UTC
REVIEW: https://review.gluster.org/23459 (geo-rep: performance improvement while syncing renames with existing gfid) merged (#2) on master by Kotresh HR

Comment 3 Sunny Kumar 2019-10-22 06:42:10 UTC
*********************
Performance Stat:

For this bug we have performed a performance run with arbiter and replica volumes with root geo-rep sessions.

Setup details:
- Number of Master servers: 3
- Number of slave servers: 3
- Brick size: 50GB
- RAM: 8GB
- Number of clients used: 1 
- Type of volume used:
  1. arbiter 1x(2+1)
  2. replica 1x3
- Workload used: for i in `seq 1 1000000`;do uuid="`uuidgen`"; date > "test$uuid"; mv "test$uuid" "test" -f; done

The results are as follows:

1. With sync type as rsync:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Volume type             Configuration         number of creates and renames done             time taken without fix                     time taken with fix
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Arbiter                  1x(2+1)                           1000000                      14 hours, 22 minutes, 56 seconds              2 hours, 4 minutes, 6 seconds
Replicate                 1x3                              1000000                      20 hours, 13 minutes, 14 seconds              8 hours, 27 minutes, 2 seconds

2. With sync type as tarssh:

---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Volume type             Configuration         number of creates and renames done             time taken without fix                     time taken with fix
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Arbiter                  1x(2+1)                           1000000                      20 hours, 22 minutes, 56 seconds              8 hours, 34 minutes, 20 seconds
Replicate                 1x3                              1000000                      22 hours, 49 minutes, 29 seconds              10 hours, 7 minutes, 43 seconds


Note You need to log in before you can comment on or make changes to this bug.