Bug 1709685

Summary: Geo-rep: Value of pending entry operations in detail status output is going up after each synchronization.
Product: [Community] GlusterFS Reporter: Kotresh HR <khiremat>
Component: geo-replicationAssignee: Kotresh HR <khiremat>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: 6CC: avishwan, bugs, khiremat, srangana, vnosov
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1512093 Environment:
Last Closed: 2019-05-17 07:48:35 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1512093    
Bug Blocks:    

Description Kotresh HR 2019-05-14 07:17:27 UTC
+++ This bug was initially created as a clone of Bug #1512093 +++

Description of problem:

Geo-replication is created, it is active, no errors is reported by CLI and at glusterfs logs. Start write files to master volume. Use geo-replication detail status command. Value of "Number of entry Operations pending" is growing after each synchronization. It never going down except after the replication is restarted.


Version-Release number of selected component (if applicable):

GlusterFS 3.12.2

How reproducible:
100%

Steps to Reproduce:

1. See the attached file "gluster_geo_repl_bug.txt".

Actual results:

Value of pending entries is not going down after synchronization.


Expected results:

In case of successful synchronization the value has to be "0".


Additional info:

1. See the attached glusterfs log files from the master system "gluster_logs_master_182.tgz".
2. See the attached glusterfs log files from the slave system "gluster_logs_slave_183.tgz".

--- Additional comment from vnosov on 2017-11-10 20:09 UTC ---



--- Additional comment from vnosov on 2017-11-10 20:10 UTC ---



--- Additional comment from Aravinda VK on 2017-11-27 06:10:58 UTC ---

IO from mount stopped while running status command?

Entry count in status changes continuously as long as IO is going on in Master volume. (Crawl -> Increment count -> Create entries in slave -> Decrement the count)

This status column is not related to checkpoint. Only if no IO in Master mount and status shows entry count then it might have some issue.

--- Additional comment from vnosov on 2018-09-13 16:20:16 UTC ---

Problem is reproduced with GlusterFS 3.12.13.

--- Additional comment from Shyamsundar on 2018-10-23 14:54:08 UTC ---

Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.

--- Additional comment from Yaniv Kaul on 2019-04-17 10:00:18 UTC ---

(In reply to Shyamsundar from comment #5)
> Release 3.12 has been EOLd and this bug was still found to be in the NEW
> state, hence moving the version to mainline, to triage the same and take
> appropriate actions.

Status?

--- Additional comment from Shyamsundar on 2019-04-17 10:12:48 UTC ---

(In reply to Yaniv Kaul from comment #6)
> (In reply to Shyamsundar from comment #5)
> > Release 3.12 has been EOLd and this bug was still found to be in the NEW
> > state, hence moving the version to mainline, to triage the same and take
> > appropriate actions.
> 
> Status?

Will need to check with the assignee or component maintainer, which is Kotresh in both cases. @Kotresh request an update here? Thanks.

--- Additional comment from Worker Ant on 2019-04-23 05:21:26 UTC ---

REVIEW: https://review.gluster.org/22603 (geo-rep: Fix entries and metadata counters in geo-rep status) posted (#1) for review on master by Kotresh HR

--- Additional comment from Worker Ant on 2019-04-24 16:18:18 UTC ---

REVIEW: https://review.gluster.org/22603 (geo-rep: Fix entries and metadata counters in geo-rep status) merged (#2) on master by Amar Tumballi

Comment 1 Worker Ant 2019-05-14 07:20:26 UTC
REVIEW: https://review.gluster.org/22722 (geo-rep: Fix entries and metadata counters in geo-rep status) posted (#1) for review on release-6 by Kotresh HR

Comment 2 Worker Ant 2019-05-17 07:48:35 UTC
REVIEW: https://review.gluster.org/22722 (geo-rep: Fix entries and metadata counters in geo-rep status) merged (#3) on release-6 by Amar Tumballi