Bug 1512093

Summary: Value of pending entry operations in detail status output is going up after each synchronization.
Product: [Community] GlusterFS Reporter: vnosov <vnosov>
Component: geo-replicationAssignee: Kotresh HR <khiremat>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: high Docs Contact:
Priority: unspecified    
Version: mainlineCC: avishwan, bugs, khiremat, srangana, vnosov
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1709685 (view as bug list) Environment:
Last Closed: 2019-04-24 16:18:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1709685    
Attachments:
Description Flags
Step-by-step procedure to reproduce the bug.
none
GlusterFS logs from the master system.
none
GlusterFS logs from the slave system none

Description vnosov 2017-11-10 20:07:59 UTC
Created attachment 1350651 [details]
Step-by-step procedure to reproduce the bug.

Description of problem:

Geo-replication is created, it is active, no errors is reported by CLI and at glusterfs logs. Start write files to master volume. Use geo-replication detail status command. Value of "Number of entry Operations pending" is growing after each synchronization. It never going down except after the replication is restarted.


Version-Release number of selected component (if applicable):

GlusterFS 3.12.2

How reproducible:
100%

Steps to Reproduce:

1. See the attached file "gluster_geo_repl_bug.txt".

Actual results:

Value of pending entries is not going down after synchronization.


Expected results:

In case of successful synchronization the value has to be "0".


Additional info:

1. See the attached glusterfs log files from the master system "gluster_logs_master_182.tgz".
2. See the attached glusterfs log files from the slave system "gluster_logs_slave_183.tgz".

Comment 1 vnosov 2017-11-10 20:09:26 UTC
Created attachment 1350652 [details]
GlusterFS logs from the master system.

Comment 2 vnosov 2017-11-10 20:10:10 UTC
Created attachment 1350653 [details]
GlusterFS logs from the slave system

Comment 3 Aravinda VK 2017-11-27 06:10:58 UTC
IO from mount stopped while running status command?

Entry count in status changes continuously as long as IO is going on in Master volume. (Crawl -> Increment count -> Create entries in slave -> Decrement the count)

This status column is not related to checkpoint. Only if no IO in Master mount and status shows entry count then it might have some issue.

Comment 4 vnosov 2018-09-13 16:20:16 UTC
Problem is reproduced with GlusterFS 3.12.13.

Comment 5 Shyamsundar 2018-10-23 14:54:08 UTC
Release 3.12 has been EOLd and this bug was still found to be in the NEW state, hence moving the version to mainline, to triage the same and take appropriate actions.

Comment 6 Yaniv Kaul 2019-04-17 10:00:18 UTC
(In reply to Shyamsundar from comment #5)
> Release 3.12 has been EOLd and this bug was still found to be in the NEW
> state, hence moving the version to mainline, to triage the same and take
> appropriate actions.

Status?

Comment 7 Shyamsundar 2019-04-17 10:12:48 UTC
(In reply to Yaniv Kaul from comment #6)
> (In reply to Shyamsundar from comment #5)
> > Release 3.12 has been EOLd and this bug was still found to be in the NEW
> > state, hence moving the version to mainline, to triage the same and take
> > appropriate actions.
> 
> Status?

Will need to check with the assignee or component maintainer, which is Kotresh in both cases. @Kotresh request an update here? Thanks.

Comment 8 Worker Ant 2019-04-23 05:21:26 UTC
REVIEW: https://review.gluster.org/22603 (geo-rep: Fix entries and metadata counters in geo-rep status) posted (#1) for review on master by Kotresh HR

Comment 9 Worker Ant 2019-04-24 16:18:18 UTC
REVIEW: https://review.gluster.org/22603 (geo-rep: Fix entries and metadata counters in geo-rep status) merged (#2) on master by Amar Tumballi