Bug 1646806

Summary:	[Geo-rep]: Faulty geo-rep sessions due to link ownership on slave volume
Product:	[Community] GlusterFS	Reporter:	Kotresh HR <khiremat>
Component:	geo-replication	Assignee:	Kotresh HR <khiremat>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	high	Docs Contact:
Priority:	high
Version:	4.1	CC:	anrobins, avishwan, bugs, csaba, rhinduja, rhs-bugs, sankarshan, storage-qa-internal
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-4.1.6	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:	1646104	Environment:
Last Closed:	2018-11-28 06:19:15 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1645916, 1646104, 1654115
Bug Blocks:

Description Kotresh HR 2018-11-06 03:49:26 UTC

+++ This bug was initially created as a clone of Bug #1646104 +++

+++ This bug was initially created as a clone of Bug #1645916 +++

Description of problem:

Geo-rep session is Faulty with following tracebacks

Master Traceback:
[2018-11-05 07:12:37.31370] E [syncdutils(worker /bricks/brick0/b0):338:log_raise_exception] <top>: FAIL: 
Traceback (most recent call last):
  File "/usr/local/libexec/glusterfs/python/syncdaemon/gsyncd.py", line 322, in main
    func(args)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/subcmds.py", line 82, in subcmd_worker
    local.service_loop(remote)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/resource.py", line 1312, in service_loop
    till=e))
  File "/usr/local/libexec/glusterfs/python/syncdaemon/master.py", line 599, in crawlwrap
    self.crawl()
  File "/usr/local/libexec/glusterfs/python/syncdaemon/master.py", line 1555, in crawl
    self.changelogs_batch_process(changes)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/master.py", line 1455, in changelogs_batch_process
    self.process(batch)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/master.py", line 1290, in process
    self.process_change(change, done, retry)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/master.py", line 1237, in process_change
    failures = self.slave.server.meta_ops(meta_entries)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/repce.py", line 233, in __call__
    return self.ins(self.meth, *a)
  File "/usr/local/libexec/glusterfs/python/syncdaemon/repce.py", line 215, in __call__
    raise res
OSError: [Errno 30] Read-only file system: '.gfid/981277d0-bf2f-4a43-be9a-ec3eabf84955'



Slave Traceback:
[2018-11-05 07:12:37.30050] E [repce(slave f281/bricks/brick0/b0):122:worker] <top>: call failed: 
Traceback (most recent call last):
  File "/usr/local/libexec/glusterfs/python/syncdaemon/repce.py", line 118, in worker
    res = getattr(self.obj, rmeth)(*in_data[2:])
  File "/usr/local/libexec/glusterfs/python/syncdaemon/resource.py", line 765, in meta_ops
    cmd_ret = errno_wrap(os.chmod, [go, mode],
  File "/usr/local/libexec/glusterfs/python/syncdaemon/syncdutils.py", line 546, in errno_wrap
    return call(*arg)
OSError: [Errno 30] Read-only file system: '.gfid/981277d0-bf2f-4a43-be9a-ec3eabf84955'


Version-Release number of selected component (if applicable):
mainline


How reproducible:
Always




Steps to Reproduce:
Use geo-replication to copy symbolic links with either the group or ownership something other than root.

Actual results:
Geo-rep sessions with a status of "Faulty"


Expected results:
Completed geo-rep sessions.

--- Additional comment from Worker Ant on 2018-11-05 04:13:25 EST ---

REVIEW: https://review.gluster.org/21546 (geo-rep: Fix traceback with symlink metadata sync) posted (#1) for review on master by Kotresh HR

--- Additional comment from Worker Ant on 2018-11-05 21:23:09 EST ---

REVIEW: https://review.gluster.org/21546 (geo-rep: Fix traceback with symlink metadata sync) posted (#2) for review on master by Atin Mukherjee

Comment 1 Worker Ant 2018-11-06 04:13:15 UTC

REVIEW: https://review.gluster.org/21565 (geo-rep: Fix traceback with symlink metadata sync) posted (#1) for review on release-4.1 by Kotresh HR

Comment 2 Worker Ant 2018-11-12 15:41:26 UTC

REVIEW: https://review.gluster.org/21565 (geo-rep: Fix traceback with symlink metadata sync) posted (#2) for review on release-4.1 by Shyamsundar Ranganathan

Comment 3 Kotresh HR 2018-11-28 06:19:15 UTC

Will be part of 4.1.7

Comment 4 Shyamsundar 2018-11-29 15:26:23 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-4.1.6, please open a new bug report.

glusterfs-4.1.6 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://lists.gluster.org/pipermail/announce/2018-November/000116.html
[2] https://www.gluster.org/pipermail/gluster-users/