+++ This bug was initially created as a clone of Bug #1771577 +++
+++ This bug was initially created as a clone of Bug #1771524 +++
Description of problem:
On a 6 node master and slave cluster, created and started one volume of each type i.e. replica(1x3),distributed-replicated(2x3),arbiter, distributed-disperse(2x4+2), disperse(1x4+2). After which mounted the volumes from the master cluster to a client and started IO. Once a good amount of data was written to the master volumes created a geo-rep session and tried to start it.
[root@dhcp43-73 ~]# gluster v geo-rep masterarb 10.70.42.167::slavearb start
geo-replication start failed for masterarb 10.70.42.167::slavearb
geo-replication command failed
Version-Release number of selected component (if applicable):
glusterfs-6.0-21(Observed only on el6 builds.)
How reproducible:
1/1
Steps to Reproduce:
1.Create a 6 node master ans slave clusters.
2.Create one volume of each type on both clusters.
3.Create a geo-rep session between the master volumes and the slave volumes.
4.Start the geo-rep sessions.
Actual results:
geo-replication session not starting.
Expected results:
geo-replication session should start without any errors.
Additional info:
################################################################################
glusterd.log
################################################################################
[2019-11-12 12:52:48.640582] E [MSGID: 106122] [glusterd-syncop.c:1445:gd_commit_op_phase] 0-management: Commit of operation 'Volume Geo-replication' failed on localhost : geo-replication start failed for masterarb 10.70.42.167::slavearb
[2019-11-12 12:52:55.847310] I [MSGID: 106327] [glusterd-geo-rep.c:4653:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:53:53.530089] I [MSGID: 106327] [glusterd-geo-rep.c:4653:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:54:00.124828] I [MSGID: 106327] [glusterd-geo-rep.c:2686:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:54:02.465503] E [MSGID: 106122] [glusterd-syncop.c:1445:gd_commit_op_phase] 0-management: Commit of operation 'Volume Geo-replication' failed on localhost : geo-replication start failed for masterarb 10.70.42.167::slavearb
[2019-11-12 12:54:34.413867] I [MSGID: 106327] [glusterd-geo-rep.c:4653:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:56:38.175956] I [MSGID: 106327] [glusterd-geo-rep.c:4653:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:56:43.557067] I [MSGID: 106499] [glusterd-handler.c:4502:__glusterd_handle_status_volume] 0-management: Received status volume req for volume gluster_shared_storage
[2019-11-12 12:56:43.584443] I [MSGID: 106499] [glusterd-handler.c:4502:__glusterd_handle_status_volume] 0-management: Received status volume req for volume masterarb
[2019-11-12 12:56:43.626712] I [MSGID: 106499] [glusterd-handler.c:4502:__glusterd_handle_status_volume] 0-management: Received status volume req for volume masterdip
[2019-11-12 12:56:43.653174] I [MSGID: 106499] [glusterd-handler.c:4502:__glusterd_handle_status_volume] 0-management: Received status volume req for volume masterdistdip
[2019-11-12 12:56:43.682468] I [MSGID: 106499] [glusterd-handler.c:4502:__glusterd_handle_status_volume] 0-management: Received status volume req for volume masterdistrep
[2019-11-12 12:56:43.710298] I [MSGID: 106499] [glusterd-handler.c:4502:__glusterd_handle_status_volume] 0-management: Received status volume req for volume masterrep
[2019-11-12 12:57:02.362344] I [MSGID: 106327] [glusterd-geo-rep.c:4653:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:57:09.915409] I [MSGID: 106327] [glusterd-geo-rep.c:2686:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:57:12.182356] E [MSGID: 106122] [glusterd-syncop.c:1445:gd_commit_op_phase] 0-management: Commit of operation 'Volume Geo-replication' failed on localhost : geo-replication start failed for masterarb 10.70.42.167::slavearb
[2019-11-12 12:58:32.749951] I [MSGID: 106327] [glusterd-geo-rep.c:4653:glusterd_read_status_file] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:58:39.308268] I [MSGID: 106327] [glusterd-geo-rep.c:2686:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
[2019-11-12 12:58:51.626331] E [MSGID: 106122] [glusterd-syncop.c:1445:gd_commit_op_phase] 0-management: Commit of operation 'Volume Geo-replication' failed on localhost : geo-replication start failed for masterarb 10.70.42.167::slavearb
[2019-11-12 12:58:49.404793] I [MSGID: 106327] [glusterd-geo-rep.c:2686:glusterd_get_statefile_name] 0-management: Using passed config template(/var/lib/glusterd/geo-replication/masterarb_10.70.42.167_slavearb/gsyncd.conf).
################################################################################
--- Additional comment from Worker Ant on 2019-11-12 16:31:47 UTC ---
REVIEW: https://review.gluster.org/23702 (geo-rep: Fix py2/py3 compatibility in repce) posted (#1) for review on master by Kotresh HR
--- Additional comment from Worker Ant on 2019-11-13 04:14:49 UTC ---
REVIEW: https://review.gluster.org/23702 (geo-rep: Fix py2/py3 compatibility in repce) merged (#1) on master by Kotresh HR