Bug 1276028

Summary:	[RFE] Geo-replication support for Volumes running in docker containers
Product:	[Community] GlusterFS	Reporter:	Aravinda VK <avishwan>
Component:	geo-replication	Assignee:	Aravinda VK <avishwan>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	mainline	CC:	bugs, chrisw, csaba, nlevinki, rhs-bugs, storage-qa-internal
Target Milestone:	---	Keywords:	FutureFeature
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	glusterfs-3.8rc2	Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:	1275971
Clones:	1283060 (view as bug list)		Environment:
Last Closed:	2016-06-16 13:41:47 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1275971
Bug Blocks:	1283060

Description Aravinda VK 2015-10-28 12:33:01 UTC

+++ This bug was initially created as a clone of Bug #1275971 +++

Description of problem:
When NET=HOST is used, Unable to create Geo-replication session.

Setup:
------
CoreOS + Centos containers
RHGS 3.1 RPMs installed inside Containers.
Net=Host setup
4 containers, 2 containers for Master and 2 containers for Slave.(One brick in each container)
Replica Volumes - Master and Slave

Issue:
--------
- Unable to create the session. Port mapping is used for ssh(Custom port is mapped to port 22). Geo-replication all SSH commands are executed without port option to ssh.
- After creating session Rsync was failing due to validation in gsyncd. (gsyncd is used as shell instead of bash)


Workaround:
-----------
- Kotresh modified gverify.sh and hook script to use custom SSH port instead of default. (https://gist.github.com/kotreshhr/dd16c5fca425b417c097)
- Geo-rep config options to use ssh options runtime.
gluster vol geo-rep<master vol> <slavehost>::<slavevol>  config ssh_command_tar "ssh -p 50002 -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/tar_ssh.pem"
gluster vol geo-rep<master vol> <slavehost>::<slavevol>  config ssh_command "ssh -p 50002 -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem"
- Suggestion to remove "command=" from authorized_keys files of Slave nodes to prevent executing all commands via gsyncd shell.
- Replaced /nonexistent/gsyncd with actual path of gsyncd(/usr/libexec/glusterfs/gsyncd) in Geo-replication session config file.

Comment 1 Vijay Bellur 2015-10-28 12:35:22 UTC

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#1) for review on master by Aravinda VK (avishwan)

Comment 2 Vijay Bellur 2015-10-29 06:14:12 UTC

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#2) for review on master by Aravinda VK (avishwan)

Comment 3 Vijay Bellur 2015-10-29 09:39:15 UTC

REVIEW: http://review.gluster.org/12459 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on master by Kotresh HR (khiremat)

Comment 4 Vijay Bellur 2015-10-30 10:44:31 UTC

REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#1) for review on master by Aravinda VK (avishwan)

Comment 5 Vijay Bellur 2015-11-02 06:26:52 UTC

REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#2) for review on master by Aravinda VK (avishwan)

Comment 6 Vijay Bellur 2015-11-02 06:27:15 UTC

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#3) for review on master by Aravinda VK (avishwan)

Comment 7 Vijay Bellur 2015-11-17 05:45:31 UTC

REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#3) for review on master by Aravinda VK (avishwan)

Comment 8 Vijay Bellur 2015-11-17 05:45:39 UTC

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#4) for review on master by Aravinda VK (avishwan)

Comment 9 Vijay Bellur 2015-11-17 15:00:11 UTC

COMMIT: http://review.gluster.org/12444 committed in master by Jeff Darcy (jdarcy) 
------
commit 7d35eb5926869ed084295600502a85ce13be506f
Author: Aravinda VK <avishwan>
Date:   Wed Oct 28 17:56:50 2015 +0530

    geo-rep: New Config option for ssh_port
    
    If different port used for SSH instead of 22, Geo-replication
    was failing to establish SSH connection.
    
    ssh_port option can be added using config:ssh_command and
    config:ssh_command_tar, but user has to remember complete
    ssh command used with parameter to add/modify ssh port.
    
    This patch adds new config option for ssh_port,
    
    gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \
            config ssh_port 52022
    
    Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4
    Signed-off-by: Aravinda VK <avishwan>
    BUG: 1276028
    Reviewed-on: http://review.gluster.org/12444
    Tested-by: NetBSD Build System <jenkins.org>
    Reviewed-by: Saravanakumar Arumugam <sarumuga>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Venky Shankar <vshankar>

Comment 10 Vijay Bellur 2015-11-17 15:06:26 UTC

COMMIT: http://review.gluster.org/12472 committed in master by Venky Shankar (vshankar) 
------
commit 7de355b42dc1f8313db3ffc775a0e1708ba85243
Author: Aravinda VK <avishwan>
Date:   Fri Oct 30 16:09:29 2015 +0530

    geo-rep: Allow setting config remote_gsyncd
    
    Restrictive ssh is not used in containerized environment
    where networking configuration is "net=host". SSH Pem keys
    pushed to the slave without gsyncd path in it. (Patch #12459)
    
    Actual remote_gsyncd path need to be set to actual path of gsyncd.
    With this patch, remote_gsyncd is removed from reserved option list.
    
    Change-Id: Ia2063e4654e378b62b2414bdad21143c86ad1b9a
    Signed-off-by: Aravinda VK <avishwan>
    BUG: 1276028
    Reviewed-on: http://review.gluster.org/12472
    Tested-by: NetBSD Build System <jenkins.org>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Saravanakumar Arumugam <sarumuga>
    Reviewed-by: Venky Shankar <vshankar>

Comment 11 Vijay Bellur 2015-11-18 19:33:46 UTC

COMMIT: http://review.gluster.org/12504 committed in master by Jeff Darcy (jdarcy) 
------
commit 5bb3c521431cc27b2826acd889bffb2f90ae7f73
Author: Kotresh HR <khiremat>
Date:   Mon Nov 2 18:52:03 2015 +0530

    glusterd/geo-rep: Adding ssh-port option for geo-rep create
    
    Geo-replication uses default ssh port 22 for setup.
    i.e., to distribute ssh keys to slaves. In container
    environments, custom port number might be used.
    Hence to support custom port number for ssh, option
    is provided in geo-rep create command to take the
    same.
    
    Change-Id: I0fb61959b1c085342b8e4c21ac4e076fba5462f1
    BUG: 1276028
    Signed-off-by: Kotresh HR <khiremat>
    Reviewed-on: http://review.gluster.org/12504
    Tested-by: NetBSD Build System <jenkins.org>
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Avra Sengupta <asengupt>
    Reviewed-by: Aravinda VK <avishwan>
    Reviewed-by: Venky Shankar <vshankar>

Comment 12 Niels de Vos 2016-06-16 13:41:47 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user