Bug 1283060 - [RFE] Geo-replication support for Volumes running in docker containers
Summary: [RFE] Geo-replication support for Volumes running in docker containers
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: GlusterFS
Classification: Community
Component: geo-replication
Version: 3.7.6
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Aravinda VK
QA Contact:
URL:
Whiteboard:
Depends On: 1275971 1276028
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-11-18 07:07 UTC by Aravinda VK
Modified: 2016-04-19 07:48 UTC (History)
6 users (show)

Fixed In Version: glusterfs-3.7.7
Doc Type: Enhancement
Doc Text:
Clone Of: 1276028
Environment:
Last Closed: 2016-02-15 06:28:46 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)

Description Aravinda VK 2015-11-18 07:07:29 UTC
+++ This bug was initially created as a clone of Bug #1276028 +++

+++ This bug was initially created as a clone of Bug #1275971 +++

Description of problem:
When NET=HOST is used, Unable to create Geo-replication session.

Setup:
------
CoreOS + Centos containers
RHGS 3.1 RPMs installed inside Containers.
Net=Host setup
4 containers, 2 containers for Master and 2 containers for Slave.(One brick in each container)
Replica Volumes - Master and Slave

Issue:
--------
- Unable to create the session. Port mapping is used for ssh(Custom port is mapped to port 22). Geo-replication all SSH commands are executed without port option to ssh.
- After creating session Rsync was failing due to validation in gsyncd. (gsyncd is used as shell instead of bash)


Workaround:
-----------
- Kotresh modified gverify.sh and hook script to use custom SSH port instead of default. (https://gist.github.com/kotreshhr/dd16c5fca425b417c097)
- Geo-rep config options to use ssh options runtime.
gluster vol geo-rep<master vol> <slavehost>::<slavevol>  config ssh_command_tar "ssh -p 50002 -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/tar_ssh.pem"
gluster vol geo-rep<master vol> <slavehost>::<slavevol>  config ssh_command "ssh -p 50002 -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem"
- Suggestion to remove "command=" from authorized_keys files of Slave nodes to prevent executing all commands via gsyncd shell.
- Replaced /nonexistent/gsyncd with actual path of gsyncd(/usr/libexec/glusterfs/gsyncd) in Geo-replication session config file.

--- Additional comment from Vijay Bellur on 2015-10-28 08:35:22 EDT ---

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#1) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-29 02:14:12 EDT ---

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#2) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-29 05:39:15 EDT ---

REVIEW: http://review.gluster.org/12459 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on master by Kotresh HR (khiremat@redhat.com)

--- Additional comment from Vijay Bellur on 2015-10-30 06:44:31 EDT ---

REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#1) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-11-02 01:26:52 EST ---

REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#2) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-11-02 01:27:15 EST ---

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#3) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-11-17 00:45:31 EST ---

REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#3) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-11-17 00:45:39 EST ---

REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#4) for review on master by Aravinda VK (avishwan@redhat.com)

--- Additional comment from Vijay Bellur on 2015-11-17 10:00:11 EST ---

COMMIT: http://review.gluster.org/12444 committed in master by Jeff Darcy (jdarcy@redhat.com) 
------
commit 7d35eb5926869ed084295600502a85ce13be506f
Author: Aravinda VK <avishwan@redhat.com>
Date:   Wed Oct 28 17:56:50 2015 +0530

    geo-rep: New Config option for ssh_port
    
    If different port used for SSH instead of 22, Geo-replication
    was failing to establish SSH connection.
    
    ssh_port option can be added using config:ssh_command and
    config:ssh_command_tar, but user has to remember complete
    ssh command used with parameter to add/modify ssh port.
    
    This patch adds new config option for ssh_port,
    
    gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \
            config ssh_port 52022
    
    Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4
    Signed-off-by: Aravinda VK <avishwan@redhat.com>
    BUG: 1276028
    Reviewed-on: http://review.gluster.org/12444
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>

--- Additional comment from Vijay Bellur on 2015-11-17 10:06:26 EST ---

COMMIT: http://review.gluster.org/12472 committed in master by Venky Shankar (vshankar@redhat.com) 
------
commit 7de355b42dc1f8313db3ffc775a0e1708ba85243
Author: Aravinda VK <avishwan@redhat.com>
Date:   Fri Oct 30 16:09:29 2015 +0530

    geo-rep: Allow setting config remote_gsyncd
    
    Restrictive ssh is not used in containerized environment
    where networking configuration is "net=host". SSH Pem keys
    pushed to the slave without gsyncd path in it. (Patch #12459)
    
    Actual remote_gsyncd path need to be set to actual path of gsyncd.
    With this patch, remote_gsyncd is removed from reserved option list.
    
    Change-Id: Ia2063e4654e378b62b2414bdad21143c86ad1b9a
    Signed-off-by: Aravinda VK <avishwan@redhat.com>
    BUG: 1276028
    Reviewed-on: http://review.gluster.org/12472
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>

Comment 1 Vijay Bellur 2015-11-18 07:16:34 UTC
REVIEW: http://review.gluster.org/12606 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 2 Vijay Bellur 2015-11-18 08:05:11 UTC
REVIEW: http://review.gluster.org/12607 (geo-rep: New Config option for ssh_port) posted (#1) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 3 Vijay Bellur 2015-11-18 12:31:16 UTC
REVIEW: http://review.gluster.org/12644 (geo-rep: Allow setting config remote_gsyncd) posted (#1) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 4 Vijay Bellur 2015-11-19 04:46:10 UTC
REVIEW: http://review.gluster.org/12652 (glusterd/geo-rep: Adding ssh-port option for geo-rep create) posted (#1) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 5 Vijay Bellur 2015-11-21 14:20:41 UTC
REVIEW: http://review.gluster.org/12644 (geo-rep: Allow setting config remote_gsyncd) posted (#2) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 6 Vijay Bellur 2015-11-21 14:21:02 UTC
REVIEW: http://review.gluster.org/12652 (glusterd/geo-rep: Adding ssh-port option for geo-rep create) posted (#2) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 7 Vijay Bellur 2015-11-23 19:08:40 UTC
COMMIT: http://review.gluster.org/12644 committed in release-3.7 by Vijay Bellur (vbellur@redhat.com) 
------
commit 1c7ffac108572b7085d8aea040f5726c6134d025
Author: Aravinda VK <avishwan@redhat.com>
Date:   Fri Oct 30 16:09:29 2015 +0530

    geo-rep: Allow setting config remote_gsyncd
    
    Restrictive ssh is not used in containerized environment
    where networking configuration is "net=host". SSH Pem keys
    pushed to the slave without gsyncd path in it. (Patch #12459)
    
    Actual remote_gsyncd path need to be set to actual path of gsyncd.
    With this patch, remote_gsyncd is removed from reserved option list.
    
    Change-Id: Ia2063e4654e378b62b2414bdad21143c86ad1b9a
    Signed-off-by: Aravinda VK <avishwan@redhat.com>
    BUG: 1283060
    Reviewed-on: http://review.gluster.org/12472
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
    (cherry picked from commit 7de355b42dc1f8313db3ffc775a0e1708ba85243)
    Reviewed-on: http://review.gluster.org/12644
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>

Comment 8 Vijay Bellur 2015-11-23 19:22:32 UTC
COMMIT: http://review.gluster.org/12652 committed in release-3.7 by Vijay Bellur (vbellur@redhat.com) 
------
commit c7b293beba0c327ad20a5c8b3e5635be80672f63
Author: Kotresh HR <khiremat@redhat.com>
Date:   Mon Nov 2 18:52:03 2015 +0530

    glusterd/geo-rep: Adding ssh-port option for geo-rep create
    
    Geo-replication uses default ssh port 22 for setup.
    i.e., to distribute ssh keys to slaves. In container
    environments, custom port number might be used.
    Hence to support custom port number for ssh, option
    is provided in geo-rep create command to take the
    same.
    
    Change-Id: I0fb61959b1c085342b8e4c21ac4e076fba5462f1
    BUG: 1283060
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12504
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Avra Sengupta <asengupt@redhat.com>
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
     (cherry picked from commit 5bb3c521431cc27b2826acd889bffb2f90ae7f73)
    Reviewed-on: http://review.gluster.org/12652
    Reviewed-by: Vijay Bellur <vbellur@redhat.com>

Comment 9 Vijay Bellur 2015-11-24 04:37:14 UTC
REVIEW: http://review.gluster.org/12607 (geo-rep: New Config option for ssh_port) posted (#3) for review on release-3.7 by Aravinda VK (avishwan@redhat.com)

Comment 10 Vijay Bellur 2015-11-24 06:18:16 UTC
REVIEW: http://review.gluster.org/12730 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on release-3.7 by Kotresh HR (khiremat@redhat.com)

Comment 11 Vijay Bellur 2015-11-25 03:17:09 UTC
REVIEW: http://review.gluster.org/12607 (geo-rep: New Config option for ssh_port) posted (#4) for review on release-3.7 by Vijay Bellur (vbellur@redhat.com)

Comment 12 Vijay Bellur 2015-11-25 07:17:23 UTC
COMMIT: http://review.gluster.org/12606 committed in release-3.7 by Venky Shankar (vshankar@redhat.com) 
------
commit 1952143a3c9639a00fe5e52e4368ea9f380a0172
Author: Kotresh HR <khiremat@redhat.com>
Date:   Thu Oct 29 09:30:15 2015 +0000

    geo-rep: Make restrictive ssh keys optional
    
    In containerized environment where networking
    configuration is "net=host", both host and
    containers use the same IP. The validations
    gsyncd shell and rsync to be the siblings
    fails. Hence, for now, creating restrictive
    ssh keys is made optional as follows.
    
    If the argument 'container' is passed, it
    will create non restrictive ssh keys else
    restrictive ssh keys.
    
    e.g.,
    gluster system:: execute gsec_create container
         Creates non restrictive ssh keys.
    gluster system:: execute gsec_create
         Creates restrictive ssh keys.
    
    Change-Id: Ibed362f64b9b4c9931207f863a2da944c6bd1d66
    BUG: 1283060
    Signed-off-by: Kotresh HR <khiremat@redhat.com>
    Reviewed-on: http://review.gluster.org/12459
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Aravinda VK <avishwan@redhat.com>
    (cherry picked from commit 6e036c758add503a170cc3134e95fea3e78e89cb)
    Reviewed-on: http://review.gluster.org/12606

Comment 13 Vijay Bellur 2015-11-25 07:18:48 UTC
COMMIT: http://review.gluster.org/12607 committed in release-3.7 by Venky Shankar (vshankar@redhat.com) 
------
commit a2c977fad73aa2df5eb5ffd51bc708354a08b88f
Author: Aravinda VK <avishwan@redhat.com>
Date:   Wed Oct 28 17:56:50 2015 +0530

    geo-rep: New Config option for ssh_port
    
    If different port used for SSH instead of 22, Geo-replication
    was failing to establish SSH connection.
    
    ssh_port option can be added using config:ssh_command and
    config:ssh_command_tar, but user has to remember complete
    ssh command used with parameter to add/modify ssh port.
    
    This patch adds new config option for ssh_port,
    
    gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \
            config ssh_port 52022
    
    Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4
    Signed-off-by: Aravinda VK <avishwan@redhat.com>
    BUG: 1283060
    Reviewed-on: http://review.gluster.org/12444
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Saravanakumar Arumugam <sarumuga@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Venky Shankar <vshankar@redhat.com>
    (cherry picked from commit 7d35eb5926869ed084295600502a85ce13be506f)
    Reviewed-on: http://review.gluster.org/12607
    Reviewed-by: Kotresh HR <khiremat@redhat.com>

Comment 14 Kaushal 2016-04-19 07:48:11 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.7, please open a new bug report.

glusterfs-3.7.7 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-users/2016-February/025292.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user


Note You need to log in before you can comment on or make changes to this bug.