+++ This bug was initially created as a clone of Bug #1276028 +++ +++ This bug was initially created as a clone of Bug #1275971 +++ Description of problem: When NET=HOST is used, Unable to create Geo-replication session. Setup: ------ CoreOS + Centos containers RHGS 3.1 RPMs installed inside Containers. Net=Host setup 4 containers, 2 containers for Master and 2 containers for Slave.(One brick in each container) Replica Volumes - Master and Slave Issue: -------- - Unable to create the session. Port mapping is used for ssh(Custom port is mapped to port 22). Geo-replication all SSH commands are executed without port option to ssh. - After creating session Rsync was failing due to validation in gsyncd. (gsyncd is used as shell instead of bash) Workaround: ----------- - Kotresh modified gverify.sh and hook script to use custom SSH port instead of default. (https://gist.github.com/kotreshhr/dd16c5fca425b417c097) - Geo-rep config options to use ssh options runtime. gluster vol geo-rep<master vol> <slavehost>::<slavevol> config ssh_command_tar "ssh -p 50002 -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/tar_ssh.pem" gluster vol geo-rep<master vol> <slavehost>::<slavevol> config ssh_command "ssh -p 50002 -oPasswordAuthentication=no -oStrictHostKeyChecking=no -i /var/lib/glusterd/geo-replication/secret.pem" - Suggestion to remove "command=" from authorized_keys files of Slave nodes to prevent executing all commands via gsyncd shell. - Replaced /nonexistent/gsyncd with actual path of gsyncd(/usr/libexec/glusterfs/gsyncd) in Geo-replication session config file. --- Additional comment from Vijay Bellur on 2015-10-28 08:35:22 EDT --- REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#1) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-10-29 02:14:12 EDT --- REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#2) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-10-29 05:39:15 EDT --- REVIEW: http://review.gluster.org/12459 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on master by Kotresh HR (khiremat) --- Additional comment from Vijay Bellur on 2015-10-30 06:44:31 EDT --- REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#1) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-11-02 01:26:52 EST --- REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#2) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-11-02 01:27:15 EST --- REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#3) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-11-17 00:45:31 EST --- REVIEW: http://review.gluster.org/12472 (geo-rep: Allow setting config remote_gsyncd) posted (#3) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-11-17 00:45:39 EST --- REVIEW: http://review.gluster.org/12444 (geo-rep: New Config option for ssh_port) posted (#4) for review on master by Aravinda VK (avishwan) --- Additional comment from Vijay Bellur on 2015-11-17 10:00:11 EST --- COMMIT: http://review.gluster.org/12444 committed in master by Jeff Darcy (jdarcy) ------ commit 7d35eb5926869ed084295600502a85ce13be506f Author: Aravinda VK <avishwan> Date: Wed Oct 28 17:56:50 2015 +0530 geo-rep: New Config option for ssh_port If different port used for SSH instead of 22, Geo-replication was failing to establish SSH connection. ssh_port option can be added using config:ssh_command and config:ssh_command_tar, but user has to remember complete ssh command used with parameter to add/modify ssh port. This patch adds new config option for ssh_port, gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \ config ssh_port 52022 Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4 Signed-off-by: Aravinda VK <avishwan> BUG: 1276028 Reviewed-on: http://review.gluster.org/12444 Tested-by: NetBSD Build System <jenkins.org> Reviewed-by: Saravanakumar Arumugam <sarumuga> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Venky Shankar <vshankar> --- Additional comment from Vijay Bellur on 2015-11-17 10:06:26 EST --- COMMIT: http://review.gluster.org/12472 committed in master by Venky Shankar (vshankar) ------ commit 7de355b42dc1f8313db3ffc775a0e1708ba85243 Author: Aravinda VK <avishwan> Date: Fri Oct 30 16:09:29 2015 +0530 geo-rep: Allow setting config remote_gsyncd Restrictive ssh is not used in containerized environment where networking configuration is "net=host". SSH Pem keys pushed to the slave without gsyncd path in it. (Patch #12459) Actual remote_gsyncd path need to be set to actual path of gsyncd. With this patch, remote_gsyncd is removed from reserved option list. Change-Id: Ia2063e4654e378b62b2414bdad21143c86ad1b9a Signed-off-by: Aravinda VK <avishwan> BUG: 1276028 Reviewed-on: http://review.gluster.org/12472 Tested-by: NetBSD Build System <jenkins.org> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Saravanakumar Arumugam <sarumuga> Reviewed-by: Venky Shankar <vshankar>
REVIEW: http://review.gluster.org/12606 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on release-3.7 by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/12607 (geo-rep: New Config option for ssh_port) posted (#1) for review on release-3.7 by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/12644 (geo-rep: Allow setting config remote_gsyncd) posted (#1) for review on release-3.7 by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/12652 (glusterd/geo-rep: Adding ssh-port option for geo-rep create) posted (#1) for review on release-3.7 by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/12644 (geo-rep: Allow setting config remote_gsyncd) posted (#2) for review on release-3.7 by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/12652 (glusterd/geo-rep: Adding ssh-port option for geo-rep create) posted (#2) for review on release-3.7 by Aravinda VK (avishwan)
COMMIT: http://review.gluster.org/12644 committed in release-3.7 by Vijay Bellur (vbellur) ------ commit 1c7ffac108572b7085d8aea040f5726c6134d025 Author: Aravinda VK <avishwan> Date: Fri Oct 30 16:09:29 2015 +0530 geo-rep: Allow setting config remote_gsyncd Restrictive ssh is not used in containerized environment where networking configuration is "net=host". SSH Pem keys pushed to the slave without gsyncd path in it. (Patch #12459) Actual remote_gsyncd path need to be set to actual path of gsyncd. With this patch, remote_gsyncd is removed from reserved option list. Change-Id: Ia2063e4654e378b62b2414bdad21143c86ad1b9a Signed-off-by: Aravinda VK <avishwan> BUG: 1283060 Reviewed-on: http://review.gluster.org/12472 Tested-by: NetBSD Build System <jenkins.org> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Saravanakumar Arumugam <sarumuga> Reviewed-by: Venky Shankar <vshankar> (cherry picked from commit 7de355b42dc1f8313db3ffc775a0e1708ba85243) Reviewed-on: http://review.gluster.org/12644 Reviewed-by: Vijay Bellur <vbellur>
COMMIT: http://review.gluster.org/12652 committed in release-3.7 by Vijay Bellur (vbellur) ------ commit c7b293beba0c327ad20a5c8b3e5635be80672f63 Author: Kotresh HR <khiremat> Date: Mon Nov 2 18:52:03 2015 +0530 glusterd/geo-rep: Adding ssh-port option for geo-rep create Geo-replication uses default ssh port 22 for setup. i.e., to distribute ssh keys to slaves. In container environments, custom port number might be used. Hence to support custom port number for ssh, option is provided in geo-rep create command to take the same. Change-Id: I0fb61959b1c085342b8e4c21ac4e076fba5462f1 BUG: 1283060 Signed-off-by: Kotresh HR <khiremat> Reviewed-on: http://review.gluster.org/12504 Tested-by: NetBSD Build System <jenkins.org> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Avra Sengupta <asengupt> Reviewed-by: Aravinda VK <avishwan> Reviewed-by: Venky Shankar <vshankar> (cherry picked from commit 5bb3c521431cc27b2826acd889bffb2f90ae7f73) Reviewed-on: http://review.gluster.org/12652 Reviewed-by: Vijay Bellur <vbellur>
REVIEW: http://review.gluster.org/12607 (geo-rep: New Config option for ssh_port) posted (#3) for review on release-3.7 by Aravinda VK (avishwan)
REVIEW: http://review.gluster.org/12730 (geo-rep: Make restrictive ssh keys optional) posted (#1) for review on release-3.7 by Kotresh HR (khiremat)
REVIEW: http://review.gluster.org/12607 (geo-rep: New Config option for ssh_port) posted (#4) for review on release-3.7 by Vijay Bellur (vbellur)
COMMIT: http://review.gluster.org/12606 committed in release-3.7 by Venky Shankar (vshankar) ------ commit 1952143a3c9639a00fe5e52e4368ea9f380a0172 Author: Kotresh HR <khiremat> Date: Thu Oct 29 09:30:15 2015 +0000 geo-rep: Make restrictive ssh keys optional In containerized environment where networking configuration is "net=host", both host and containers use the same IP. The validations gsyncd shell and rsync to be the siblings fails. Hence, for now, creating restrictive ssh keys is made optional as follows. If the argument 'container' is passed, it will create non restrictive ssh keys else restrictive ssh keys. e.g., gluster system:: execute gsec_create container Creates non restrictive ssh keys. gluster system:: execute gsec_create Creates restrictive ssh keys. Change-Id: Ibed362f64b9b4c9931207f863a2da944c6bd1d66 BUG: 1283060 Signed-off-by: Kotresh HR <khiremat> Reviewed-on: http://review.gluster.org/12459 Tested-by: NetBSD Build System <jenkins.org> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Aravinda VK <avishwan> (cherry picked from commit 6e036c758add503a170cc3134e95fea3e78e89cb) Reviewed-on: http://review.gluster.org/12606
COMMIT: http://review.gluster.org/12607 committed in release-3.7 by Venky Shankar (vshankar) ------ commit a2c977fad73aa2df5eb5ffd51bc708354a08b88f Author: Aravinda VK <avishwan> Date: Wed Oct 28 17:56:50 2015 +0530 geo-rep: New Config option for ssh_port If different port used for SSH instead of 22, Geo-replication was failing to establish SSH connection. ssh_port option can be added using config:ssh_command and config:ssh_command_tar, but user has to remember complete ssh command used with parameter to add/modify ssh port. This patch adds new config option for ssh_port, gluster volume geo-replication <MASTERVOL> <SLAVEHOST::<SLAVEVOL> \ config ssh_port 52022 Change-Id: I7753a09485f0b1f49d2b2a80b962c720817c96f4 Signed-off-by: Aravinda VK <avishwan> BUG: 1283060 Reviewed-on: http://review.gluster.org/12444 Tested-by: NetBSD Build System <jenkins.org> Reviewed-by: Saravanakumar Arumugam <sarumuga> Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Venky Shankar <vshankar> (cherry picked from commit 7d35eb5926869ed084295600502a85ce13be506f) Reviewed-on: http://review.gluster.org/12607 Reviewed-by: Kotresh HR <khiremat>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.7, please open a new bug report. glusterfs-3.7.7 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] https://www.gluster.org/pipermail/gluster-users/2016-February/025292.html [2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user