Bug 1151745

Summary: Option transport.socket.bind-address ignored
Product: [Community] GlusterFS Reporter: Niels de Vos <ndevos>
Component: glusterdAssignee: Niels de Vos <ndevos>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: bugs
Target Milestone: ---Keywords: EasyFix, Patch, Triaged
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: glusterfs-3.6.1 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-11-10 15:14:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1149863    
Bug Blocks: 1117822    

Description Niels de Vos 2014-10-11 10:06:45 UTC
+++ This bug was initially created as a clone of Bug #1149863 +++
+++                                                           +++
+++ This bug has been filed to get a fix in release-3.6, when +++
+++ a patch has been merged, please post it to the bugs       +++
+++ listed in the 'blocks' field.                             +++

Description of problem: when setting transport.socket.bind-address it is ignored except for actually binding the glusterd daemon. Basically NFS, SHD and everything else tries to connect to 127.0.0.1:24007 instead of the actual address bound. This means FUSE is the only option for connects and of course the volume get corrupted shortly after.

Look in xlators/mgmt/glusterd/src/glusterd-utils.c:glusterd_nodesvc_start() for possible fix. ie:

glusterd_nodesvc_start() calls runner_add_args( ... , "-s", "localhost", ...)

above provided by ndevos in IRC


Version-Release number of selected component (if applicable): 3.5.2


How reproducible: set transport.socket.bind-address option and restart. viola!


Steps to Reproduce:
1. set transport.socket.bind-address option
2. restart glusterfs-server
3.

Actual results: no NFS, no SHD and anything else that attempts to connect to 127.0.0.1:24007


Expected results: it should work


Additional info:

--- Additional comment from Charles Williams on 2014-10-06 21:04:47 CEST ---

forgot to add:

in the "volume management" section of glusterd.vol is where you can set this "option transport.socket.bind-address 123.123.123.123"

--- Additional comment from Anand Avati on 2014-10-06 23:37:59 CEST ---

REVIEW: http://review.gluster.org/8908 (glusterd: pass the bind-address to starting services) posted (#1) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-10-07 10:19:22 CEST ---

REVIEW: http://review.gluster.org/8908 (glusterd: pass the bind-address to starting services) posted (#2) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-10-07 10:19:25 CEST ---

REVIEW: http://review.gluster.org/8910 (glusterd: make bricks respect 'transport.socket.bind-address') posted (#1) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-10-08 04:57:23 CEST ---

COMMIT: http://review.gluster.org/8908 committed in master by Krishnan Parthasarathi (kparthas) 
------
commit 283fa797f4bf98130b42c36972305b8cb6e5aaaf
Author: Niels de Vos <ndevos>
Date:   Mon Oct 6 22:51:22 2014 +0200

    glusterd: pass the bind-address to starting services
    
    When the transport.socket.bind-address option is set to a hostname or
    ip-address, the services started by GlusterD fail to connect to the
    management daemon. GlusterD always forces the services to connect to the
    "localhost" hostname, even if it is not listening on that address.
    
    GlusterD should take the transport.socket.bind-address option into
    consideration, and pass that to the glusterfs-clients with the -s or
    --volfile commandline parameter.
    
    Note that this is not a change that removes all hard-coded dependencies
    on "localhost". This change merely makes it possible to start required
    services when the transport.socket.bind-address option is set.
    
    Change-Id: I36a0ed6c69342e6327adc258fea023929055d7f2
    BUG: 1149863
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/8908
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Jeff Darcy <jdarcy>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Tested-by: Krishnan Parthasarathi <kparthas>

--- Additional comment from Anand Avati on 2014-10-08 15:22:50 CEST ---

REVIEW: http://review.gluster.org/8910 (glusterd: make bricks respect 'transport.socket.bind-address') posted (#2) for review on master by Niels de Vos (ndevos)

--- Additional comment from Anand Avati on 2014-10-09 06:10:12 CEST ---

COMMIT: http://review.gluster.org/8910 committed in master by Krishnan Parthasarathi (kparthas) 
------
commit 430b874c4f1a171c106a9e1e6507e14e79805a1d
Author: Niels de Vos <ndevos>
Date:   Tue Oct 7 10:08:20 2014 +0200

    glusterd: make bricks respect 'transport.socket.bind-address'
    
    When GlusterD starts the brick processes, these will listen on all
    interfaces. When the 'transport.socket.bind-address' option is set in
    glusterd.vol, the brick processes should only listen on the specified
    hostname or IP-address.
    
    Change-Id: I8e7d1f294904081137c23f3446261329d0d13bba
    BUG: 1149863
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/8910
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Tested-by: Krishnan Parthasarathi <kparthas>

Comment 1 Niels de Vos 2014-10-11 10:11:44 UTC
Backport guidelines:
- http://www.gluster.org/community/documentation/index.php/Backport_Guidelines

Patches that need backporting to 3.6:
1. http://review.gluster.org/8908
2. http://review.gluster.org/8910

Comment 2 Anand Avati 2014-10-21 07:09:23 UTC
REVIEW: http://review.gluster.org/8950 (glusterd: pass the bind-address to starting services) posted (#1) for review on release-3.6 by Niels de Vos (ndevos)

Comment 3 Anand Avati 2014-10-21 07:09:26 UTC
REVIEW: http://review.gluster.org/8951 (glusterd: make bricks respect 'transport.socket.bind-address') posted (#1) for review on release-3.6 by Niels de Vos (ndevos)

Comment 4 Anand Avati 2014-10-21 12:30:19 UTC
COMMIT: http://review.gluster.org/8950 committed in release-3.6 by Vijay Bellur (vbellur) 
------
commit 8e598bb02ae431cc08e7bfe449fede21e0ed2f35
Author: Niels de Vos <ndevos>
Date:   Tue Oct 21 09:04:23 2014 +0200

    glusterd: pass the bind-address to starting services
    
    When the transport.socket.bind-address option is set to a hostname or
    ip-address, the services started by GlusterD fail to connect to the
    management daemon. GlusterD always forces the services to connect to the
    "localhost" hostname, even if it is not listening on that address.
    
    GlusterD should take the transport.socket.bind-address option into
    consideration, and pass that to the glusterfs-clients with the -s or
    --volfile commandline parameter.
    
    Note that this is not a change that removes all hard-coded dependencies
    on "localhost". This change merely makes it possible to start required
    services when the transport.socket.bind-address option is set.
    
    Cherry picked from commit 283fa797f4bf98130b42c36972305b8cb6e5aaaf:
    > Change-Id: I36a0ed6c69342e6327adc258fea023929055d7f2
    > BUG: 1149863
    > Signed-off-by: Niels de Vos <ndevos>
    > Reviewed-on: http://review.gluster.org/8908
    > Tested-by: Gluster Build System <jenkins.com>
    > Reviewed-by: Jeff Darcy <jdarcy>
    > Reviewed-by: Krishnan Parthasarathi <kparthas>
    > Tested-by: Krishnan Parthasarathi <kparthas>
    
    Change-Id: I36a0ed6c69342e6327adc258fea023929055d7f2
    BUG: 1151745
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/8950
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 5 Anand Avati 2014-10-21 17:16:08 UTC
COMMIT: http://review.gluster.org/8951 committed in release-3.6 by Vijay Bellur (vbellur) 
------
commit d32bc8aa365dbb804af8c8ff18eecbf349eba9e8
Author: Niels de Vos <ndevos>
Date:   Tue Oct 21 09:05:43 2014 +0200

    glusterd: make bricks respect 'transport.socket.bind-address'
    
    When GlusterD starts the brick processes, these will listen on all
    interfaces. When the 'transport.socket.bind-address' option is set in
    glusterd.vol, the brick processes should only listen on the specified
    hostname or IP-address.
    
    Cherry picked from commit 430b874c4f1a171c106a9e1e6507e14e79805a1d:
    > Change-Id: I8e7d1f294904081137c23f3446261329d0d13bba
    > BUG: 1149863
    > Signed-off-by: Niels de Vos <ndevos>
    > Reviewed-on: http://review.gluster.org/8910
    > Tested-by: Gluster Build System <jenkins.com>
    > Reviewed-by: Krishnan Parthasarathi <kparthas>
    > Tested-by: Krishnan Parthasarathi <kparthas>
    
    Change-Id: I8e7d1f294904081137c23f3446261329d0d13bba
    BUG: 1151745
    Signed-off-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/8951
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Vijay Bellur <vbellur>

Comment 6 Niels de Vos 2014-11-10 15:14:13 UTC
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.1, please reopen this bug report.

glusterfs-3.6.1 has been announced [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://supercolony.gluster.org/pipermail/gluster-users/2014-November/019410.html
[2] http://supercolony.gluster.org/mailman/listinfo/gluster-users