Bug 1166515

REVIEW: http://review.gluster.org/9173 (rdma: mount hangs for rdma type transport.) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9174 (rdma: glusterd crash if rdma_disconnect is called as soon as connect a request.) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9175 (rdma:setting rdma REUSEADDR flag to rdma id.) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9176 (rdma:rdma fuse mount hangs for tcp,rdma volumes if brick is down.) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9177 (rdma: client connection establishment takes more time) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9178 (rdma:client process will hang if server is started to send the request before completing connection establishment) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9179 (mount:Handle -o transport option in mount.glusterfs) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9180 (rdma:Removing RDMA tech preview cli message.) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9181 (mount: Verify mount failure in mount.glusterfs wrapper.) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9182 (rdma: Wrong volfile fetch on fuse mounting tcp,rdma volume via rdma) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9183 (rdma: Client volfile name change for supporting rdma) posted (#1) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9178 (rdma:client process will hang if server is started to send the request before completing connection establishment) posted (#2) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9175 (rdma:setting rdma REUSEADDR flag to rdma id.) posted (#2) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9180 (rdma:Removing RDMA tech preview cli message.) posted (#2) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9182 (rdma: Wrong volfile fetch on fuse mounting tcp,rdma volume via rdma) posted (#2) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9182 (rdma: Wrong volfile fetch on fuse mounting tcp,rdma volume via rdma) posted (#3) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9181 (mount: Verify mount failure in mount.glusterfs wrapper.) posted (#2) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9183 (rdma: Client volfile name change for supporting rdma) posted (#2) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9182 (rdma: Wrong volfile fetch on fuse mounting tcp,rdma volume via rdma) posted (#4) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9175 (rdma:setting rdma REUSEADDR flag to rdma id.) posted (#3) for review on release-3.6 by Anoop C S (achiraya)

REVIEW: http://review.gluster.org/9175 (rdma:setting rdma REUSEADDR flag to rdma id.) posted (#4) for review on release-3.6 by Anoop C S (achiraya)

COMMIT: http://review.gluster.org/9173 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 964c73566a452bcfd3e2ef3119f5407091f977b3
Author: Mohammed Rafi KC <rkavunga>
Date:   Thu Sep 25 15:36:30 2014 +0530

    rdma: mount hangs for rdma type transport.
    
            Backport of http://review.gluster.org/8850
    
    rdma transport type mount will hang if there is a delay
    in network to receive,we will set transport as connected
    if we get an event type RDMA_CM_EVENT_ESTABLISHED,
    we cannot assure whether client or server will get the
    event first, the only condition is that the side which
    sends the first request should wait for the event.
    If client gets the event first, then it sends DUMP request,
    in server side the request will reach, but it will reject
    the rpc request since it didn't get the RDMA_CM_EVENT_ESTABLISHED.
    So in server we will set the connected flag as soon
    as rdma_accept is called.
    
    Change-Id: Ia15884ab28a0c51cccfef84258c3671a468054d9
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/8850
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9173
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9174 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit fa7a3e0b39692b392d325d009386d4d2ad0306f5
Author: Mohammed Rafi KC <rkavunga>
Date:   Mon Oct 13 11:12:14 2014 +0530

    rdma: glusterd crash if rdma_disconnect is called as soon as connect a request.
    
            Backport of http://review.gluster.org/8925
    
    we are initializing connection in server side immediately after
    rdma_accept is called. But we are delaying adding the transport
    to listener list until getting RDMA_CM_EVENT_ESTABLISHED event.
    Before getting this event if disconnect is called glusterd will
    try to remove the transport from list which is not added. So if
    the list is empty it causes a glusterd crash . In this patch we
    will call the function to initialize the connection as soon as
    rdma_accept is called.
    
    Change-Id: Ie783fdbb4342459e5bc162bf8600bf47c1e2e909
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/8925
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9174
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9175 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 707ef16edcf4b14f46bb515b3464fa4368ce9b7c
Author: Mohammed Rafi KC <rkavunga>
Date:   Thu Oct 30 11:46:56 2014 +0530

    rdma:setting rdma REUSEADDR flag to rdma id.
    
            Backport of http://review.gluster.org/9005
    
    When we restart the process, it will go TIME_WAIT state to make sure
    that all the data in the transport is successfully delivered. REUSEADDR
    allows server to bind to an address which is in TIME_WAIT state.
    
    Change-Id: Ifb191967930d23bac31ceef1b1745d6b9fb0007b
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9005
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9175
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9176 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit bb1601b94039b27b5a148479b61895a100ce8e6d
Author: Mohammed Rafi KC <rkavunga>
Date:   Thu Sep 18 04:21:04 2014 -0400

    rdma:rdma fuse mount hangs for tcp,rdma volumes if brick is down.
    
            Backport of http://review.gluster.org/8762
    
    When we try to mount a tcp,rdma volume as rdma
    transport using FUSE protocol, then mount will
    hang if the brick is down. When we kill a process,
    signal will be received in glusterfsd process and
    it will call pmap_signout with port listening on tcp only.
    In case of the tcp,rdma there will be two ports,
    and port which is listening for rdma will not
    called for sign out.
    So the mount process will try to connect to a port
    which is not open and it will keep trying to connect.
    This patch will call pmap_signout for rdma port also,
    So when mount tries to get the brick port,it will fail.
    
    Change-Id: I73f90d7340afa3b0b1278924206f1488e4094a62
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/8762
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9176
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9177 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 7efd84b1b743d3a91e23fd97dbf8a1d89b0d1f44
Author: Mohammed Rafi KC <rkavunga>
Date:   Thu Oct 16 11:28:33 2014 +0530

    rdma: client connection establishment takes more time
    
            Backport of http://review.gluster.org/8934
    
    For rdma type only volume client connection establishment
    with server takes more than three seconds. Because for
    tcp,rdma type volume, will have 2 ports one for tcp and
    one for rdma, tcp port is stored with brickname and rdma
    port is stored as "brickname.rdma" during pamap_sighin.
    During the handshake when trying to get the brick port
    for rdma clients, since we are not aware of server
    transport type, we will append '.rdma' with brick name.
    So for tcp,rdma volume there will be an entry with
    '.rdma', but it will fail for rdma type only volume.
    So we will try again, this time without appending '.rdma'
    using a flag variable need_different_port, and it will succeed,
    but the reconnection happens only after 3 seconds.
    In this patch for rdma only type volume
    we will append '.rdma' during the pmap_signin. So during the
    handshake we will get the correct port for first try itself.
    Since we don't need to retry , we can remove the
    need_different_port flag variable.
    
    Change-Id: I82a8a27f0e65a2e287f321e5e8292d86c6baf5b4
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/8934
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9177
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9178 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 5908a73f975c271f53e8539b2429cc1cdafd8184
Author: Mohammed Rafi KC <rkavunga>
Date:   Thu Oct 30 10:35:57 2014 +0530

    rdma:client process will hang if server is started to send the request before completing connection establishment
    
            Backport of http://review.gluster.org/9003
    
    in rdma, client and server will interchange their available
    buffers during the handshake to post incoming messages.
    Initially the available buffer is set to one, for the first message
    during handshake,when first message is received, quota for the
    buffer will set to proper value. So before receiving the message
    if server started to send the message, then the reserverd buffer for
    handshake will be utilised, then the handshake will fail because
    of lack of buffers. So we should block sending messages by server
    before proper connection establishment.
    
    Change-Id: I1797ae8a25163864c338641d74cd4eef6fb34bb5
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9003
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9178
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9179 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 00b4c62db3d17c47e672fdd5c49e9306d6c2de88
Author: Mohammed Rafi KC <rkavunga>
Date:   Wed Nov 19 17:22:21 2014 +0530

    mount:Handle -o transport option in mount.glusterfs
    
            Backport of http://review.gluster.org/#/c/9147/
    
    In current scenario ,when tcp transport type(default)
    specified for mounting,glusterfs mount script won't
    append '.tcp' to volume name.But to accommodate the change
    in http://review.gluster.org/#/c/9146/, we need to
    append ".tcp" with volfile-id if '-o transport=tcp' is given.
    
    Change-Id: I5444ac38846192de4af02535435b86bb00422aab
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9179
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9180 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 2163e3e8d6184c6ae8f6bf5909389783b1bd40f4
Author: Mohammed Rafi KC <rkavunga>
Date:   Tue Nov 18 14:58:20 2014 +0530

    rdma:Removing RDMA tech preview cli message.
    
            Backport of http://review.gluster.org/#/c/9141/
    
    Creation of rdma and tcp,rdma volume will display a warning
    message since it was in tech preview. This patch will remove
    the warning message during the volume creation.
    
    Change-Id: If4adb22cb20e2ef8d32bc798a8002c3e8e23fbdd
    BUG: 1166515
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9180
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Humble Devassy Chirammal <humble.devassy>
    Reviewed-by: Raghavendra G <rgowdapp>
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9181 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit de7205ddddeb56e10d04a000c3e011099c1cbc55
Author: Humble Chirammal <hchiramm>
Date:   Fri Aug 8 15:54:03 2014 +0530

    mount: Verify mount failure in mount.glusterfs wrapper.
    
            Backport of http://review.gluster.org/8438
    
    The result of mount command execution is not checked properly, thus no
    proper message given for the end user. This patch fix the same.
    
    Change-Id: If1bd43fb6b7916575743e04b48af8cf382f9da3e
    BUG: 1166515
    Signed-off-by: Humble Chirammal <hchiramm>
    Reviewed-on: http://review.gluster.org/8438
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>
    Reviewed-on: http://review.gluster.org/9181
    Reviewed-by: Humble Devassy Chirammal <humble.devassy>
    Reviewed-by: Raghavendra G <rgowdapp>
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9182 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 605db00dbad94d5bba6510e7695383715ef5035a
Author: Anoop C S <achiraya>
Date:   Wed Nov 12 18:02:15 2014 +0530

    rdma: Wrong volfile fetch on fuse mounting tcp,rdma volume via rdma
    
            Backport of http://review.gluster.org/8498
    
    As of now for both tcp only volumes and rdma only volumes, volfile
    names are in the format <volname>-fuse.vol. This patch will change
    the client volfile namings as shown below.
    
     * TCP mounts always use <volname>-fuse.vol
     * RDMA mounts always use <volname>.rdma-fuse.vol
    
    Following the above naming convention, for tcp,rdma volumes both
    volfiles will be present under /var/lib/glusterd/vols/<volname>/
    such that rdma only volume can be mounted as
    
    mount -t glusterfs -o transport=rdma <server/ip>:/<volname> <mount-point>
    OR
    mount -t glusterfs <server/ip>:/<volname>.rdma <mount-point>
    
    The above command format can also be used to fuse mount a tcp,rdma
    volume via rdma transport.
    
    When we try to fuse mount a tcp,rdma volume with transport-type as rdma
    it silently mounts via tcp. This change will also make sure that it
    fetches the correct volfile based on the transport-type specified
    from client side.
    
    Change-Id: Id8b74c1c3e1e7fd323463061f8b13dd623fa6876
    BUG: 1166515
    Signed-off-by: Anoop C S <achiraya>
    Reviewed-on: http://review.gluster.org/8498
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Krishnan Parthasarathi <kparthas>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9182
    Reviewed-by: Raghavendra Bhat <raghavendra>

COMMIT: http://review.gluster.org/9183 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit 50952cda111c84c966dc0427bbdb618e31bf8d78
Author: Anoop C S <achiraya>
Date:   Wed Nov 19 16:27:05 2014 +0530

    rdma: Client volfile name change for supporting rdma
    
            Backport of http://review.gluster.org/9146
    
    For rdma only volumes, daemons like snapd, glustershd etc make
    use of tcp transport for their operations. This patch will introduce
    the support of rdma by default for those daemons in rdma only volumes.
    In order to accomodate this change we rename the tcp client volfile
    labels from
    
    <volname>-fuse.vol
    to
    <volname>.tcp-fuse.vol
    
    Change-Id: Id5e5db0680a07fa6b6d003bad45748464cd7658e
    BUG: 1166515
    Signed-off-by: Anoop C S <achiraya>
    Reviewed-on: http://review.gluster.org/9146
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaushal M <kaushal>
    Reviewed-on: http://review.gluster.org/9183
    Reviewed-by: Raghavendra G <rgowdapp>
    Reviewed-by: Raghavendra Bhat <raghavendra>

REVIEW: http://review.gluster.org/9392 (rdma:vectored write fails for rdma.) posted (#2) for review on release-3.6 by mohammed rafi  kc (rkavunga)

COMMIT: http://review.gluster.org/9392 committed in release-3.6 by Raghavendra Bhat (raghavendra) 
------
commit e281c3dfac7a4178447436356c7e72e4e12bd09a
Author: Mohammed Rafi KC <rkavunga>
Date:   Fri Dec 5 19:46:53 2014 +0530

    rdma:vectored write fails for rdma.
    
            Backport of http://review.gluster.org/9247
    
    For rdma write with payload count greater than one
    will fail due to insuffient memory to hold the
    buffers in rpc transport layer. It was expecting
    only one vector in payload, So it can only able
    to decode the first iovec from payload, and the
    rest will be discarded.
    
    Thnaks to Raghavendra Gowdappa for fixing the
    same.
    
    BUG: 1166515
    Change-Id: I5e44d240366af78df35df326fe0660ef8497147b
    Signed-off-by: Mohammed Rafi KC <rkavunga>
    Reviewed-on: http://review.gluster.org/9247
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Raghavendra G <rgowdapp>
    Tested-by: Raghavendra G <rgowdapp>
    Reviewed-on: http://review.gluster.org/9392
    Reviewed-by: Raghavendra Bhat <raghavendra>

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.2, please reopen this bug report.

glusterfs-3.6.2 has been announced on the Gluster Developers mailinglist [1], packages for several distributions should already be or become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

The fix for this bug likely to be included in all future GlusterFS releases i.e. release > 3.6.2.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/5978
[2] http://news.gmane.org/gmane.comp.file-systems.gluster.user
[3] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/6137

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.6.2, please reopen this bug report.

glusterfs-3.6.2 has been announced on the Gluster Developers mailinglist [1], packages for several distributions should already be or become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

The fix for this bug likely to be included in all future GlusterFS releases i.e. release > 3.6.2.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/5978
[2] http://news.gmane.org/gmane.comp.file-systems.gluster.user
[3] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/6137