Bug 764026 - (GLUSTER-2294) Currently there is no way through cli to make a volume listen on both the transports (socket/rdma)
Currently there is no way through cli to make a volume listen on both the tra...
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: cli (Show other bugs)
3.1.1
All Linux
high Severity high
: ---
: ---
Assigned To: Amar Tumballi
: Reopened
: GLUSTER-2952 GLUSTER-2953 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2011-01-14 05:09 EST by Amar Tumballi
Modified: 2015-12-01 11:45 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-01-31 21:15:50 EST
Type: ---
Regression: RTA
Mount Type: fuse
Documentation: DA
CRM: 3274
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Description Amar Tumballi 2011-01-14 05:09:21 EST
The main challenge is generating two client volume files (one for socket, another for rdma). Other than that, as RPC layer now understands 'option transport-type socket,rdma' sending this string directly from cli to glusterd and to volfile is not so hard..
Comment 1 Anand Avati 2011-02-22 08:45:48 EST
PATCH: http://patches.gluster.com/patch/6216 in master (glusterd/cli: option added to create volume with both transports)
Comment 2 Raghavendra Bhat 2011-02-22 22:34:07 EST
Now cli takes both tcp as well as rdma as the trasport options and generate the volume files for them.

root@bigbang:/home/raghu# gluster volume create vol transport tcp,rdma bigbang:/d/glusterfs/export/export bigbang:/e/glusterfs/export/exportCreation of volume vol has been successful. Please start the volume to access data.



root@bigbang:/home/raghu# ls /etc/glusterd/vols/vol/
bricks  cksum  info  vol.bigbang.d-glusterfs-export-export.vol  vol.bigbang.e-glusterfs-export-export.vol  vol-fuse.vol  vol-rdma-fuse.vol



But gluster volume info shows the volume type as rdma only.



root@bigbang:/home/raghu# gluster volume info

Volume Name: vol
Type: Distribute
Status: Created
Number of Bricks: 2
Transport-type: rdma
Bricks:
Brick1: bigbang:/d/glusterfs/export/export
Brick2: bigbang:/e/glusterfs/export/export
Comment 3 Anand Avati 2011-02-28 23:10:28 EST
PATCH: http://patches.gluster.com/patch/6241 in master (if volume created is both of tcp and rdma type show it in volume info)
Comment 4 Raghavendra Bhat 2011-03-01 02:13:01 EST
Now this is fixed with the latest git head[7e546e16925e50dc33db05c67b8b5cad1b3922ef]. Now one create a volume with transport type of both tcp and rdma. This is the output:


root@bigbang:/home/raghu/work/src/Regression# gluster volume create vol transport tcp,rdma bigbang:/d/glusterfs/export/export 
Creation of volume vol has been successful. Please start the volume to access data.
root@bigbang:/home/raghu/work/src/Regression# gluster volume info

Volume Name: vol
Type: Distribute
Status: Created
Number of Bricks: 1
Transport-type: tcp,rdma
Bricks:
Brick1: bigbang:/d/glusterfs/export/export
Comment 5 Divya 2011-03-21 00:25:51 EDT
Added the following information in "Sharing Volume through RDMA and TCP/IP Simultaneously - You can create and share volumes with both RDMA and TCP/IP transport type. The set transport type of the volume can be viewed by issuing gluster volume info command." in What is New... section of 3.1.3 Release Notes.
Comment 6 Renee 2011-05-27 11:57:05 EDT
My understanding is that engineering is aware this is not working, but it is documented as working.

we currently have a customer trying to use this - they are a platinum level, key customer and we need to get an idea of how long it will take to fix this.
Comment 7 Amar Tumballi 2011-05-31 01:45:58 EDT
This bug was created mainly to handle to 'volgen' part of the 'gluster volume create'.

The issue we noticed now is that we have some conflicts in the way we start the 'glusterfsd' on a single port (in case of 'tcp,rdma' transport), and the proper fix is bit disruptive and would need a bigger round of QA with everything involved with glusterd. 

The proposed fix is by introducing industry standard handshake for RDMA called 'RDMA-CM' (connection manager for rdma), through which we can handle all these issues, and the changes in glusterd won't be very disruptive. Estimated time requirement for bringing in RDMA-CM is around 2weeks. So this fix will not be making it to 3.2.1 release, but hopefully make it to 3.2.2.
Comment 8 Amar Tumballi 2011-06-06 22:50:25 EDT
*** Bug 2952 has been marked as a duplicate of this bug. ***
Comment 9 Amar Tumballi 2011-06-06 22:50:41 EDT
*** Bug 2953 has been marked as a duplicate of this bug. ***
Comment 10 Anand Avati 2011-06-08 11:17:10 EDT
PATCH: http://patches.gluster.com/patch/7418 in master (glusterd-volgen: fix rdma volume file path in case of 'tcp, rdma' transport.)
Comment 11 Anand Avati 2011-06-08 11:17:16 EDT
PATCH: http://patches.gluster.com/patch/7419 in master (fix multiple transport portmap issues in client handshake)
Comment 12 Anand Avati 2011-06-08 11:17:21 EDT
PATCH: http://patches.gluster.com/patch/7420 in master (nfs:command to change the transport type of nfs server for volumes of transport tcp, rdma)
Comment 13 Anand Avati 2011-06-08 11:17:28 EDT
PATCH: http://patches.gluster.com/patch/7422 in master (rpc-transport/rdma: don't return '0' in case of un-initiated rdma_connect())
Comment 14 Anand Avati 2011-06-08 11:19:21 EDT
PATCH: http://patches.gluster.com/patch/7416 in release-3.2 (fix multiple transport related portmap issues in client handshake)
Comment 15 Anand Avati 2011-06-08 11:19:26 EDT
PATCH: http://patches.gluster.com/patch/7417 in release-3.2 (nfs:command to change the transport type of nfs server for volumes of transport tcp, rdma)
Comment 16 Amar Tumballi 2011-06-08 11:26:02 EDT
Patch available in release3.2 branch and master branch.
Comment 17 Raghavendra Bhat 2011-06-09 05:54:07 EDT
Its working now. Created a tcp,rdma volume and then mounted through rdma using vol.rdma where vol is the volume name. Ran some tests and while tests are running (both fuse, rdma and the nfs mounts) changed the transport type of the nfs for the volume. Tests continued and the transport type was changed for nfs.
Comment 18 Harshavardhana 2011-12-13 13:29:14 EST
This bug should be closed.
Comment 19 Amar Tumballi 2012-01-31 21:15:50 EST
This bug is no more valid as the ability to create a volume with both transport was added with 3.2.x releases itself, and is also present in mainline. Please open a specific bug about the issue if there is some functionality not working.

Note You need to log in before you can comment on or make changes to this bug.