The main challenge is generating two client volume files (one for socket, another for rdma). Other than that, as RPC layer now understands 'option transport-type socket,rdma' sending this string directly from cli to glusterd and to volfile is not so hard..
PATCH: http://patches.gluster.com/patch/6216 in master (glusterd/cli: option added to create volume with both transports)
Now cli takes both tcp as well as rdma as the trasport options and generate the volume files for them. root@bigbang:/home/raghu# gluster volume create vol transport tcp,rdma bigbang:/d/glusterfs/export/export bigbang:/e/glusterfs/export/exportCreation of volume vol has been successful. Please start the volume to access data. root@bigbang:/home/raghu# ls /etc/glusterd/vols/vol/ bricks cksum info vol.bigbang.d-glusterfs-export-export.vol vol.bigbang.e-glusterfs-export-export.vol vol-fuse.vol vol-rdma-fuse.vol But gluster volume info shows the volume type as rdma only. root@bigbang:/home/raghu# gluster volume info Volume Name: vol Type: Distribute Status: Created Number of Bricks: 2 Transport-type: rdma Bricks: Brick1: bigbang:/d/glusterfs/export/export Brick2: bigbang:/e/glusterfs/export/export
PATCH: http://patches.gluster.com/patch/6241 in master (if volume created is both of tcp and rdma type show it in volume info)
Now this is fixed with the latest git head[7e546e16925e50dc33db05c67b8b5cad1b3922ef]. Now one create a volume with transport type of both tcp and rdma. This is the output: root@bigbang:/home/raghu/work/src/Regression# gluster volume create vol transport tcp,rdma bigbang:/d/glusterfs/export/export Creation of volume vol has been successful. Please start the volume to access data. root@bigbang:/home/raghu/work/src/Regression# gluster volume info Volume Name: vol Type: Distribute Status: Created Number of Bricks: 1 Transport-type: tcp,rdma Bricks: Brick1: bigbang:/d/glusterfs/export/export
Added the following information in "Sharing Volume through RDMA and TCP/IP Simultaneously - You can create and share volumes with both RDMA and TCP/IP transport type. The set transport type of the volume can be viewed by issuing gluster volume info command." in What is New... section of 3.1.3 Release Notes.
My understanding is that engineering is aware this is not working, but it is documented as working. we currently have a customer trying to use this - they are a platinum level, key customer and we need to get an idea of how long it will take to fix this.
This bug was created mainly to handle to 'volgen' part of the 'gluster volume create'. The issue we noticed now is that we have some conflicts in the way we start the 'glusterfsd' on a single port (in case of 'tcp,rdma' transport), and the proper fix is bit disruptive and would need a bigger round of QA with everything involved with glusterd. The proposed fix is by introducing industry standard handshake for RDMA called 'RDMA-CM' (connection manager for rdma), through which we can handle all these issues, and the changes in glusterd won't be very disruptive. Estimated time requirement for bringing in RDMA-CM is around 2weeks. So this fix will not be making it to 3.2.1 release, but hopefully make it to 3.2.2.
*** Bug 2952 has been marked as a duplicate of this bug. ***
*** Bug 2953 has been marked as a duplicate of this bug. ***
PATCH: http://patches.gluster.com/patch/7418 in master (glusterd-volgen: fix rdma volume file path in case of 'tcp, rdma' transport.)
PATCH: http://patches.gluster.com/patch/7419 in master (fix multiple transport portmap issues in client handshake)
PATCH: http://patches.gluster.com/patch/7420 in master (nfs:command to change the transport type of nfs server for volumes of transport tcp, rdma)
PATCH: http://patches.gluster.com/patch/7422 in master (rpc-transport/rdma: don't return '0' in case of un-initiated rdma_connect())
PATCH: http://patches.gluster.com/patch/7416 in release-3.2 (fix multiple transport related portmap issues in client handshake)
PATCH: http://patches.gluster.com/patch/7417 in release-3.2 (nfs:command to change the transport type of nfs server for volumes of transport tcp, rdma)
Patch available in release3.2 branch and master branch.
Its working now. Created a tcp,rdma volume and then mounted through rdma using vol.rdma where vol is the volume name. Ran some tests and while tests are running (both fuse, rdma and the nfs mounts) changed the transport type of the nfs for the volume. Tests continued and the transport type was changed for nfs.
This bug should be closed.
This bug is no more valid as the ability to create a volume with both transport was added with 3.2.x releases itself, and is also present in mainline. Please open a specific bug about the issue if there is some functionality not working.