Description of problem: Very often gluster cli commands fail on rdma. In the glusterd logs, I see errors saying request being received from non privileged port. Version-Release number of selected component (if applicable): 3.3.0qa42 How reproducible: Often Steps to Reproduce: 1. gluster volume info 2. 3. Actual results: [root@gqaib-02 ~]# gluster volume info No volumes present Connection failed. Please check if gluster daemon is operational. Expected results: Additional info: glusterd log: [2012-05-23 12:59:31.525740] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-05-23 12:59:31.535714] I [glusterd-handler.c:860:glusterd_handle_cli_get_volume] 0-glusterd: Received get vol req [2012-05-23 12:59:31.596585] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:588) after handshake is complete [2012-05-23 12:59:31.608517] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:587) after handshake is complete [2012-05-23 12:59:31.681073] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:585) after handshake is complete [2012-05-23 12:59:31.693070] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:584) after handshake is complete [2012-05-23 12:59:31.765521] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:582) after handshake is complete [2012-05-23 12:59:31.777442] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:581) after handshake is complete [2012-05-23 12:59:31.849780] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:579) after handshake is complete [2012-05-23 12:59:31.861732] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:578) after handshake is complete [2012-05-23 12:59:31.934745] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:576) after handshake is complete [2012-05-23 12:59:31.946490] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:575) after handshake is complete [2012-05-23 12:59:32.018730] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:573) after handshake is complete [2012-05-23 12:59:32.030652] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:572) after handshake is complete [2012-05-23 12:59:32.102673] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:570) after handshake is complete [2012-05-23 12:59:32.114553] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:569) after handshake is complete [2012-05-23 12:59:32.187868] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:567) after handshake is complete [2012-05-23 12:59:32.199826] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:566) after handshake is complete [2012-05-23 12:59:32.272364] E [rdma.c:4512:gf_rdma_event_handler] 0-rpc-transport/rdma: rdma.management: pollin received on tcp socket (peer: 10.16.157.97:564) after handshake is complete [2012-05-23 12:59:32.281562] E [rpcsvc.c:491:rpcsvc_handle_rpc_call] 0-glusterd: Request received from non-privileged port. Failing request [2012-05-23 12:59:32.540491] E [rpcsvc.c:491:rpcsvc_handle_rpc_call] 0-glusterd: Request received from non-privileged port. Failing request
Need more testing with 'glusterd' being run with RDMA transport. Not much coverage as of now.
in CLI -> glusterd, there is a high chance of getting XDR with more than 2k size, and hence this is possible with the way our RDMA is implemented (inline Vs RDMA).... either need to change all the CLI cmds to go through RDMA, or increase the inline size to higher in case of glusterd For now closing with WONTFIX. Lets get a real use case of RDMA only glusterd, and then we can fix it.