Description of problem: Shares with an nfs.rpc-auth-allow and a more than approx 400 character crashes the corresponding rpc services. We do see below error message in /var/log/glusterfs/nfs.log: [2016-05-18 23:54:30.348639] E [rpcsvc.c:565:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully The message "E [MSGID: 112067] [nfs3.c:4702:nfs3_fsstat] 0-nfs-nfsv3: Bad Handle" repeated 2 times between [2016-05-18 23:53:30.731217] and [2016-05-18 23:54:31.785566] [2016-05-18 23:54:31.785585] W [MSGID: 112199] [nfs3-helpers.c:3418:nfs3_log_common_res] 0-nfs-nfsv3: (null) => (XID: e3929939, FSSTAT: NFS: 10001(Illegal NFS file handle), POSIX: 14(Bad address)) [No data available] [2016-05-18 23:54:31.785763] E [nfs3.c:341:__nfs3_get_volume_id] (-->/usr/lib64/glusterfs/3.7.5/xlator/nfs/server.so(nfs3_fsstat_reply+0x41) [0x7f4508728e01] -->/usr/lib64/glusterfs/3.7.5/xlator/nfs/server.so(nfs3_request_xlator_deviceid+0x78) [0x7f4508728478] -->/usr/lib64/glusterfs/3.7.5/xlator/nfs/server.so(__nfs3_get_volume_id+0xae) [0x7f45087283be] ) 0-nfs-nfsv3: invalid argument: xl [Invalid argument] [2016-05-18 23:54:31.960984] E [MSGID: 112067] [nfs3.c:4702:nfs3_fsstat] 0-nfs-nfsv3: Bad Handle [2016-05-18 23:54:31.961021] W [MSGID: 112199] [nfs3-helpers.c:3418:nfs3_log_common_res] 0-nfs-nfsv3: (null) => (XID: e4929939, FSSTAT: NFS: 10001(Illegal NFS file handle), POSIX: 14(Bad address)) [Invalid argument] [2016-05-18 23:54:31.961073] E [nfs3.c:341:__nfs3_get_volume_id] (-->/usr/lib64/glusterfs/3.7.5/xlator/nfs/server.so(nfs3_fsstat_reply+0x41) [0x7f4508728e01] -->/usr/lib64/glusterfs/3.7.5/xlator/nfs/server.so(nfs3_request_xlator_deviceid+0x78) [0x7f4508728478] -->/usr/lib64/glusterfs/3.7.5/xlator/nfs/server.so(__nfs3_get_volume_id+0xae) [0x7f45087283be] ) 0-nfs-nfsv3: invalid argument: xl [Invalid argument] [2016-05-18 23:54:33.252289] W [rpcsvc.c:278:rpcsvc_program_actor] 0-rpc-service: RPC program version not available (req 100003 4) for 10.222.160.25:776 [2016-05-18 23:54:33.252319] E [rpcsvc.c:565:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully [2016-05-18 23:54:33.252825] W [rpcsvc.c:278:rpcsvc_program_actor] 0-rpc-service: RPC program version not available (req 100003 4) for 10.222.160.25:776 [2016-05-18 23:54:33.252841] E [rpcsvc.c:565:rpcsvc_check_and_reply_error] 0-rpcsvc: rpc actor failed to complete successfully The rpc service for glusternfs failing with invalid NFS handle. Version-Release number of selected component (if applicable): How reproducible: reproducible Steps to Reproduce: 1. Create gluster volume 2. Allow access to list of hosts using nfs.rpc-auth-allow (more than 400 characters) # gluster volume set <vol-name> nfs.rpc-auth-allow ${HOSTS} 3. Show mount # showmount -e <hostname> rpc mount export: RPC: Timed out Actual results: rpc service failed Expected results: Should allow access to all hosts from ${HOSTS} list Additional info: sosreport is uploaded at :- http://collab-shell.usersys.redhat.com/01637136/
More information from customer: It is complete reproducible on a test cluster (installed from rhgs 3.1.2 iso). queries will timeout when nfs.rpc-auth-allow exceed 256 characters. Steps to reproduce will follow in a private comment.
Niels, May I know the reason of moving back the component back to gluster-nfs? We have realligned the downstream components with upstream to keep a parity and in upstream we have nfs component and hence the change.
(In reply to Atin Mukherjee from comment #9) > Niels, > > May I know the reason of moving back the component back to gluster-nfs? We > have realligned the downstream components with upstream to keep a parity and > in upstream we have nfs component and hence the change. This is a Gluster/NFS (gNFS) bug, we use the "nfs" component for changes to GlusterFS in relation with NFS-Ganesha.
A patch for this has been included in RHGS-3.2.0 since it contains a rebase of GlusterFS 3.8 (http://review.gluster.org/14700).
Rahul - can this BZ be tested with latest build?
Verified this Bug on glusterfs-3.8.4-5.el7rhgs.x86_64 Steps: 1.HOSTS=$(echo 192.168.10.{1..40} | tr ' ' ',') 2.[root@dhcp47-159 ganesha]# gluster volume set Vol1 nfs.rpc-auth-allow ${HOSTS} volume set: success 3.[root@dhcp47-159 ganesha]# showmount -e localhost Export list for localhost: /Vol1 192.168.10.1,192.168.10.2,192.168.10.3,192.168.10.4,192.168.10.5,192.168.10.6,192.168.10.7,192.168.10.8,192.168.10.9,192.168.10.10,192.168.10.11,192.168.10.12,192.168.10.13,192.168.10.14,192.168.10.15,192.168.10.16,192.168.10.17,192.168.10.18,192.168.10.19,192.168.10.20,192.168.10.21,192.168.10.22,192.168.10.23,192.168.10.24,192.168.10.25,192.168.10.26,192.168.10.27,192.168.10.28,192.168.10.29,192.168.10.30,192.168.10.31,192.168.10.32,192.168.10.33,192.168.10.34,192.168.10.35,192.168.10.36,192.168.10.37,192.168.10.38,192.168.10.39,192.168.10.40 [root@dhcp46-241 ganesha]# gluster v info Volume Name: Vol1 Type: Distributed-Replicate Volume ID: 9678475b-3ecb-4f22-995b-346c5bcdecca Status: Started Snapshot Count: 0 Number of Bricks: 6 x 2 = 12 Transport-type: tcp Bricks: Brick1: 10.70.47.3:/mnt/data1/b1 Brick2: 10.70.47.159:/mnt/data1/b1 Brick3: 10.70.46.241:/mnt/data1/b1 Brick4: 10.70.46.219:/mnt/data1/b1 Brick5: 10.70.47.3:/mnt/data2/b2 Brick6: 10.70.47.159:/mnt/data2/b2 Brick7: 10.70.46.241:/mnt/data2/b2 Brick8: 10.70.46.219:/mnt/data2/b2 Brick9: 10.70.47.3:/mnt/data3/b3 Brick10: 10.70.47.159:/mnt/data3/b3 Brick11: 10.70.46.241:/mnt/data3/b3 Brick12: 10.70.46.219:/mnt/data3/b3 Options Reconfigured: nfs.rpc-auth-allow: 192.168.10.1,192.168.10.2,192.168.10.3,192.168.10.4,192.168.10.5,192.168.10.6,192.168.10.7,192.168.10.8,192.168.10.9,192.168.10.10,192.168.10.11,192.168.10.12,192.168.10.13,192.168.10.14,192.168.10.15,192.168.10.16,192.168.10.17,192.168.10.18,192.168.10.19,192.168.10.20,192.168.10.21,192.168.10.22,192.168.10.23,192.168.10.24,192.168.10.25,192.168.10.26,192.168.10.27,192.168.10.28,192.168.10.29,192.168.10.30,192.168.10.31,192.168.10.32,192.168.10.33,192.168.10.34,192.168.10.35,192.168.10.36,192.168.10.37,192.168.10.38,192.168.10.39,192.168.10.40 transport.address-family: inet performance.readdir-ahead: on nfs.disable: off nfs-ganesha: disable cluster.enable-shared-storage: disable As the issue reported is no more observed with this build,Hence marking this Bug as Verified
Doc-text Looks good to me.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2017-0486.html