Red Hat Bugzilla – Bug 1040418
NFS: Scalability issue with subdir authentication
Last modified: 2015-02-04 05:23:48 EST
Description of problem: Version-Release number of selected component (if applicable): Gluster NFS has subdir authentication feature. It needs to be validated against possible scalability issues. e.g. 1. NFS Volume file size 2. rpc size during volfile change 3. Time for re-configure 4. Watch for mount timeouts 5. ~64K configurations (try with 64k entries) 6. Vary the number of hostnames/IP-addresses (range from 16 - 256) 7. RHS-C This is in fact not a defect but a tracker bug for scalability of subdir authentication RFE.
Can you get some performance numbers with the above parameters?
gluster vol set nfs.export-dir is working fine with 4k no of directories. ,/dir3878(10.70.36.240),/dir3879(10.70.36.240),/dir3880(10.70.36.240),/dir3881(10.70.36.240),/dir3882(10.70.36.240),/dir3883(10.70.36.240),/dir3884(10.70.36.240),/dir3885(10.70.36.240),/dir3886(10.70.36.240),/dir3887(10.70.36.240),/dir3888(10.70.36.240),/dir3889(10.70.36.240),/dir3890(10.70.36.240),/dir3891(10.70.36.240),/dir3892(10.70.36.240),/dir3893(10.70.36.240),/dir3894(10.70.36.240),/dir3895(10.70.36.240),/dir3896(10.70.36.240),/dir3897(10.70.36.240),/dir3898(10.70.36.240),/dir3899(10.70.36.240),/dir3900(10.70.36.240),/dir3901(10.70.36.240),/dir3902(10.70.36.240),/dir3903(10.70.36.240),/dir3904(10.70.36.240),/dir3905(10.70.36.240),/dir3906(10.70.36.240),/dir3907(10.70.36.240),/dir3908(10.70.36.240),/dir3909(10.70.36.240),/dir3910(10.70.36.240),/dir3911(10.70.36.240),/dir3912(10.70.36.240),/dir3913(10.70.36.240),/dir3914(10.70.36.240),/dir3915(10.70.36.240),/dir3916(10.70.36.240),/dir3917(10.70.36.240),/dir3918(10.70.36.240),/dir3919(10.70.36.240),/dir3920(10.70.36.240),/dir3921(10.70.36.240),/dir3922(10.70.36.240),/dir3923(10.70.36.240),/dir3924(10.70.36.240),/dir3925(10.70.36.240),/dir3926(10.70.36.240),/dir3927(10.70.36.240),/dir3928(10.70.36.240),/dir3929(10.70.36.240),/dir3930(10.70.36.240),/dir3931(10.70.36.240),/dir3932(10.70.36.240),/dir3933(10.70.36.240),/dir3934(10.70.36.240),/dir3935(10.70.36.240),/dir3936(10.70.36.240),/dir3937(10.70.36.240),/dir3938(10.70.36.240),/dir3939(10.70.36.240),/dir3940(10.70.36.240),/dir3941(10.70.36.240),/dir3942(10.70.36.240),/dir3943(10.70.36.240),/dir3944(10.70.36.240),/dir3945(10.70.36.240),/dir3946(10.70.36.240),/dir3947(10.70.36.240),/dir3948(10.70.36.240),/dir3949(10.70.36.240),/dir3950(10.70.36.240),/dir3951(10.70.36.240),/dir3952(10.70.36.240),/dir3953(10.70.36.240),/dir3954(10.70.36.240),/dir3955(10.70.36.240),/dir3956(10.70.36.240),/dir3957(10.70.36.240),/dir3958(10.70.36.240),/dir3959(10.70.36.240),/dir3960(10.70.36.240),/dir3961(10.70.36.240),/dir3962(10.70.36.240),/dir3963(10.70.36.240),/dir3964(10.70.36.240),/dir3965(10.70.36.240),/dir3966(10.70.36.240),/dir3967(10.70.36.240),/dir3968(10.70.36.240),/dir3969(10.70.36.240),/dir3970(10.70.36.240),/dir3971(10.70.36.240),/dir3972(10.70.36.240),/dir3973(10.70.36.240),/dir3974(10.70.36.240),/dir3975(10.70.36.240),/dir3976(10.70.36.240),/dir3977(10.70.36.240),/dir3978(10.70.36.240),/dir3979(10.70.36.240),/dir3980(10.70.36.240),/dir3981(10.70.36.240),/dir3982(10.70.36.240),/dir3983(10.70.36.240),/dir3984(10.70.36.240),/dir3985(10.70.36.240),/dir3986(10.70.36.240),/dir3987(10.70.36.240),/dir3988(10.70.36.240),/dir3989(10.70.36.240),/dir3990(10.70.36.240),/dir3991(10.70.36.240),/dir3992(10.70.36.240),/dir3993(10.70.36.240),/dir3994(10.70.36.240),/dir3995(10.70.36.240),/dir3996(10.70.36.240),/dir3997(10.70.36.240),/dir3998(10.70.36.240),/dir3999(10.70.36.240),/dir4000(10.70.36.240)" volume set: success real 1m1.550s user 0m54.144s sys 0m0.169s
however, gluster vol set nfs.export-dir command for 8k and 16K no. of directories is falling with below error. 2.197|10.70.42.101|10.70.42.200),/dir15989(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15990(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15991(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15992(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15993(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15994(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15995(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15996(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15997(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15998(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir15999(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200),/dir16000(10.70.42.200|10.70.42.197|10.70.42.101|10.70.42.200)" ./export-dir.sh: line 68: /usr/sbin/gluster: Argument list too long
./export-dir.sh: line 68: /usr/sbin/gluster: Argument list too long The error is thrown by the SHELL itself, not by the gluster. Rajesh will share the data, he collected from the SHELL. (bash)
I tested nfs.export-dir command in my Fedora 19 machine and the command fails with "Argument list too long" as observed by Anil and Santosh. Then I checked the current limit for the maximum allowed commandline argument length (using xargs --show-limits) and I got the following result POSIX upper limit on argument length (this system): 2091957 POSIX smallest allowable upper limit on argument length (all systems): 4096 Maximum length of command we could actually use: 2088810 Size of command buffer we are actually using: 131072 On RHEL 6.5 I got the following result. Your environment variables take up 2009 bytes POSIX upper limit on argument length (this system): 2617383 POSIX smallest allowable upper limit on argument length (all systems): 4096 Maximum length of command we could actually use: 2615374 Size of command buffer we are actually using: 131072 So it seems like as of now the limiting factor is command buffer size which is 131072.
Reviewed on: https://code.engineering.redhat.com/gerrit/13847
per discussion 12/23, this will go as a known issue as it is related to shell limitation. Removing it from the must fix list for corbett
As this bug is identified as Known Issue, please add the doctext.
May be Rajesh can rectify the text if he wants.
The modified text looks good to me.
Based on discussion with ndevos (dev) and asriram (doc), closing this and raising a doc bug for this to be known issue.