Description of problem: I am unable to ls -l the mount point from a MAC OSX 10.9 client when stripe is < 1. I can rsync a file to the mount point but I cannot list the directory and its contents. Version-Release number of selected component (if applicable):3.5.2 How reproducible:Very Steps to Reproduce: 1.Create a pool with stripe < 1 2.mount via NFS from MAC client 3.ls -l the mount point. Actual results: tcp dump from main brick 11:03:49.159146 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2349793, win 8183, options [nop,nop,TS val 481967906 ecr 6529176], length 0 11:03:49.159217 IP 10.3.22.172.2348092412 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.159452 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627141:2627293, ack 2419701, win 6146, options [nop,nop,TS val 6529178 ecr 4917145], length 152 11:03:49.159636 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2419701:2419841, ack 2627293, win 3593, options [nop,nop,TS val 4917147 ecr 6529178], length 140 11:03:49.159690 IP 10.4.2.21.nfs > 10.3.22.172.2348092412: reply ok 132 readdir 11:03:49.160759 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2349929, win 8183, options [nop,nop,TS val 481967908 ecr 6529178], length 0 11:03:49.161312 IP 10.3.22.172.2348092413 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.161539 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627293:2627445, ack 2419841, win 6146, options [nop,nop,TS val 6529180 ecr 4917147], length 152 11:03:49.161743 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2419841:2419981, ack 2627445, win 3593, options [nop,nop,TS val 4917149 ecr 6529180], length 140 11:03:49.161785 IP 10.4.2.21.nfs > 10.3.22.172.2348092413: reply ok 132 readdir 11:03:49.162787 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350065, win 8183, options [nop,nop,TS val 481967910 ecr 6529180], length 0 11:03:49.163275 IP 10.3.22.172.2348092414 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.163500 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627445:2627597, ack 2419981, win 6146, options [nop,nop,TS val 6529182 ecr 4917149], length 152 11:03:49.163673 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2419981:2420121, ack 2627597, win 3593, options [nop,nop,TS val 4917151 ecr 6529182], length 140 11:03:49.163726 IP 10.4.2.21.nfs > 10.3.22.172.2348092414: reply ok 132 readdir 11:03:49.164710 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350201, win 8183, options [nop,nop,TS val 481967911 ecr 6529182], length 0 11:03:49.165168 IP 10.3.22.172.2348092415 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.165391 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627597:2627749, ack 2420121, win 6146, options [nop,nop,TS val 6529184 ecr 4917151], length 152 11:03:49.165602 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420121:2420261, ack 2627749, win 3593, options [nop,nop,TS val 4917153 ecr 6529184], length 140 11:03:49.165644 IP 10.4.2.21.nfs > 10.3.22.172.2348092415: reply ok 132 readdir 11:03:49.166615 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350337, win 8183, options [nop,nop,TS val 481967912 ecr 6529184], length 0 11:03:49.167114 IP 10.3.22.172.2348092416 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.167342 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627749:2627901, ack 2420261, win 6146, options [nop,nop,TS val 6529186 ecr 4917153], length 152 11:03:49.167525 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420261:2420401, ack 2627901, win 3593, options [nop,nop,TS val 4917155 ecr 6529186], length 140 11:03:49.167594 IP 10.4.2.21.nfs > 10.3.22.172.2348092416: reply ok 132 readdir 11:03:49.169256 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350473, win 8183, options [nop,nop,TS val 481967914 ecr 6529186], length 0 11:03:49.169643 IP 10.3.22.172.2348092417 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.169896 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627901:2628053, ack 2420401, win 6146, options [nop,nop,TS val 6529188 ecr 4917155], length 152 11:03:49.170081 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420401:2420541, ack 2628053, win 3593, options [nop,nop,TS val 4917157 ecr 6529188], length 140 11:03:49.170112 IP 10.4.2.21.nfs > 10.3.22.172.2348092417: reply ok 132 readdir 11:03:49.171074 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350609, win 8183, options [nop,nop,TS val 481967915 ecr 6529188], length 0 11:03:49.171548 IP 10.3.22.172.2348092418 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.171779 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628053:2628205, ack 2420541, win 6146, options [nop,nop,TS val 6529190 ecr 4917157], length 152 11:03:49.171953 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420541:2420681, ack 2628205, win 3593, options [nop,nop,TS val 4917159 ecr 6529190], length 140 11:03:49.172005 IP 10.4.2.21.nfs > 10.3.22.172.2348092418: reply ok 132 readdir 11:03:49.172994 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350745, win 8183, options [nop,nop,TS val 481967916 ecr 6529190], length 0 11:03:49.173452 IP 10.3.22.172.2348092419 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.173686 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628205:2628357, ack 2420681, win 6146, options [nop,nop,TS val 6529192 ecr 4917159], length 152 11:03:49.173882 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420681:2420821, ack 2628357, win 3593, options [nop,nop,TS val 4917161 ecr 6529192], length 140 11:03:49.173921 IP 10.4.2.21.nfs > 10.3.22.172.2348092419: reply ok 132 readdir 11:03:49.174917 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350881, win 8183, options [nop,nop,TS val 481967917 ecr 6529192], length 0 11:03:49.175441 IP 10.3.22.172.2348092420 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.175687 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628357:2628509, ack 2420821, win 6146, options [nop,nop,TS val 6529194 ecr 4917161], length 152 11:03:49.175870 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420821:2420961, ack 2628509, win 3593, options [nop,nop,TS val 4917163 ecr 6529194], length 140 11:03:49.175905 IP 10.4.2.21.nfs > 10.3.22.172.2348092420: reply ok 132 readdir 11:03:49.176866 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351017, win 8183, options [nop,nop,TS val 481967918 ecr 6529194], length 0 11:03:49.177450 IP 10.3.22.172.2348092421 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.177680 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628509:2628661, ack 2420961, win 6146, options [nop,nop,TS val 6529196 ecr 4917163], length 152 11:03:49.177892 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420961:2421101, ack 2628661, win 3593, options [nop,nop,TS val 4917165 ecr 6529196], length 140 11:03:49.177929 IP 10.4.2.21.nfs > 10.3.22.172.2348092421: reply ok 132 readdir 11:03:49.179705 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351153, win 8183, options [nop,nop,TS val 481967919 ecr 6529196], length 0 11:03:49.179774 IP 10.3.22.172.2348092422 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.179998 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628661:2628813, ack 2421101, win 6146, options [nop,nop,TS val 6529198 ecr 4917165], length 152 11:03:49.180174 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421101:2421241, ack 2628813, win 3593, options [nop,nop,TS val 4917168 ecr 6529198], length 140 11:03:49.180232 IP 10.4.2.21.nfs > 10.3.22.172.2348092422: reply ok 132 readdir 11:03:49.181306 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351289, win 8183, options [nop,nop,TS val 481967921 ecr 6529198], length 0 11:03:49.181851 IP 10.3.22.172.2348092423 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.182110 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628813:2628965, ack 2421241, win 6146, options [nop,nop,TS val 6529200 ecr 4917168], length 152 11:03:49.182309 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421241:2421381, ack 2628965, win 3593, options [nop,nop,TS val 4917170 ecr 6529200], length 140 11:03:49.182339 IP 10.4.2.21.nfs > 10.3.22.172.2348092423: reply ok 132 readdir 11:03:49.183303 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351425, win 8183, options [nop,nop,TS val 481967923 ecr 6529201], length 0 11:03:49.183834 IP 10.3.22.172.2348092424 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.184057 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628965:2629117, ack 2421381, win 6146, options [nop,nop,TS val 6529202 ecr 4917170], length 152 11:03:49.184270 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421381:2421521, ack 2629117, win 3593, options [nop,nop,TS val 4917172 ecr 6529202], length 140 11:03:49.184307 IP 10.4.2.21.nfs > 10.3.22.172.2348092424: reply ok 132 readdir 11:03:49.185280 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351561, win 8183, options [nop,nop,TS val 481967924 ecr 6529202], length 0 11:03:49.185807 IP 10.3.22.172.2348092425 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.186032 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629117:2629269, ack 2421521, win 6146, options [nop,nop,TS val 6529204 ecr 4917172], length 152 11:03:49.186220 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421521:2421661, ack 2629269, win 3593, options [nop,nop,TS val 4917174 ecr 6529204], length 140 11:03:49.186264 IP 10.4.2.21.nfs > 10.3.22.172.2348092425: reply ok 132 readdir 11:03:49.187311 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351697, win 8183, options [nop,nop,TS val 481967925 ecr 6529204], length 0 11:03:49.187796 IP 10.3.22.172.2348092426 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.188049 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629269:2629421, ack 2421661, win 6146, options [nop,nop,TS val 6529206 ecr 4917174], length 152 11:03:49.188222 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421661:2421801, ack 2629421, win 3593, options [nop,nop,TS val 4917176 ecr 6529206], length 140 11:03:49.188252 IP 10.4.2.21.nfs > 10.3.22.172.2348092426: reply ok 132 readdir 11:03:49.189760 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351833, win 8183, options [nop,nop,TS val 481967927 ecr 6529206], length 0 11:03:49.190317 IP 10.3.22.172.2348092427 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.190569 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629421:2629573, ack 2421801, win 6146, options [nop,nop,TS val 6529209 ecr 4917176], length 152 11:03:49.190771 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421801:2421941, ack 2629573, win 3593, options [nop,nop,TS val 4917178 ecr 6529209], length 140 11:03:49.190801 IP 10.4.2.21.nfs > 10.3.22.172.2348092427: reply ok 132 readdir 11:03:49.191705 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351969, win 8183, options [nop,nop,TS val 481967928 ecr 6529209], length 0 11:03:49.192301 IP 10.3.22.172.2348092428 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.192530 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629573:2629725, ack 2421941, win 6146, options [nop,nop,TS val 6529211 ecr 4917178], length 152 11:03:49.192736 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421941:2422081, ack 2629725, win 3593, options [nop,nop,TS val 4917180 ecr 6529211], length 140 11:03:49.192774 IP 10.4.2.21.nfs > 10.3.22.172.2348092428: reply ok 132 readdir 11:03:49.193711 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352105, win 8183, options [nop,nop,TS val 481967929 ecr 6529211], length 0 11:03:49.194092 IP 10.3.22.172.2348092429 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.194316 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629725:2629877, ack 2422081, win 6146, options [nop,nop,TS val 6529212 ecr 4917180], length 152 11:03:49.194507 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422081:2422221, ack 2629877, win 3593, options [nop,nop,TS val 4917182 ecr 6529212], length 140 11:03:49.194552 IP 10.4.2.21.nfs > 10.3.22.172.2348092429: reply ok 132 readdir 11:03:49.195396 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352241, win 8183, options [nop,nop,TS val 481967930 ecr 6529213], length 0 11:03:49.195814 IP 10.3.22.172.2348092430 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.196042 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629877:2630029, ack 2422221, win 6146, options [nop,nop,TS val 6529214 ecr 4917182], length 152 11:03:49.196211 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422221:2422361, ack 2630029, win 3593, options [nop,nop,TS val 4917184 ecr 6529214], length 140 11:03:49.196256 IP 10.4.2.21.nfs > 10.3.22.172.2348092430: reply ok 132 readdir 11:03:49.197129 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352377, win 8183, options [nop,nop,TS val 481967931 ecr 6529214], length 0 11:03:49.197730 IP 10.3.22.172.2348092431 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.197954 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630029:2630181, ack 2422361, win 6146, options [nop,nop,TS val 6529216 ecr 4917184], length 152 11:03:49.198168 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422361:2422501, ack 2630181, win 3593, options [nop,nop,TS val 4917185 ecr 6529216], length 140 11:03:49.198207 IP 10.4.2.21.nfs > 10.3.22.172.2348092431: reply ok 132 readdir 11:03:49.199583 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352513, win 8183, options [nop,nop,TS val 481967933 ecr 6529216], length 0 11:03:49.199989 IP 10.3.22.172.2348092432 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.200211 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630181:2630333, ack 2422501, win 6146, options [nop,nop,TS val 6529218 ecr 4917185], length 152 11:03:49.200422 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422501:2422641, ack 2630333, win 3593, options [nop,nop,TS val 4917188 ecr 6529218], length 140 11:03:49.200464 IP 10.4.2.21.nfs > 10.3.22.172.2348092432: reply ok 132 readdir 11:03:49.201534 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352649, win 8183, options [nop,nop,TS val 481967934 ecr 6529219], length 0 11:03:49.202065 IP 10.3.22.172.2348092433 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.202295 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630333:2630485, ack 2422641, win 6146, options [nop,nop,TS val 6529220 ecr 4917188], length 152 11:03:49.202478 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422641:2422781, ack 2630485, win 3593, options [nop,nop,TS val 4917190 ecr 6529220], length 140 11:03:49.202533 IP 10.4.2.21.nfs > 10.3.22.172.2348092433: reply ok 132 readdir 11:03:49.203507 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352785, win 8183, options [nop,nop,TS val 481967935 ecr 6529221], length 0 11:03:49.204900 IP 10.3.22.172.2348092434 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.205124 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630485:2630637, ack 2422781, win 6146, options [nop,nop,TS val 6529223 ecr 4917190], length 152 11:03:49.205353 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422781:2422921, ack 2630637, win 3593, options [nop,nop,TS val 4917193 ecr 6529223], length 140 11:03:49.205404 IP 10.4.2.21.nfs > 10.3.22.172.2348092434: reply ok 132 readdir 11:03:49.206380 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352921, win 8183, options [nop,nop,TS val 481967937 ecr 6529224], length 0 11:03:49.206967 IP 10.3.22.172.2348092435 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.207189 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630637:2630789, ack 2422921, win 6146, options [nop,nop,TS val 6529225 ecr 4917193], length 152 11:03:49.207371 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422921:2423061, ack 2630789, win 3593, options [nop,nop,TS val 4917195 ecr 6529225], length 140 11:03:49.207437 IP 10.4.2.21.nfs > 10.3.22.172.2348092435: reply ok 132 readdir 11:03:49.209500 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353057, win 8183, options [nop,nop,TS val 481967939 ecr 6529226], length 0 11:03:49.209960 IP 10.3.22.172.2348092436 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.210185 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630789:2630941, ack 2423061, win 6146, options [nop,nop,TS val 6529228 ecr 4917195], length 152 11:03:49.210374 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423061:2423201, ack 2630941, win 3593, options [nop,nop,TS val 4917198 ecr 6529228], length 140 11:03:49.210433 IP 10.4.2.21.nfs > 10.3.22.172.2348092436: reply ok 132 readdir 11:03:49.211308 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353193, win 8183, options [nop,nop,TS val 481967940 ecr 6529229], length 0 11:03:49.211806 IP 10.3.22.172.2348092437 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.212060 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630941:2631093, ack 2423201, win 6146, options [nop,nop,TS val 6529230 ecr 4917198], length 152 11:03:49.212229 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423201:2423341, ack 2631093, win 3593, options [nop,nop,TS val 4917200 ecr 6529230], length 140 11:03:49.212260 IP 10.4.2.21.nfs > 10.3.22.172.2348092437: reply ok 132 readdir 11:03:49.213510 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353329, win 8183, options [nop,nop,TS val 481967941 ecr 6529230], length 0 11:03:49.213774 IP 10.3.22.172.2348092438 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.213996 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631093:2631245, ack 2423341, win 6146, options [nop,nop,TS val 6529232 ecr 4917200], length 152 11:03:49.214213 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423341:2423481, ack 2631245, win 3593, options [nop,nop,TS val 4917202 ecr 6529232], length 140 11:03:49.214248 IP 10.4.2.21.nfs > 10.3.22.172.2348092438: reply ok 132 readdir 11:03:49.215168 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353465, win 8183, options [nop,nop,TS val 481967942 ecr 6529232], length 0 11:03:49.215586 IP 10.3.22.172.2348092439 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.215810 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631245:2631397, ack 2423481, win 6146, options [nop,nop,TS val 6529234 ecr 4917202], length 152 11:03:49.216000 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423481:2423621, ack 2631397, win 3593, options [nop,nop,TS val 4917203 ecr 6529234], length 140 11:03:49.216059 IP 10.4.2.21.nfs > 10.3.22.172.2348092439: reply ok 132 readdir 11:03:49.217245 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353601, win 8183, options [nop,nop,TS val 481967943 ecr 6529234], length 0 11:03:49.217674 IP 10.3.22.172.2348092440 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.217929 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631397:2631549, ack 2423621, win 6146, options [nop,nop,TS val 6529236 ecr 4917203], length 152 11:03:49.218111 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423621:2423761, ack 2631549, win 3593, options [nop,nop,TS val 4917205 ecr 6529236], length 140 11:03:49.218142 IP 10.4.2.21.nfs > 10.3.22.172.2348092440: reply ok 132 readdir 11:03:49.219518 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353737, win 8183, options [nop,nop,TS val 481967945 ecr 6529236], length 0 11:03:49.220006 IP 10.3.22.172.2348092441 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.220230 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631549:2631701, ack 2423761, win 6146, options [nop,nop,TS val 6529238 ecr 4917205], length 152 11:03:49.220398 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423761:2423901, ack 2631701, win 3593, options [nop,nop,TS val 4917208 ecr 6529238], length 140 11:03:49.220448 IP 10.4.2.21.nfs > 10.3.22.172.2348092441: reply ok 132 readdir 11:03:49.221261 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353873, win 8183, options [nop,nop,TS val 481967946 ecr 6529239], length 0 11:03:49.221770 IP 10.3.22.172.2348092442 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.221994 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631701:2631853, ack 2423901, win 6146, options [nop,nop,TS val 6529240 ecr 4917208], length 152 11:03:49.222213 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423901:2424041, ack 2631853, win 3593, options [nop,nop,TS val 4917210 ecr 6529240], length 140 11:03:49.222256 IP 10.4.2.21.nfs > 10.3.22.172.2348092442: reply ok 132 readdir 11:03:49.223312 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354009, win 8183, options [nop,nop,TS val 481967948 ecr 6529240], length 0 11:03:49.223805 IP 10.3.22.172.2348092443 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.224058 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631853:2632005, ack 2424041, win 6146, options [nop,nop,TS val 6529242 ecr 4917210], length 152 11:03:49.224244 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424041:2424181, ack 2632005, win 3593, options [nop,nop,TS val 4917212 ecr 6529242], length 140 11:03:49.224275 IP 10.4.2.21.nfs > 10.3.22.172.2348092443: reply ok 132 readdir 11:03:49.225248 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354145, win 8183, options [nop,nop,TS val 481967949 ecr 6529242], length 0 11:03:49.225770 IP 10.3.22.172.2348092444 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.225993 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632005:2632157, ack 2424181, win 6146, options [nop,nop,TS val 6529244 ecr 4917212], length 152 11:03:49.226205 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424181:2424321, ack 2632157, win 3593, options [nop,nop,TS val 4917214 ecr 6529244], length 140 11:03:49.226239 IP 10.4.2.21.nfs > 10.3.22.172.2348092444: reply ok 132 readdir 11:03:49.227352 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354281, win 8183, options [nop,nop,TS val 481967950 ecr 6529244], length 0 11:03:49.227885 IP 10.3.22.172.2348092445 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.228109 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632157:2632309, ack 2424321, win 6146, options [nop,nop,TS val 6529246 ecr 4917214], length 152 11:03:49.228277 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424321:2424461, ack 2632309, win 3593, options [nop,nop,TS val 4917216 ecr 6529246], length 140 11:03:49.228325 IP 10.4.2.21.nfs > 10.3.22.172.2348092445: reply ok 132 readdir 11:03:49.229665 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354417, win 8183, options [nop,nop,TS val 481967952 ecr 6529246], length 0 11:03:49.230132 IP 10.3.22.172.2348092446 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.230368 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632309:2632461, ack 2424461, win 6146, options [nop,nop,TS val 6529249 ecr 4917216], length 152 11:03:49.230565 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424461:2424601, ack 2632461, win 3593, options [nop,nop,TS val 4917218 ecr 6529249], length 140 11:03:49.230604 IP 10.4.2.21.nfs > 10.3.22.172.2348092446: reply ok 132 readdir 11:03:49.231705 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354553, win 8183, options [nop,nop,TS val 481967953 ecr 6529249], length 0 11:03:49.232072 IP 10.3.22.172.2348092447 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.232295 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632461:2632613, ack 2424601, win 6146, options [nop,nop,TS val 6529250 ecr 4917218], length 152 11:03:49.232485 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424601:2424741, ack 2632613, win 3593, options [nop,nop,TS val 4917220 ecr 6529250], length 140 11:03:49.232529 IP 10.4.2.21.nfs > 10.3.22.172.2348092447: reply ok 132 readdir 11:03:49.233546 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354689, win 8183, options [nop,nop,TS val 481967954 ecr 6529251], length 0 11:03:49.233850 IP 10.3.22.172.2348092448 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.234073 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632613:2632765, ack 2424741, win 6146, options [nop,nop,TS val 6529252 ecr 4917220], length 152 11:03:49.234262 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424741:2424881, ack 2632765, win 3593, options [nop,nop,TS val 4917222 ecr 6529252], length 140 11:03:49.234320 IP 10.4.2.21.nfs > 10.3.22.172.2348092448: reply ok 132 readdir 11:03:49.235220 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354825, win 8183, options [nop,nop,TS val 481967955 ecr 6529252], length 0 11:03:49.235699 IP 10.3.22.172.2348092449 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.235952 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632765:2632917, ack 2424881, win 6146, options [nop,nop,TS val 6529254 ecr 4917222], length 152 11:03:49.236119 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424881:2425021, ack 2632917, win 3593, options [nop,nop,TS val 4917223 ecr 6529254], length 140 11:03:49.236149 IP 10.4.2.21.nfs > 10.3.22.172.2348092449: reply ok 132 readdir 11:03:49.237030 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354961, win 8183, options [nop,nop,TS val 481967956 ecr 6529254], length 0 11:03:49.237487 IP 10.3.22.172.2348092450 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512 11:03:49.237738 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632917:2633069, ack 2425021, win 6146, options [nop,nop,TS val 6529256 ecr 4917223], length 152 11:03:49.237936 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2425021:2425161, ack 2633069, win 3593, options [nop,nop,TS val 4917225 ecr 6529256], length 140 11:03:49.237987 IP 10.4.2.21.nfs > 10.3.22.172.2348092450: reply ok 132 readdir Expected results: Additional info:
Is this possible to reproduce from 'master' branch? while using FUSE on OSX? - there has been know problems with Mac NFS client
I also verified NFS does not work on version 3.3.2-2 of gluster.
I have not tested FUSE on OSX. Is there a pre built version of gluster for OSX?
(In reply to E3GH75 from comment #3) > I have not tested FUSE on OSX. > > Is there a pre built version of gluster for OSX? There isn't one, we are planning to support OSX for GlusterFS 3.6 release - and currently 'master' branch has all the fixes necessary to get it running.
okay so I should build gluster from 'master' on github for the servers and then attempt NFS test again?
Yep, that's worth trying. As a note, GitHub only has a mirror of the GlusterFS source code. It sometimes gets a bit behind, but (just checked) it's fine at the moment, so you're good to use that. :)
built and installed 3.7dev NFS no longer works... [2014-08-11 17:21:38.621760] E [nfs3.c:840:nfs3_getattr] 0-nfs-nfsv3: Bad Handle [2014-08-11 17:21:38.621801] W [nfs3-helpers.c:3401:nfs3_log_common_res] 0-nfs-nfsv3: XID: 8bf55f4f, GETATTR: NFS: 10001(Illegal NFS file handle), POSIX: 14(Bad address) [2014-08-11 17:21:38.621878] E [nfs3.c:301:__nfs3_get_volume_id] (-->/usr/lib64/glusterfs/3.7dev/xlator/nfs/server.so(nfs3_getattr+0x1d4) [0x7fb6bd730654] (-->/usr/lib64/glusterfs/3.7dev/xlator/nfs/server.so(nfs3_getattr_reply+0x29) [0x7fb6bd7266a9] (-->/usr/lib64/glusterfs/3.7dev/xlator/nfs/server.so(nfs3_request_xlator_deviceid+0x78) [0x7fb6bd7251e8]))) 0-nfs-nfsv3: invalid argument: xl
Sorry yeah, I misread what you meant. :( Would you be able to test: * Compile gluster from master for the servers, and * Compile gluster from master for the OSX clients In theory (!) this should let you do FUSE mounts instead of NFS. Once all of the bugs are ironed out, this should be both more reliable and have better performance. (NFS on OSX has a bad rep, and not just with Gluster ;>)
unfortunately I am not familiar with how to build this on OSX....
Same general approach as on Linux. $ git clone <url> $ ./autogen.sh $ configure $ make $ sudo make install That being said, if it goes wrong then the make install can leave bits in wrong places. As an easier thought, are you already familiar with OSX Homebrew? I've been working on a Homebrew formula for OSX for this. If that would be helpful, I can knock it into shape quickly and share it. ?
This is an initial Homebrew formula for OSX: https://github.com/justinclift/homebrew/commit/0a6af80642bd0ea3580be2fff94206b650d65269 Or you can grab it directly from here: https://github.com/justinclift/homebrew/raw/0a6af80642bd0ea3580be2fff94206b650d65269/Library/Formula/glusterfs.rb To use it, drop it into your Homebrew formula directory. Generally that's /usr/local/Library/Formula, then run: $ brew install glusterfs --HEAD The --HEAD option is important atm, as that tells Homebrew to grab the latest code from git master branch. :) Does this seem helpful?
configure: error: cmockery2 library is required to build glusterfs Any thoughts?
Arrrgh. I'd forgotten about that. I need to get the cmockery2 Homebrew formula done first. I'll jump on that now. Um... you might need to wait until tomorrow. :(
well is there source I can compile from or have I let the goose out of the cage?
Heh Heh Heh, the cmockery2 source is here: https://github.com/lpabon/cmockery2 I'm not that familiar with it (yet), but Luis (the main author) is fairly responsive if you hit issues. :)
got it to build! :-)
Building gluster failed... xdr-generic.c:27:29: error: too few arguments to function call, expected 3, have 2 if (!proc (&xdr, res)) { ~~~~ ^ xdr-generic.c:51:30: error: too few arguments to function call, expected 3, have 2 if (!proc (&xdr, args)) { ~~~~ ^ xdr-generic.c:75:30: error: too few arguments to function call, expected 3, have 2 if (!proc (&xdr, args)) { ~~~~ ^ 3 errors generated. make[4]: *** [libgfxdr_la-xdr-generic.lo] Error 1 make[3]: *** [all-recursive] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2
Harsha, any ideas? :) + Justin
(In reply to Justin Clift from comment #18) > Harsha, any ideas? :) > Yes in OSX 'proc()' doesn't take variable arguments as in BSD* or Linux. Also this should be just a warning? are you compiling this with "-Werror" ?
no I will try that but where in the command list to I add that flag?
I tried export CFLAGS="-Werror" but it still errors out when I run make
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory --quiet all-recursive Making all in ./contrib/argp-standalone Making all in . Making all in libglusterfs Making all in src Making all in rpc Making all in xdr Making all in src CC libgfxdr_la-xdr-generic.lo xdr-generic.c:27:29: error: too few arguments to function call, expected 3, have 2 if (!proc (&xdr, res)) { ~~~~ ^ xdr-generic.c:51:30: error: too few arguments to function call, expected 3, have 2 if (!proc (&xdr, args)) { ~~~~ ^ xdr-generic.c:75:30: error: too few arguments to function call, expected 3, have 2 if (!proc (&xdr, args)) { ~~~~ ^ 3 errors generated. make[4]: *** [libgfxdr_la-xdr-generic.lo] Error 1 make[3]: *** [all-recursive] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2
I think Harsha was saying that using -Werror will cause the compile to fail, and you should try to avoid it. It sounds like your compiler setting might be pulling it in automatically (thus the failure), and maybe try to stop it doing that? :)
> xdr-generic.c:27:29: error: too few arguments to function call, expected 3, > have 2 > if (!proc (&xdr, res)) { > ~~~~ ^ > xdr-generic.c:51:30: error: too few arguments to function call, expected 3, > have 2 > if (!proc (&xdr, args)) { > ~~~~ ^ > xdr-generic.c:75:30: error: too few arguments to function call, expected 3, > have 2 > if (!proc (&xdr, args)) { > ~~~~ ^ > 3 errors generated. > make[4]: *** [libgfxdr_la-xdr-generic.lo] Error 1 > make[3]: *** [all-recursive] Error 1 > make[2]: *** [all-recursive] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 Okay - i think the problem doesn't occur on 10.8.5 version - it occurs on 10.9 > - RPC code is changed and made stricter. It was fixed before, but due to some other porting efforts it was reverted for NetBSD by manu - ~~~~commit 618d465295df02ae6d53be1327947a210bb8b47d Author: Emmanuel Dreyfus <manu> Date: Fri May 16 16:58:20 2014 +0200 NetBSD build fixes - Shell scripts: == is specific to bash and ksh. Use = instead. - Shell scripts: use sh instead of bash if bash functionnality is not used - Shell scripts: ${var/search/replace} is specific to bash - sed: The -i option is specific to GNU sed. - Makefiles: $< outside of generic rules only work in GNU make. - xdrproc_t() is not universally defined as variadic. Do not specify third argument if it is not used - NetBSD FUSE specific: only include <perfuse.h> in FUSE client code, it harms in other locations - configure: Search for gettext() in libintl as NetBSD stores it there - Like MacOS X, NetBSD has unmount(2) and not umount(2) (un vs u) Some other build issues previously included in this change were removed: - __THROW macro, addressed in http://review.gluster.com/#/c/7757/ - getmntent() compat shared with MacOS X, in http://review.gluster.com/#/c/7722/ This patchset adds warning fixes for mount_glusterfs BUG: 764655 Change-Id: I2f1faf8ff96362d3e2baf237b943df619011f1f4 Signed-off-by: Emmanuel Dreyfus <manu> Reviewed-on: http://review.gluster.org/7783 Tested-by: Gluster Build System <jenkins.com> Reviewed-by: Harshavardhana <harsha> ~~~~ Especially this change - xdrproc_t() is not universally defined as variadic. Do not specify third argument if it is not used On OSX 10.8.5 ~~~~~~~~~~~ /* * A xdrproc_t exists for each data type which is to be encoded or decoded. * * The second argument to the xdrproc_t is a pointer to an opaque pointer. * The opaque pointer generally points to a structure of the data type * to be decoded. If this pointer is 0, then the type routines should * allocate dynamic storage of the appropriate size and return it. */ #ifdef _KERNEL typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int); #else /* * XXX can't actually prototype it, because some take three args!!! */ typedef bool_t (*xdrproc_t)(XDR *, ...); #endif ~~~~~~~~~~~ On Linux ~~~~~~~~~~~ /* * A xdrproc_t exists for each data type which is to be encoded or decoded. * * The second argument to the xdrproc_t is a pointer to an opaque pointer. * The opaque pointer generally points to a structure of the data type * to be decoded. If this pointer is 0, then the type routines should * allocate dynamic storage of the appropriate size and return it. * bool_t (*xdrproc_t)(XDR *, caddr_t *); */ typedef bool_t (*xdrproc_t) (XDR *, void *,...); ~~~~~~~~~~~~ On OSX 10.9.4 ~~~~~~~~~~~~ /* * A xdrproc_t exists for each data type which is to be encoded or decoded. * * The second argument to the xdrproc_t is a pointer to an opaque pointer. * The opaque pointer generally points to a structure of the data type * to be decoded. If this pointer is 0, then the type routines should * allocate dynamic storage of the appropriate size and return it. * * * IMPORTANT NOTE: Apple iOS and OS X * * Previous versions of this header file defined the xdrproc_t prototype as * typedef bool_t (*xdrproc_t)(XDR *, ...); * * This prototype is incorrect. One may not use a varargs function pointer * in place of pointer to a function with positional parameters. * This mistake has been masked by the fact that clients of this API would * generally cast their xdr functions as (xdrproc_t), and thus silence compiler * warnings. The code worked because the ABI for varargs routines that pass a * small number of arguments has been the same as the ABI for routines with a * few positional parameters. However, if the ABI differs this will cause the * compiled code to fail. * * Historically, some client xdr routines expected three parameters. This * does not seem to be the case in more recent code, but we have decided to * retain this definition in the XDR library in case some legacy code still * expects three parameters. The library will pass zero as the third * parameter. Routines that expect two parameters will work correctly. * * If your client-side xdr routine is of the form: * bool_t xdr_my_datatype(XDR *, void *); * and you pass your function pointer to routines like xdr_pointer or * xdr_reference using "(xdrproc_t)xdr_my_datatype", then your code will * compile and run correctly. If your code invokes an xdrproc_t callback, * it must be modified to pass a third parameter, which may simply be zero. */ typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int); ~~~~~~~~~~~~~ A terrible mistake by OSX folks!
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#1) for review on master by Harshavardhana (harsha)
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#2) for review on master by Harshavardhana (harsha)
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#3) for review on master by Harshavardhana (harsha)
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#4) for review on master by Harshavardhana (harsha)
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#5) for review on master by Harshavardhana (harsha)
I appreciate all the hard work you guys are doing to fix the Fuze mount for native osx support but what can be done to fix the nfs issues. The compiled version I made yesterday totally broke all nfs support. I can no longer access the volume from my linux systems. It's complaining that the rpcbind doesn't match and the transport layer is not supported. I can post logs as soon as get to the office. It looks like whatever change apple made to nfs in mavericks 10.9 really goofed stuff up. I don't have any 10.8 machines but does nfs work from those clients? Again thanks for all your help.
COMMIT: http://review.gluster.org/8458 committed in master by Harshavardhana (harsha) ------ commit fd6765b4a3f8162bf36054cf3de6e88a6bdfadd3 Author: Harshavardhana <harsha> Date: Mon Aug 11 17:36:12 2014 -0700 porting: OSX/Darwin 10.9 porting issues xdrproc_t() arguments are variadic and non-variadic On OSX > 10.9 ------------- typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int); On OSX < 10.9 ------------ typedef bool_t (*xdrproc_t)(XDR *, ...); FreeBSD all versions ------------ typedef bool_t (*xdrproc_t)(XDR *, ...); NetBSD 6.1.4 ----------- typedef bool_t (*xdrproc_t)(XDR *, const void *); Linux all versions ----------- typedef bool_t (*xdrproc_t)(XDR *, void *,...); This weird and odd implementations across various platforms should be handled properly. Change-Id: Iad8b7da2e5b82526bf3708cff31ab10ce09f59c9 BUG: 1128820 Signed-off-by: Harshavardhana <harsha> Reviewed-on: http://review.gluster.org/8458 Reviewed-by: Emmanuel Dreyfus <manu> Tested-by: Gluster Build System <jenkins.com>
> > It looks like whatever change apple made to nfs in mavericks 10.9 really > goofed stuff up. I don't have any 10.8 machines but does nfs work from those > clients? Again thanks for all your help. The real issue is really interoperability issues, there is nothing stopping us from fixing it. You should provide a tcpdump from linux server in this fashion # tcpdump -s0 -i <device> -w /tmp/tcpdump.cap host <mac_osx_client> Attach it back here, where we can see what is going on. Questions - are you not able to mount NFS even on Linux clients from master branch code? - Can you provide "sw_vers" output? - Can you attach server nfs.log to this bugzilla? Thanks -- Harsha
Trying to mount from MAC i get this from the logs... Is this a fuse issue? [2014-08-12 16:49:42.482455] I [MSGID: 100030] [glusterfsd.c:2021:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.7dev (args: /usr/local/sbin/glusterfs --volfile-server=10.4.2.21 --volfile-id=/glfs_vol01 /mnt/objstor01) [2014-08-12 16:49:42.483576] E [mount_darwin.c:249:gf_fuse_mount] 0-glusterfs-fuse: failed to open device (No such file or directory) [2014-08-12 16:49:42.483607] E [xlator.c:425:xlator_init] 0-fuse: Initialization of volume 'fuse' failed, review your volfile again
(In reply to Eric Horwitz from comment #33) > Trying to mount from MAC i get this from the logs... Is this a fuse issue? > > > [2014-08-12 16:49:42.482455] I [MSGID: 100030] [glusterfsd.c:2021:main] > 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs > version 3.7dev (args: /usr/local/sbin/glusterfs --volfile-server=10.4.2.21 > --volfile-id=/glfs_vol01 /mnt/objstor01) > [2014-08-12 16:49:42.483576] E [mount_darwin.c:249:gf_fuse_mount] > 0-glusterfs-fuse: failed to open device (No such file or directory) > [2014-08-12 16:49:42.483607] E [xlator.c:425:xlator_init] 0-fuse: > Initialization of volume 'fuse' failed, review your volfile again Did you install 'osxfuse' from brew?
yes I did brew install osxfuse Warning: osxfuse-2.7.0 already installed
sw_vers ProductName: Mac OS X ProductVersion: 10.9.2 BuildVersion: 13C64
gluster volume set nfs.disable yes then gluster volume set nfs.disable no allowed nfs to linux clients to work again....
bash-3.2# df -h /mnt/gfs Filesystem Size Used Avail Capacity iused ifree %iused Mounted on glusterfsd@osxfuse0 698Gi 394Gi 304Gi 57% 103367279 79566463 57% /mnt/gfs bash-3.2# sw_vers ProductName: Mac OS X ProductVersion: 10.9.4 BuildVersion: 13E28 bash-3.2# Not able to reproduce this behavior. Even the volume is created on the same box bash-3.2# gluster volume info Volume Name: p2p Type: Distribute Volume ID: d3eaa9bc-c5e9-4e29-948d-e9c17bfc3a85 Status: Started Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: 10.0.0.46:/mnt/gfs1 bash-3.2#
One thing to verify would be do you have?? # ls /dev/osxfuse* /dev/osxfuse0 /dev/osxfuse11 /dev/osxfuse14 /dev/osxfuse17 /dev/osxfuse2 /dev/osxfuse22 /dev/osxfuse4 /dev/osxfuse7 /dev/osxfuse1 /dev/osxfuse12 /dev/osxfuse15 /dev/osxfuse18 /dev/osxfuse20 /dev/osxfuse23 /dev/osxfuse5 /dev/osxfuse8 /dev/osxfuse10 /dev/osxfuse13 /dev/osxfuse16 /dev/osxfuse19 /dev/osxfuse21 /dev/osxfuse3 /dev/osxfuse6 /dev/osxfuse9
I do not have any of the osxfuse files in dev.... Maybe a reinstall of fuse?
that worked... I also needed to do the following.... sudo /bin/cp -RfX /usr/local/opt/osxfuse/Library/Filesystems/osxfusefs.fs /Library/Filesystems sudo chmod +s /Library/Filesystems/osxfusefs.fs/Support/load_osxfusefs
Glad it worked, let me know how it goes. FUSE based implementation should be much more reliable than using NFS IMHO. BTW do you happen to gather the tcpdump? and nfs.log it would help us look into NFS issue further.
Created attachment 926198 [details] Here is the nfs.log file
So I have some questions regarding performance. I have 4 machines with small RAID0 arrays that each get 150+MB/s throughput. This is setup with 2 stripes and 2 replicas. Everything is on 1Gbe. I am seeing 53MB/s write and 110MB/s read. Does this sound correct?
Created attachment 926199 [details] tcpdump of NFS mounted MAC 10.9 system I captured the logs before mounting the share via NFS and then tried to ls -l the path which just sits and hangs.
(In reply to Eric Horwitz from comment #44) > So I have some questions regarding performance. I have 4 machines with small > RAID0 arrays that each get 150+MB/s throughput. This is setup with 2 stripes > and 2 replicas. Everything is on 1Gbe. I am seeing 53MB/s write and 110MB/s > read. Does this sound correct? That sounds correct replicate takes away 50% of the bandwidth due to synchronous replication (hitting both the servers at the same time) - read is always served from one of the servers at a point in time. For more architectural discussion - you should move this discussion to 'gluster-users' list.
Okay so was able to mount and also was able to write a file but when i do a 'ls' the mount hangs bash-3.2# mount_nfs -o vers=3 10.14.19.108:/patchy /mnt/ gfs/ sshfs/ bash-3.2# mount_nfs -o vers=3 10.14.19.108:/patchy /mnt/gfs bash-3.2# cd /mnt/gfs bash-3.2# touch a bash-3.2# stat a 436207619 11617678988833378175 -rw-r--r-- 1 root wheel 0 0 "Aug 12 16:23:47 2014" "Aug 12 16:23:47 2014" "Aug 12 16:23:47 2014" "Aug 12 16:23:47 2014" 1048576 0 0 a bash-3.2# ls -l ^C All i can see is on Linux NFS server, so in-fact it looks like a READDIR bug it causes infinite loops. Must be some sort of an overflow which OSX NFS client doesn't understand - this would require some time to get it fixed. [2014-08-12 20:25:51.828257] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789414, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:51.931971] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789415, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:51.933740] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789415, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.032864] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789416, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.034827] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789416, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.195200] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789417, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.196712] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789417, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.301950] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789418, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.304043] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789418, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.398762] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789419, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.400700] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789419, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.503713] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941a, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.505680] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941a, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.624283] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941b, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.626307] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941b, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.775226] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941c, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.777282] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941c, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 [2014-08-12 20:25:52.909012] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941d, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768 [2014-08-12 20:25:52.910975] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941d, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0 -- Harsha
Created attachment 926220 [details] NFS ganesha configs I was able to get NFS client on MacOSX working with nfs-ganesha + FSAL_gluster support [1] Attached configs used for sample test. [1] - https://github.com/nfs-ganesha/nfs-ganesha/tree/master/src/FSAL/FSAL_GLUSTER I don't know if we have thought about deprecating GlusterNFS, but it would seem like NFS-Ganesha would be certainly the better approach.
REVIEW: http://review.gluster.org/8476 (porting: OSX/Darwin 10.9 porting issues) posted (#1) for review on dht-stale-layout-fixes by Raghavendra G (rgowdapp)
> I don't know if we have thought about deprecating GlusterNFS, but it > would seem like NFS-Ganesha would be certainly the better approach. Maybe start that discussion on gluster-devel or gluster-users mailing list?
(In reply to Anand Avati from comment #49) > REVIEW: http://review.gluster.org/8476 (porting: OSX/Darwin 10.9 porting > issues) posted (#1) for review on dht-stale-layout-fixes by Raghavendra G > (rgowdapp) This patch has been abandoned, it was probably not related anyway. I think this is a duplicate of bug 1132391. Could someone check/verify the patch from http://review.gluster.org/8509 ? It's been merged in the master branch already, so you could probably test it there too. Thanks, Niels
I'm now marking this as a duplicate of Bug 1132391. Please re-open if this does not fix the reported issue with Gluster/NFS and a stripe volume when mounting from OSX. *** This bug has been marked as a duplicate of bug 1132391 ***