Bug 1128820 - Unable to ls -l NFS mount from OSX 10.9 client on pool created with stripe
Summary: Unable to ls -l NFS mount from OSX 10.9 client on pool created with stripe
Keywords:
Status: CLOSED DUPLICATE of bug 1132391
Alias: None
Product: GlusterFS
Classification: Community
Component: nfs
Version: 3.5.2
Hardware: x86_64
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Harshavardhana
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-08-11 15:23 UTC by Eric Horwitz
Modified: 2015-03-23 01:04 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-09 07:26:31 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:


Attachments (Terms of Use)
Here is the nfs.log file (4.07 MB, text/plain)
2014-08-12 19:40 UTC, Eric Horwitz
no flags Details
tcpdump of NFS mounted MAC 10.9 system (3.29 MB, application/x-gzip)
2014-08-12 19:53 UTC, Eric Horwitz
no flags Details
NFS ganesha configs (725 bytes, application/gzip)
2014-08-12 21:31 UTC, Harshavardhana
no flags Details

Description Eric Horwitz 2014-08-11 15:23:41 UTC
Description of problem:

I am unable to ls -l the mount point from a MAC OSX 10.9 client when stripe is < 1. I can rsync a file to the mount point but I cannot list the directory and its contents.

Version-Release number of selected component (if applicable):3.5.2


How reproducible:Very


Steps to Reproduce:
1.Create a pool with stripe < 1
2.mount via NFS from MAC client
3.ls -l the mount point.

Actual results:
tcp dump from main brick 

11:03:49.159146 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2349793, win 8183, options [nop,nop,TS val 481967906 ecr 6529176], length 0
11:03:49.159217 IP 10.3.22.172.2348092412 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.159452 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627141:2627293, ack 2419701, win 6146, options [nop,nop,TS val 6529178 ecr 4917145], length 152
11:03:49.159636 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2419701:2419841, ack 2627293, win 3593, options [nop,nop,TS val 4917147 ecr 6529178], length 140
11:03:49.159690 IP 10.4.2.21.nfs > 10.3.22.172.2348092412: reply ok 132 readdir
11:03:49.160759 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2349929, win 8183, options [nop,nop,TS val 481967908 ecr 6529178], length 0
11:03:49.161312 IP 10.3.22.172.2348092413 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.161539 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627293:2627445, ack 2419841, win 6146, options [nop,nop,TS val 6529180 ecr 4917147], length 152
11:03:49.161743 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2419841:2419981, ack 2627445, win 3593, options [nop,nop,TS val 4917149 ecr 6529180], length 140
11:03:49.161785 IP 10.4.2.21.nfs > 10.3.22.172.2348092413: reply ok 132 readdir
11:03:49.162787 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350065, win 8183, options [nop,nop,TS val 481967910 ecr 6529180], length 0
11:03:49.163275 IP 10.3.22.172.2348092414 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.163500 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627445:2627597, ack 2419981, win 6146, options [nop,nop,TS val 6529182 ecr 4917149], length 152
11:03:49.163673 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2419981:2420121, ack 2627597, win 3593, options [nop,nop,TS val 4917151 ecr 6529182], length 140
11:03:49.163726 IP 10.4.2.21.nfs > 10.3.22.172.2348092414: reply ok 132 readdir
11:03:49.164710 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350201, win 8183, options [nop,nop,TS val 481967911 ecr 6529182], length 0
11:03:49.165168 IP 10.3.22.172.2348092415 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.165391 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627597:2627749, ack 2420121, win 6146, options [nop,nop,TS val 6529184 ecr 4917151], length 152
11:03:49.165602 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420121:2420261, ack 2627749, win 3593, options [nop,nop,TS val 4917153 ecr 6529184], length 140
11:03:49.165644 IP 10.4.2.21.nfs > 10.3.22.172.2348092415: reply ok 132 readdir
11:03:49.166615 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350337, win 8183, options [nop,nop,TS val 481967912 ecr 6529184], length 0
11:03:49.167114 IP 10.3.22.172.2348092416 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.167342 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627749:2627901, ack 2420261, win 6146, options [nop,nop,TS val 6529186 ecr 4917153], length 152
11:03:49.167525 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420261:2420401, ack 2627901, win 3593, options [nop,nop,TS val 4917155 ecr 6529186], length 140
11:03:49.167594 IP 10.4.2.21.nfs > 10.3.22.172.2348092416: reply ok 132 readdir
11:03:49.169256 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350473, win 8183, options [nop,nop,TS val 481967914 ecr 6529186], length 0
11:03:49.169643 IP 10.3.22.172.2348092417 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.169896 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2627901:2628053, ack 2420401, win 6146, options [nop,nop,TS val 6529188 ecr 4917155], length 152
11:03:49.170081 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420401:2420541, ack 2628053, win 3593, options [nop,nop,TS val 4917157 ecr 6529188], length 140
11:03:49.170112 IP 10.4.2.21.nfs > 10.3.22.172.2348092417: reply ok 132 readdir
11:03:49.171074 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350609, win 8183, options [nop,nop,TS val 481967915 ecr 6529188], length 0
11:03:49.171548 IP 10.3.22.172.2348092418 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.171779 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628053:2628205, ack 2420541, win 6146, options [nop,nop,TS val 6529190 ecr 4917157], length 152
11:03:49.171953 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420541:2420681, ack 2628205, win 3593, options [nop,nop,TS val 4917159 ecr 6529190], length 140
11:03:49.172005 IP 10.4.2.21.nfs > 10.3.22.172.2348092418: reply ok 132 readdir
11:03:49.172994 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350745, win 8183, options [nop,nop,TS val 481967916 ecr 6529190], length 0
11:03:49.173452 IP 10.3.22.172.2348092419 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.173686 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628205:2628357, ack 2420681, win 6146, options [nop,nop,TS val 6529192 ecr 4917159], length 152
11:03:49.173882 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420681:2420821, ack 2628357, win 3593, options [nop,nop,TS val 4917161 ecr 6529192], length 140
11:03:49.173921 IP 10.4.2.21.nfs > 10.3.22.172.2348092419: reply ok 132 readdir
11:03:49.174917 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2350881, win 8183, options [nop,nop,TS val 481967917 ecr 6529192], length 0
11:03:49.175441 IP 10.3.22.172.2348092420 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.175687 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628357:2628509, ack 2420821, win 6146, options [nop,nop,TS val 6529194 ecr 4917161], length 152
11:03:49.175870 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420821:2420961, ack 2628509, win 3593, options [nop,nop,TS val 4917163 ecr 6529194], length 140
11:03:49.175905 IP 10.4.2.21.nfs > 10.3.22.172.2348092420: reply ok 132 readdir
11:03:49.176866 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351017, win 8183, options [nop,nop,TS val 481967918 ecr 6529194], length 0
11:03:49.177450 IP 10.3.22.172.2348092421 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.177680 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628509:2628661, ack 2420961, win 6146, options [nop,nop,TS val 6529196 ecr 4917163], length 152
11:03:49.177892 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2420961:2421101, ack 2628661, win 3593, options [nop,nop,TS val 4917165 ecr 6529196], length 140
11:03:49.177929 IP 10.4.2.21.nfs > 10.3.22.172.2348092421: reply ok 132 readdir
11:03:49.179705 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351153, win 8183, options [nop,nop,TS val 481967919 ecr 6529196], length 0
11:03:49.179774 IP 10.3.22.172.2348092422 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.179998 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628661:2628813, ack 2421101, win 6146, options [nop,nop,TS val 6529198 ecr 4917165], length 152
11:03:49.180174 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421101:2421241, ack 2628813, win 3593, options [nop,nop,TS val 4917168 ecr 6529198], length 140
11:03:49.180232 IP 10.4.2.21.nfs > 10.3.22.172.2348092422: reply ok 132 readdir
11:03:49.181306 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351289, win 8183, options [nop,nop,TS val 481967921 ecr 6529198], length 0
11:03:49.181851 IP 10.3.22.172.2348092423 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.182110 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628813:2628965, ack 2421241, win 6146, options [nop,nop,TS val 6529200 ecr 4917168], length 152
11:03:49.182309 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421241:2421381, ack 2628965, win 3593, options [nop,nop,TS val 4917170 ecr 6529200], length 140
11:03:49.182339 IP 10.4.2.21.nfs > 10.3.22.172.2348092423: reply ok 132 readdir
11:03:49.183303 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351425, win 8183, options [nop,nop,TS val 481967923 ecr 6529201], length 0
11:03:49.183834 IP 10.3.22.172.2348092424 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.184057 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2628965:2629117, ack 2421381, win 6146, options [nop,nop,TS val 6529202 ecr 4917170], length 152
11:03:49.184270 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421381:2421521, ack 2629117, win 3593, options [nop,nop,TS val 4917172 ecr 6529202], length 140
11:03:49.184307 IP 10.4.2.21.nfs > 10.3.22.172.2348092424: reply ok 132 readdir
11:03:49.185280 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351561, win 8183, options [nop,nop,TS val 481967924 ecr 6529202], length 0
11:03:49.185807 IP 10.3.22.172.2348092425 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.186032 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629117:2629269, ack 2421521, win 6146, options [nop,nop,TS val 6529204 ecr 4917172], length 152
11:03:49.186220 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421521:2421661, ack 2629269, win 3593, options [nop,nop,TS val 4917174 ecr 6529204], length 140
11:03:49.186264 IP 10.4.2.21.nfs > 10.3.22.172.2348092425: reply ok 132 readdir
11:03:49.187311 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351697, win 8183, options [nop,nop,TS val 481967925 ecr 6529204], length 0
11:03:49.187796 IP 10.3.22.172.2348092426 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.188049 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629269:2629421, ack 2421661, win 6146, options [nop,nop,TS val 6529206 ecr 4917174], length 152
11:03:49.188222 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421661:2421801, ack 2629421, win 3593, options [nop,nop,TS val 4917176 ecr 6529206], length 140
11:03:49.188252 IP 10.4.2.21.nfs > 10.3.22.172.2348092426: reply ok 132 readdir
11:03:49.189760 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351833, win 8183, options [nop,nop,TS val 481967927 ecr 6529206], length 0
11:03:49.190317 IP 10.3.22.172.2348092427 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.190569 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629421:2629573, ack 2421801, win 6146, options [nop,nop,TS val 6529209 ecr 4917176], length 152
11:03:49.190771 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421801:2421941, ack 2629573, win 3593, options [nop,nop,TS val 4917178 ecr 6529209], length 140
11:03:49.190801 IP 10.4.2.21.nfs > 10.3.22.172.2348092427: reply ok 132 readdir
11:03:49.191705 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2351969, win 8183, options [nop,nop,TS val 481967928 ecr 6529209], length 0
11:03:49.192301 IP 10.3.22.172.2348092428 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.192530 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629573:2629725, ack 2421941, win 6146, options [nop,nop,TS val 6529211 ecr 4917178], length 152
11:03:49.192736 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2421941:2422081, ack 2629725, win 3593, options [nop,nop,TS val 4917180 ecr 6529211], length 140
11:03:49.192774 IP 10.4.2.21.nfs > 10.3.22.172.2348092428: reply ok 132 readdir
11:03:49.193711 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352105, win 8183, options [nop,nop,TS val 481967929 ecr 6529211], length 0
11:03:49.194092 IP 10.3.22.172.2348092429 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.194316 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629725:2629877, ack 2422081, win 6146, options [nop,nop,TS val 6529212 ecr 4917180], length 152
11:03:49.194507 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422081:2422221, ack 2629877, win 3593, options [nop,nop,TS val 4917182 ecr 6529212], length 140
11:03:49.194552 IP 10.4.2.21.nfs > 10.3.22.172.2348092429: reply ok 132 readdir
11:03:49.195396 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352241, win 8183, options [nop,nop,TS val 481967930 ecr 6529213], length 0
11:03:49.195814 IP 10.3.22.172.2348092430 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.196042 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2629877:2630029, ack 2422221, win 6146, options [nop,nop,TS val 6529214 ecr 4917182], length 152
11:03:49.196211 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422221:2422361, ack 2630029, win 3593, options [nop,nop,TS val 4917184 ecr 6529214], length 140
11:03:49.196256 IP 10.4.2.21.nfs > 10.3.22.172.2348092430: reply ok 132 readdir
11:03:49.197129 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352377, win 8183, options [nop,nop,TS val 481967931 ecr 6529214], length 0
11:03:49.197730 IP 10.3.22.172.2348092431 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.197954 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630029:2630181, ack 2422361, win 6146, options [nop,nop,TS val 6529216 ecr 4917184], length 152
11:03:49.198168 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422361:2422501, ack 2630181, win 3593, options [nop,nop,TS val 4917185 ecr 6529216], length 140
11:03:49.198207 IP 10.4.2.21.nfs > 10.3.22.172.2348092431: reply ok 132 readdir
11:03:49.199583 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352513, win 8183, options [nop,nop,TS val 481967933 ecr 6529216], length 0
11:03:49.199989 IP 10.3.22.172.2348092432 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.200211 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630181:2630333, ack 2422501, win 6146, options [nop,nop,TS val 6529218 ecr 4917185], length 152
11:03:49.200422 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422501:2422641, ack 2630333, win 3593, options [nop,nop,TS val 4917188 ecr 6529218], length 140
11:03:49.200464 IP 10.4.2.21.nfs > 10.3.22.172.2348092432: reply ok 132 readdir
11:03:49.201534 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352649, win 8183, options [nop,nop,TS val 481967934 ecr 6529219], length 0
11:03:49.202065 IP 10.3.22.172.2348092433 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.202295 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630333:2630485, ack 2422641, win 6146, options [nop,nop,TS val 6529220 ecr 4917188], length 152
11:03:49.202478 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422641:2422781, ack 2630485, win 3593, options [nop,nop,TS val 4917190 ecr 6529220], length 140
11:03:49.202533 IP 10.4.2.21.nfs > 10.3.22.172.2348092433: reply ok 132 readdir
11:03:49.203507 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352785, win 8183, options [nop,nop,TS val 481967935 ecr 6529221], length 0
11:03:49.204900 IP 10.3.22.172.2348092434 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.205124 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630485:2630637, ack 2422781, win 6146, options [nop,nop,TS val 6529223 ecr 4917190], length 152
11:03:49.205353 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422781:2422921, ack 2630637, win 3593, options [nop,nop,TS val 4917193 ecr 6529223], length 140
11:03:49.205404 IP 10.4.2.21.nfs > 10.3.22.172.2348092434: reply ok 132 readdir
11:03:49.206380 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2352921, win 8183, options [nop,nop,TS val 481967937 ecr 6529224], length 0
11:03:49.206967 IP 10.3.22.172.2348092435 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.207189 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630637:2630789, ack 2422921, win 6146, options [nop,nop,TS val 6529225 ecr 4917193], length 152
11:03:49.207371 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2422921:2423061, ack 2630789, win 3593, options [nop,nop,TS val 4917195 ecr 6529225], length 140
11:03:49.207437 IP 10.4.2.21.nfs > 10.3.22.172.2348092435: reply ok 132 readdir
11:03:49.209500 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353057, win 8183, options [nop,nop,TS val 481967939 ecr 6529226], length 0
11:03:49.209960 IP 10.3.22.172.2348092436 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.210185 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630789:2630941, ack 2423061, win 6146, options [nop,nop,TS val 6529228 ecr 4917195], length 152
11:03:49.210374 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423061:2423201, ack 2630941, win 3593, options [nop,nop,TS val 4917198 ecr 6529228], length 140
11:03:49.210433 IP 10.4.2.21.nfs > 10.3.22.172.2348092436: reply ok 132 readdir
11:03:49.211308 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353193, win 8183, options [nop,nop,TS val 481967940 ecr 6529229], length 0
11:03:49.211806 IP 10.3.22.172.2348092437 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.212060 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2630941:2631093, ack 2423201, win 6146, options [nop,nop,TS val 6529230 ecr 4917198], length 152
11:03:49.212229 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423201:2423341, ack 2631093, win 3593, options [nop,nop,TS val 4917200 ecr 6529230], length 140
11:03:49.212260 IP 10.4.2.21.nfs > 10.3.22.172.2348092437: reply ok 132 readdir
11:03:49.213510 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353329, win 8183, options [nop,nop,TS val 481967941 ecr 6529230], length 0
11:03:49.213774 IP 10.3.22.172.2348092438 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.213996 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631093:2631245, ack 2423341, win 6146, options [nop,nop,TS val 6529232 ecr 4917200], length 152
11:03:49.214213 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423341:2423481, ack 2631245, win 3593, options [nop,nop,TS val 4917202 ecr 6529232], length 140
11:03:49.214248 IP 10.4.2.21.nfs > 10.3.22.172.2348092438: reply ok 132 readdir
11:03:49.215168 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353465, win 8183, options [nop,nop,TS val 481967942 ecr 6529232], length 0
11:03:49.215586 IP 10.3.22.172.2348092439 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.215810 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631245:2631397, ack 2423481, win 6146, options [nop,nop,TS val 6529234 ecr 4917202], length 152
11:03:49.216000 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423481:2423621, ack 2631397, win 3593, options [nop,nop,TS val 4917203 ecr 6529234], length 140
11:03:49.216059 IP 10.4.2.21.nfs > 10.3.22.172.2348092439: reply ok 132 readdir
11:03:49.217245 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353601, win 8183, options [nop,nop,TS val 481967943 ecr 6529234], length 0
11:03:49.217674 IP 10.3.22.172.2348092440 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.217929 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631397:2631549, ack 2423621, win 6146, options [nop,nop,TS val 6529236 ecr 4917203], length 152
11:03:49.218111 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423621:2423761, ack 2631549, win 3593, options [nop,nop,TS val 4917205 ecr 6529236], length 140
11:03:49.218142 IP 10.4.2.21.nfs > 10.3.22.172.2348092440: reply ok 132 readdir
11:03:49.219518 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353737, win 8183, options [nop,nop,TS val 481967945 ecr 6529236], length 0
11:03:49.220006 IP 10.3.22.172.2348092441 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.220230 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631549:2631701, ack 2423761, win 6146, options [nop,nop,TS val 6529238 ecr 4917205], length 152
11:03:49.220398 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423761:2423901, ack 2631701, win 3593, options [nop,nop,TS val 4917208 ecr 6529238], length 140
11:03:49.220448 IP 10.4.2.21.nfs > 10.3.22.172.2348092441: reply ok 132 readdir
11:03:49.221261 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2353873, win 8183, options [nop,nop,TS val 481967946 ecr 6529239], length 0
11:03:49.221770 IP 10.3.22.172.2348092442 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.221994 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631701:2631853, ack 2423901, win 6146, options [nop,nop,TS val 6529240 ecr 4917208], length 152
11:03:49.222213 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2423901:2424041, ack 2631853, win 3593, options [nop,nop,TS val 4917210 ecr 6529240], length 140
11:03:49.222256 IP 10.4.2.21.nfs > 10.3.22.172.2348092442: reply ok 132 readdir
11:03:49.223312 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354009, win 8183, options [nop,nop,TS val 481967948 ecr 6529240], length 0
11:03:49.223805 IP 10.3.22.172.2348092443 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.224058 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2631853:2632005, ack 2424041, win 6146, options [nop,nop,TS val 6529242 ecr 4917210], length 152
11:03:49.224244 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424041:2424181, ack 2632005, win 3593, options [nop,nop,TS val 4917212 ecr 6529242], length 140
11:03:49.224275 IP 10.4.2.21.nfs > 10.3.22.172.2348092443: reply ok 132 readdir
11:03:49.225248 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354145, win 8183, options [nop,nop,TS val 481967949 ecr 6529242], length 0
11:03:49.225770 IP 10.3.22.172.2348092444 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.225993 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632005:2632157, ack 2424181, win 6146, options [nop,nop,TS val 6529244 ecr 4917212], length 152
11:03:49.226205 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424181:2424321, ack 2632157, win 3593, options [nop,nop,TS val 4917214 ecr 6529244], length 140
11:03:49.226239 IP 10.4.2.21.nfs > 10.3.22.172.2348092444: reply ok 132 readdir
11:03:49.227352 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354281, win 8183, options [nop,nop,TS val 481967950 ecr 6529244], length 0
11:03:49.227885 IP 10.3.22.172.2348092445 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.228109 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632157:2632309, ack 2424321, win 6146, options [nop,nop,TS val 6529246 ecr 4917214], length 152
11:03:49.228277 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424321:2424461, ack 2632309, win 3593, options [nop,nop,TS val 4917216 ecr 6529246], length 140
11:03:49.228325 IP 10.4.2.21.nfs > 10.3.22.172.2348092445: reply ok 132 readdir
11:03:49.229665 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354417, win 8183, options [nop,nop,TS val 481967952 ecr 6529246], length 0
11:03:49.230132 IP 10.3.22.172.2348092446 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.230368 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632309:2632461, ack 2424461, win 6146, options [nop,nop,TS val 6529249 ecr 4917216], length 152
11:03:49.230565 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424461:2424601, ack 2632461, win 3593, options [nop,nop,TS val 4917218 ecr 6529249], length 140
11:03:49.230604 IP 10.4.2.21.nfs > 10.3.22.172.2348092446: reply ok 132 readdir
11:03:49.231705 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354553, win 8183, options [nop,nop,TS val 481967953 ecr 6529249], length 0
11:03:49.232072 IP 10.3.22.172.2348092447 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.232295 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632461:2632613, ack 2424601, win 6146, options [nop,nop,TS val 6529250 ecr 4917218], length 152
11:03:49.232485 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424601:2424741, ack 2632613, win 3593, options [nop,nop,TS val 4917220 ecr 6529250], length 140
11:03:49.232529 IP 10.4.2.21.nfs > 10.3.22.172.2348092447: reply ok 132 readdir
11:03:49.233546 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354689, win 8183, options [nop,nop,TS val 481967954 ecr 6529251], length 0
11:03:49.233850 IP 10.3.22.172.2348092448 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.234073 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632613:2632765, ack 2424741, win 6146, options [nop,nop,TS val 6529252 ecr 4917220], length 152
11:03:49.234262 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424741:2424881, ack 2632765, win 3593, options [nop,nop,TS val 4917222 ecr 6529252], length 140
11:03:49.234320 IP 10.4.2.21.nfs > 10.3.22.172.2348092448: reply ok 132 readdir
11:03:49.235220 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354825, win 8183, options [nop,nop,TS val 481967955 ecr 6529252], length 0
11:03:49.235699 IP 10.3.22.172.2348092449 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.235952 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632765:2632917, ack 2424881, win 6146, options [nop,nop,TS val 6529254 ecr 4917222], length 152
11:03:49.236119 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2424881:2425021, ack 2632917, win 3593, options [nop,nop,TS val 4917223 ecr 6529254], length 140
11:03:49.236149 IP 10.4.2.21.nfs > 10.3.22.172.2348092449: reply ok 132 readdir
11:03:49.237030 IP 10.3.22.172.62536 > 10.4.2.21.nfs: Flags [.], ack 2354961, win 8183, options [nop,nop,TS val 481967956 ecr 6529254], length 0
11:03:49.237487 IP 10.3.22.172.2348092450 > 10.4.2.21.nfs: 176 readdir fh Unknown/3A4F474C7051C60366384C1082D8BD98149927FA000000000000000000000000 32768 bytes @ 512
11:03:49.237738 IP 10.4.2.21.1007 > 10.4.2.24.49154: Flags [P.], seq 2632917:2633069, ack 2425021, win 6146, options [nop,nop,TS val 6529256 ecr 4917223], length 152
11:03:49.237936 IP 10.4.2.24.49154 > 10.4.2.21.1007: Flags [P.], seq 2425021:2425161, ack 2633069, win 3593, options [nop,nop,TS val 4917225 ecr 6529256], length 140
11:03:49.237987 IP 10.4.2.21.nfs > 10.3.22.172.2348092450: reply ok 132 readdir


Expected results:


Additional info:

Comment 1 Harshavardhana 2014-08-11 16:01:35 UTC
Is this possible to reproduce from 'master' branch? while using FUSE on OSX? - there has been know problems with Mac NFS client

Comment 2 Eric Horwitz 2014-08-11 16:10:34 UTC
I also verified NFS does not work on version 3.3.2-2 of gluster.

Comment 3 Eric Horwitz 2014-08-11 16:17:16 UTC
I have not tested FUSE on OSX. 

Is there a pre built version of gluster for OSX?

Comment 4 Harshavardhana 2014-08-11 16:29:35 UTC
(In reply to E3GH75 from comment #3)
> I have not tested FUSE on OSX. 
> 
> Is there a pre built version of gluster for OSX?

There isn't one, we are planning to support OSX for GlusterFS 3.6 release - and currently 'master' branch has all the fixes necessary to get it running.

Comment 5 Eric Horwitz 2014-08-11 16:36:30 UTC
okay so I should build gluster from 'master' on github for the servers and then attempt NFS test again?

Comment 6 Justin Clift 2014-08-11 17:27:02 UTC
Yep, that's worth trying.

As a note, GitHub only has a mirror of the GlusterFS source code.  It sometimes gets a bit behind, but (just checked) it's fine at the moment, so you're good to use that. :)

Comment 7 Eric Horwitz 2014-08-11 17:30:29 UTC
built and installed 3.7dev NFS no longer works...

[2014-08-11 17:21:38.621760] E [nfs3.c:840:nfs3_getattr] 0-nfs-nfsv3: Bad Handle
[2014-08-11 17:21:38.621801] W [nfs3-helpers.c:3401:nfs3_log_common_res] 0-nfs-nfsv3: XID: 8bf55f4f, GETATTR: NFS: 10001(Illegal NFS file handle), POSIX: 14(Bad address)
[2014-08-11 17:21:38.621878] E [nfs3.c:301:__nfs3_get_volume_id] (-->/usr/lib64/glusterfs/3.7dev/xlator/nfs/server.so(nfs3_getattr+0x1d4) [0x7fb6bd730654] (-->/usr/lib64/glusterfs/3.7dev/xlator/nfs/server.so(nfs3_getattr_reply+0x29) [0x7fb6bd7266a9] (-->/usr/lib64/glusterfs/3.7dev/xlator/nfs/server.so(nfs3_request_xlator_deviceid+0x78) [0x7fb6bd7251e8]))) 0-nfs-nfsv3: invalid argument: xl

Comment 8 Justin Clift 2014-08-11 17:32:02 UTC
Sorry yeah, I misread what you meant. :(

Would you be able to test:

* Compile gluster from master for the servers, and
* Compile gluster from master for the OSX clients

In theory (!) this should let you do FUSE mounts instead of NFS.  Once all of the bugs are ironed out, this should be both more reliable and have better performance.  (NFS on OSX has a bad rep, and not just with Gluster ;>)

Comment 9 Eric Horwitz 2014-08-11 17:40:19 UTC
unfortunately I am not familiar with how to build this on OSX....

Comment 10 Justin Clift 2014-08-11 17:45:30 UTC
Same general approach as on Linux.

  $ git clone <url>
  $ ./autogen.sh
  $ configure
  $ make
  $ sudo make install

That being said, if it goes wrong then the make install can leave bits in wrong places.

As an easier thought, are you already familiar with OSX Homebrew?  I've been working on a Homebrew formula for OSX for this.  If that would be helpful, I can knock it into shape quickly and share it.

?

Comment 11 Justin Clift 2014-08-11 18:07:43 UTC
This is an initial Homebrew formula for OSX:

  https://github.com/justinclift/homebrew/commit/0a6af80642bd0ea3580be2fff94206b650d65269

Or you can grab it directly from here:

  https://github.com/justinclift/homebrew/raw/0a6af80642bd0ea3580be2fff94206b650d65269/Library/Formula/glusterfs.rb

To use it, drop it into your Homebrew formula directory.  Generally that's /usr/local/Library/Formula, then run:

  $ brew install glusterfs --HEAD

The --HEAD option is important atm, as that tells Homebrew to grab the latest code from git master branch. :)

Does this seem helpful?

Comment 12 Eric Horwitz 2014-08-11 18:40:07 UTC
configure: error: cmockery2 library is required to build glusterfs

Any thoughts?

Comment 13 Justin Clift 2014-08-11 18:42:19 UTC
Arrrgh.  I'd forgotten about that.  I need to get the cmockery2 Homebrew formula done first.  I'll jump on that now.

Um... you might need to wait until tomorrow. :(

Comment 14 Eric Horwitz 2014-08-11 18:46:54 UTC
well is there source I can compile from or have I let the goose out of the cage?

Comment 15 Justin Clift 2014-08-11 18:51:19 UTC
Heh Heh Heh, the cmockery2 source is here:

  https://github.com/lpabon/cmockery2

I'm not that familiar with it (yet), but Luis (the main author) is fairly responsive if you hit issues. :)

Comment 16 Eric Horwitz 2014-08-11 18:51:50 UTC
got it to build! :-)

Comment 17 Eric Horwitz 2014-08-11 18:54:19 UTC
Building gluster failed...


xdr-generic.c:27:29: error: too few arguments to function call, expected 3, have 2
        if (!proc (&xdr, res)) {
             ~~~~           ^
xdr-generic.c:51:30: error: too few arguments to function call, expected 3, have 2
        if (!proc (&xdr, args)) {
             ~~~~            ^
xdr-generic.c:75:30: error: too few arguments to function call, expected 3, have 2
        if (!proc (&xdr, args)) {
             ~~~~            ^
3 errors generated.
make[4]: *** [libgfxdr_la-xdr-generic.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Comment 18 Justin Clift 2014-08-11 19:04:01 UTC
Harsha, any ideas? :)

+ Justin

Comment 19 Harshavardhana 2014-08-11 20:03:52 UTC
(In reply to Justin Clift from comment #18)
> Harsha, any ideas? :)
> 

Yes in OSX 'proc()' doesn't take variable arguments as in BSD* or Linux. Also this should be just a warning? are you compiling this with "-Werror" ?

Comment 20 Eric Horwitz 2014-08-11 20:11:17 UTC
no I will try that but where in the command list to I add that flag?

Comment 21 Eric Horwitz 2014-08-11 20:17:35 UTC
I tried export CFLAGS="-Werror" but it still errors out when I run make

Comment 22 Eric Horwitz 2014-08-11 20:23:36 UTC
/Applications/Xcode.app/Contents/Developer/usr/bin/make --no-print-directory --quiet all-recursive
Making all in ./contrib/argp-standalone
Making all in .
Making all in libglusterfs
Making all in src
Making all in rpc
Making all in xdr
Making all in src
  CC       libgfxdr_la-xdr-generic.lo
xdr-generic.c:27:29: error: too few arguments to function call, expected 3, have 2
        if (!proc (&xdr, res)) {
             ~~~~           ^
xdr-generic.c:51:30: error: too few arguments to function call, expected 3, have 2
        if (!proc (&xdr, args)) {
             ~~~~            ^
xdr-generic.c:75:30: error: too few arguments to function call, expected 3, have 2
        if (!proc (&xdr, args)) {
             ~~~~            ^
3 errors generated.
make[4]: *** [libgfxdr_la-xdr-generic.lo] Error 1
make[3]: *** [all-recursive] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2

Comment 23 Justin Clift 2014-08-11 20:36:43 UTC
I think Harsha was saying that using -Werror will cause the compile to fail, and you should try to avoid it.

It sounds like your compiler setting might be pulling it in automatically (thus the failure), and maybe try to stop it doing that? :)

Comment 24 Harshavardhana 2014-08-11 21:21:48 UTC
> xdr-generic.c:27:29: error: too few arguments to function call, expected 3,
> have 2
>         if (!proc (&xdr, res)) {
>              ~~~~           ^
> xdr-generic.c:51:30: error: too few arguments to function call, expected 3,
> have 2
>         if (!proc (&xdr, args)) {
>              ~~~~            ^
> xdr-generic.c:75:30: error: too few arguments to function call, expected 3,
> have 2
>         if (!proc (&xdr, args)) {
>              ~~~~            ^
> 3 errors generated.
> make[4]: *** [libgfxdr_la-xdr-generic.lo] Error 1
> make[3]: *** [all-recursive] Error 1
> make[2]: *** [all-recursive] Error 1
> make[1]: *** [all-recursive] Error 1
> make: *** [all] Error 2

Okay - i think the problem doesn't occur on 10.8.5 version - it occurs on 10.9 > - RPC code is changed and made stricter.

It was fixed before, but due to some other porting efforts it was reverted for NetBSD by manu - 

~~~~commit 618d465295df02ae6d53be1327947a210bb8b47d
Author: Emmanuel Dreyfus <manu@netbsd.org>
Date:   Fri May 16 16:58:20 2014 +0200

    NetBSD build fixes
    
    - Shell scripts: == is specific to bash and ksh. Use = instead.
    - Shell scripts: use sh instead of bash if bash functionnality is not used
    - Shell scripts: ${var/search/replace} is specific to bash
    - sed: The -i option is specific to GNU sed.
    - Makefiles: $< outside of generic rules only work in GNU make.
    - xdrproc_t() is not universally defined as variadic. Do not specify third
      argument if it is not used
    - NetBSD FUSE specific: only include <perfuse.h> in FUSE client code,
      it harms in other locations
    - configure: Search for gettext() in libintl as NetBSD stores it there
    - Like MacOS X, NetBSD has unmount(2) and not umount(2) (un vs u)
    
    Some other build issues previously included in this change were
    removed:
    - __THROW macro, addressed in http://review.gluster.com/#/c/7757/
    - getmntent() compat shared with MacOS X, in http://review.gluster.com/#/c/7722/
    
    This patchset adds warning fixes for mount_glusterfs
    
    BUG: 764655
    Change-Id: I2f1faf8ff96362d3e2baf237b943df619011f1f4
    Signed-off-by: Emmanuel Dreyfus <manu@netbsd.org>
    Reviewed-on: http://review.gluster.org/7783
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Harshavardhana <harsha@harshavardhana.net>
~~~~

Especially this change

    - xdrproc_t() is not universally defined as variadic. Do not specify third
      argument if it is not used

On OSX 10.8.5
~~~~~~~~~~~
/*
 * A xdrproc_t exists for each data type which is to be encoded or decoded.
 *
 * The second argument to the xdrproc_t is a pointer to an opaque pointer.
 * The opaque pointer generally points to a structure of the data type
 * to be decoded.  If this pointer is 0, then the type routines should
 * allocate dynamic storage of the appropriate size and return it.
 */
#ifdef _KERNEL
typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int);
#else
/*
 * XXX can't actually prototype it, because some take three args!!!
 */
typedef bool_t (*xdrproc_t)(XDR *, ...);
#endif
~~~~~~~~~~~

On Linux
~~~~~~~~~~~
/*
 * A xdrproc_t exists for each data type which is to be encoded or decoded.
 *
 * The second argument to the xdrproc_t is a pointer to an opaque pointer.
 * The opaque pointer generally points to a structure of the data type
 * to be decoded.  If this pointer is 0, then the type routines should
 * allocate dynamic storage of the appropriate size and return it.
 * bool_t       (*xdrproc_t)(XDR *, caddr_t *);
 */
typedef bool_t (*xdrproc_t) (XDR *, void *,...);
~~~~~~~~~~~~

On OSX 10.9.4
~~~~~~~~~~~~

/*
 * A xdrproc_t exists for each data type which is to be encoded or decoded.
 *
 * The second argument to the xdrproc_t is a pointer to an opaque pointer.
 * The opaque pointer generally points to a structure of the data type
 * to be decoded.  If this pointer is 0, then the type routines should
 * allocate dynamic storage of the appropriate size and return it.
 *
 *
 * IMPORTANT NOTE: Apple iOS and OS X
 * 
 * Previous versions of this header file defined the xdrproc_t prototype as 
 *     typedef bool_t (*xdrproc_t)(XDR *, ...); 
 * 
 * This prototype is incorrect.  One may not use a varargs function pointer
 * in place of pointer to a function with positional parameters.
 * This mistake has been masked by the fact that clients of this API would
 * generally cast their xdr functions as (xdrproc_t), and thus silence compiler
 * warnings.  The code worked because the ABI for varargs routines that pass a
 * small number of arguments has been the same as the ABI for routines with a
 * few positional parameters.  However, if the ABI differs this will cause the
 * compiled code to fail.
 *
 * Historically, some client xdr routines expected three parameters.  This 
 * does not seem to be the case in more recent code, but we have decided to
 * retain this definition in the XDR library in case some legacy code still
 * expects three parameters.  The library will pass zero as the third
 * parameter.  Routines that expect two parameters will work correctly.
 *
 * If your client-side xdr routine is of the form:
 *   bool_t xdr_my_datatype(XDR *, void *);
 * and you pass your function pointer to routines like xdr_pointer or
 * xdr_reference using "(xdrproc_t)xdr_my_datatype", then your code will
 * compile and run correctly.  If your code invokes an xdrproc_t callback,
 * it must be modified to pass a third parameter, which may simply be zero.
 */
typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int);
~~~~~~~~~~~~~

A terrible mistake by OSX folks!

Comment 25 Anand Avati 2014-08-12 01:06:09 UTC
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#1) for review on master by Harshavardhana (harsha@harshavardhana.net)

Comment 26 Anand Avati 2014-08-12 01:09:37 UTC
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#2) for review on master by Harshavardhana (harsha@harshavardhana.net)

Comment 27 Anand Avati 2014-08-12 04:48:17 UTC
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#3) for review on master by Harshavardhana (harsha@harshavardhana.net)

Comment 28 Anand Avati 2014-08-12 09:24:11 UTC
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#4) for review on master by Harshavardhana (harsha@harshavardhana.net)

Comment 29 Anand Avati 2014-08-12 09:47:33 UTC
REVIEW: http://review.gluster.org/8458 (porting: OSX/Darwin 10.9 porting issues) posted (#5) for review on master by Harshavardhana (harsha@harshavardhana.net)

Comment 30 Eric Horwitz 2014-08-12 11:34:28 UTC
I appreciate all the hard work you guys are doing to fix the Fuze mount for native osx support but what can be done to fix the nfs issues. The compiled version I made yesterday totally broke all nfs support. I can no longer access the volume from my linux systems. It's complaining that the rpcbind doesn't match and the transport layer is not supported. I can post logs as soon as get to the office. 

It looks like whatever change apple made to nfs in mavericks 10.9 really goofed stuff up. I don't have any 10.8 machines but does nfs work from those clients? Again thanks for all your help.

Comment 31 Anand Avati 2014-08-12 16:02:17 UTC
COMMIT: http://review.gluster.org/8458 committed in master by Harshavardhana (harsha@harshavardhana.net) 
------
commit fd6765b4a3f8162bf36054cf3de6e88a6bdfadd3
Author: Harshavardhana <harsha@harshavardhana.net>
Date:   Mon Aug 11 17:36:12 2014 -0700

    porting: OSX/Darwin 10.9 porting issues
    
    xdrproc_t() arguments are variadic and non-variadic
    
    On OSX > 10.9
    -------------
    typedef bool_t (*xdrproc_t)(XDR *, void *, unsigned int);
    
    On OSX < 10.9
    ------------
    typedef bool_t (*xdrproc_t)(XDR *, ...);
    
    FreeBSD all versions
    ------------
    typedef bool_t (*xdrproc_t)(XDR *, ...);
    
    NetBSD 6.1.4
    -----------
    typedef bool_t (*xdrproc_t)(XDR *, const void *);
    
    Linux all versions
    -----------
    typedef bool_t (*xdrproc_t)(XDR *, void *,...);
    
    This weird and odd implementations across various platforms
    should be handled properly.
    
    Change-Id: Iad8b7da2e5b82526bf3708cff31ab10ce09f59c9
    BUG: 1128820
    Signed-off-by: Harshavardhana <harsha@harshavardhana.net>
    Reviewed-on: http://review.gluster.org/8458
    Reviewed-by: Emmanuel Dreyfus <manu@netbsd.org>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>

Comment 32 Harshavardhana 2014-08-12 16:25:47 UTC
> 
> It looks like whatever change apple made to nfs in mavericks 10.9 really
> goofed stuff up. I don't have any 10.8 machines but does nfs work from those
> clients? Again thanks for all your help.

The real issue is really interoperability issues, there is nothing stopping us from fixing it.

You should provide a tcpdump from linux server in this fashion

# tcpdump -s0 -i <device> -w /tmp/tcpdump.cap host <mac_osx_client>

Attach it back here, where we can see what is going on.

Questions

- are you not able to mount NFS even on Linux clients from master branch code?
- Can you provide "sw_vers" output? 
- Can you attach server nfs.log to this bugzilla? 

Thanks

-- Harsha

Comment 33 Eric Horwitz 2014-08-12 16:50:42 UTC
Trying to mount from MAC i get this from the logs... Is this a fuse issue?


[2014-08-12 16:49:42.482455] I [MSGID: 100030] [glusterfsd.c:2021:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.7dev (args: /usr/local/sbin/glusterfs --volfile-server=10.4.2.21 --volfile-id=/glfs_vol01 /mnt/objstor01)
[2014-08-12 16:49:42.483576] E [mount_darwin.c:249:gf_fuse_mount] 0-glusterfs-fuse: failed to open device (No such file or directory)
[2014-08-12 16:49:42.483607] E [xlator.c:425:xlator_init] 0-fuse: Initialization of volume 'fuse' failed, review your volfile again

Comment 34 Harshavardhana 2014-08-12 16:52:11 UTC
(In reply to Eric Horwitz from comment #33)
> Trying to mount from MAC i get this from the logs... Is this a fuse issue?
> 
> 
> [2014-08-12 16:49:42.482455] I [MSGID: 100030] [glusterfsd.c:2021:main]
> 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs
> version 3.7dev (args: /usr/local/sbin/glusterfs --volfile-server=10.4.2.21
> --volfile-id=/glfs_vol01 /mnt/objstor01)
> [2014-08-12 16:49:42.483576] E [mount_darwin.c:249:gf_fuse_mount]
> 0-glusterfs-fuse: failed to open device (No such file or directory)
> [2014-08-12 16:49:42.483607] E [xlator.c:425:xlator_init] 0-fuse:
> Initialization of volume 'fuse' failed, review your volfile again

Did you install 'osxfuse' from brew?

Comment 35 Eric Horwitz 2014-08-12 17:00:15 UTC
yes I did 

brew install osxfuse

Warning: osxfuse-2.7.0 already installed

Comment 36 Eric Horwitz 2014-08-12 17:03:11 UTC
sw_vers

ProductName:	Mac OS X
ProductVersion:	10.9.2
BuildVersion:	13C64

Comment 37 Eric Horwitz 2014-08-12 17:04:05 UTC
gluster volume set nfs.disable yes 

then

gluster volume set nfs.disable no

allowed nfs to linux clients to work again....

Comment 38 Harshavardhana 2014-08-12 17:42:46 UTC
bash-3.2# df -h /mnt/gfs
Filesystem            Size   Used  Avail Capacity   iused    ifree %iused  Mounted on
glusterfsd@osxfuse0  698Gi  394Gi  304Gi    57% 103367279 79566463   57%   /mnt/gfs
bash-3.2# sw_vers 
ProductName:    Mac OS X
ProductVersion: 10.9.4
BuildVersion:   13E28
bash-3.2#

Not able to reproduce this behavior.

Even the volume is created on the same box

bash-3.2# gluster volume info
 
Volume Name: p2p
Type: Distribute
Volume ID: d3eaa9bc-c5e9-4e29-948d-e9c17bfc3a85
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: 10.0.0.46:/mnt/gfs1
bash-3.2#

Comment 39 Harshavardhana 2014-08-12 17:44:50 UTC
One thing to verify would be do you have??

# ls /dev/osxfuse*
/dev/osxfuse0   /dev/osxfuse11  /dev/osxfuse14  /dev/osxfuse17  /dev/osxfuse2   /dev/osxfuse22  /dev/osxfuse4   /dev/osxfuse7
/dev/osxfuse1   /dev/osxfuse12  /dev/osxfuse15  /dev/osxfuse18  /dev/osxfuse20  /dev/osxfuse23  /dev/osxfuse5   /dev/osxfuse8
/dev/osxfuse10  /dev/osxfuse13  /dev/osxfuse16  /dev/osxfuse19  /dev/osxfuse21  /dev/osxfuse3   /dev/osxfuse6   /dev/osxfuse9

Comment 40 Eric Horwitz 2014-08-12 18:31:01 UTC
I do not have any of the osxfuse files in dev....


Maybe a reinstall of fuse?

Comment 41 Eric Horwitz 2014-08-12 18:35:46 UTC
that worked... I also needed to do the following....

sudo /bin/cp -RfX /usr/local/opt/osxfuse/Library/Filesystems/osxfusefs.fs /Library/Filesystems
sudo chmod +s /Library/Filesystems/osxfusefs.fs/Support/load_osxfusefs

Comment 42 Harshavardhana 2014-08-12 18:40:46 UTC
Glad it worked, let me know how it goes. FUSE based implementation should be much more reliable than using NFS IMHO.

BTW do you happen to gather the tcpdump? and nfs.log it would help us look into NFS issue further.

Comment 43 Eric Horwitz 2014-08-12 19:40:25 UTC
Created attachment 926198 [details]
Here is the nfs.log file

Comment 44 Eric Horwitz 2014-08-12 19:42:55 UTC
So I have some questions regarding performance. I have 4 machines with small RAID0 arrays that each get 150+MB/s throughput. This is setup with 2 stripes and 2 replicas. Everything is on 1Gbe. I am seeing 53MB/s write and 110MB/s read. Does this sound correct?

Comment 45 Eric Horwitz 2014-08-12 19:53:20 UTC
Created attachment 926199 [details]
tcpdump of NFS mounted MAC 10.9 system

I captured the logs before mounting the share via NFS and then tried to ls -l the path which just sits and hangs.

Comment 46 Harshavardhana 2014-08-12 19:55:01 UTC
(In reply to Eric Horwitz from comment #44)
> So I have some questions regarding performance. I have 4 machines with small
> RAID0 arrays that each get 150+MB/s throughput. This is setup with 2 stripes
> and 2 replicas. Everything is on 1Gbe. I am seeing 53MB/s write and 110MB/s
> read. Does this sound correct?

That sounds correct replicate takes away 50% of the bandwidth due to synchronous replication (hitting both the servers at the same time) - read is always served from one of the servers at a point in time. 

For more architectural discussion - you should move this discussion to 'gluster-users@gluster.org' list.

Comment 47 Harshavardhana 2014-08-12 20:27:31 UTC
Okay so was able to mount and also was able to write a file but when i do a 'ls' the mount hangs

bash-3.2# mount_nfs -o vers=3 10.14.19.108:/patchy /mnt/
gfs/   sshfs/
bash-3.2# mount_nfs -o vers=3 10.14.19.108:/patchy /mnt/gfs
bash-3.2# cd /mnt/gfs
bash-3.2# touch a
bash-3.2# stat a
436207619 11617678988833378175 -rw-r--r-- 1 root wheel 0 0 "Aug 12 16:23:47 2014" "Aug 12 16:23:47 2014" "Aug 12 16:23:47 2014" "Aug 12 16:23:47 2014" 1048576 0 0 a
bash-3.2# ls -l
^C

All i can see is on Linux NFS server, so in-fact it looks like a READDIR bug it causes infinite loops. Must be some sort of an overflow which OSX NFS client doesn't understand - this would require some time to get it fixed. 

[2014-08-12 20:25:51.828257] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789414, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:51.931971] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789415, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:51.933740] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789415, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.032864] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789416, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.034827] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789416, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.195200] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789417, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.196712] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789417, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.301950] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789418, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.304043] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789418, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.398762] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a7789419, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.400700] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a7789419, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.503713] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941a, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.505680] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941a, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.624283] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941b, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.626307] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941b, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.775226] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941c, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.777282] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941c, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0
[2014-08-12 20:25:52.909012] D [nfs3-helpers.c:3531:nfs3_log_readdir_call] 0-nfs-nfsv3: XID: a778941d, READDIR: args: FH: exportid fecd9929-ff89-45ea-adc7-5bbd39eb3cc8, gfid 00000000-0000-0000-0000-000000000001, count: 32768
[2014-08-12 20:25:52.910975] D [nfs3-helpers.c:3485:nfs3_log_readdir_res] 0-nfs-nfsv3: XID: a778941d, READDIR: NFS: 0(Call completed successfully.), POSIX: 117(Structure needs cleaning), count: 32768, cverf: 17639020, is_eof: 0

-- Harsha

Comment 48 Harshavardhana 2014-08-12 21:31:43 UTC
Created attachment 926220 [details]
NFS ganesha configs

I was able to get NFS client on MacOSX working with nfs-ganesha + FSAL_gluster support [1]

Attached configs used for sample test. 

[1] - https://github.com/nfs-ganesha/nfs-ganesha/tree/master/src/FSAL/FSAL_GLUSTER

I don't know if we have thought about deprecating GlusterNFS, but it would seem like NFS-Ganesha would be certainly the better approach.

Comment 49 Anand Avati 2014-08-13 08:43:19 UTC
REVIEW: http://review.gluster.org/8476 (porting: OSX/Darwin 10.9 porting issues) posted (#1) for review on dht-stale-layout-fixes by Raghavendra G (rgowdapp@redhat.com)

Comment 50 Justin Clift 2014-08-22 07:45:38 UTC
> I don't know if we have thought about deprecating GlusterNFS, but it
> would seem like NFS-Ganesha would be certainly the better approach.

Maybe start that discussion on gluster-devel or gluster-users mailing list?

Comment 51 Niels de Vos 2014-08-22 08:45:44 UTC
(In reply to Anand Avati from comment #49)
> REVIEW: http://review.gluster.org/8476 (porting: OSX/Darwin 10.9 porting
> issues) posted (#1) for review on dht-stale-layout-fixes by Raghavendra G
> (rgowdapp@redhat.com)

This patch has been abandoned, it was probably not related anyway.

I think this is a duplicate of bug 1132391. Could someone check/verify the patch from http://review.gluster.org/8509 ? It's been merged in the master branch already, so you could probably test it there too.

Thanks,
Niels

Comment 52 Niels de Vos 2014-09-09 07:26:31 UTC
I'm now marking this as a duplicate of Bug 1132391. Please re-open if this does not fix the reported issue with Gluster/NFS and a stripe volume when mounting from OSX.

*** This bug has been marked as a duplicate of bug 1132391 ***


Note You need to log in before you can comment on or make changes to this bug.