| Summary: | first access of automounted glusterfs mount point fails | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Krishna Srinivas <krishna> |
| Component: | core | Assignee: | Krishna Srinivas <krishna> |
| Status: | CLOSED DUPLICATE | QA Contact: | |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | pre-2.0 | CC: | aavati, gluster-bugs, mark |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Krishna Srinivas
2009-08-17 10:34:11 UTC
On 09/06/2009 11:19 PM, Mark Mielke wrote: > On 09/06/2009 10:42 PM, Mark Mielke wrote: >> This seems to happen about 50% of the time: >> >> [root@wcarh035 ~]# ls /gluster/data >> ls: cannot open directory /gluster/data: No such file or directory >> [root@wcarh035 ~]# ls /gluster/data >> 00 06.fun 15 23.fun 32 40.fun 47 55.fun 64 >> 00.fun 07 15.fun 24 32.fun 41 47.fun 56 64.fun > >> My current guess is that GlusterFS is saying the mount is complete to AutoFS before the actual mount operation takes effect. 50% of the time GlusterFS is able to complete the mount before AutoFS let's the user continue, and all is well. The other 50% of the time, GlusterFS does not quite finish the mount, and AutoFS gives the user a broken directory. >> >> I might try and prove this by adding a sleep 5 to /sbin/mount.glusterfs, although I do not consider this a valid solution, as it just reduces the effect of the race - it does not eliminate the race. > > Uhh... Hmm... It already has a "sleep 3", and changing it to "sleep 5" does not reduce the frequency of the problem. Changing it to "sleep 10" also has no effect. > > Why does it sometimes work and sometimes not? I note that the fusermount from the FUSE libraries does not seem to have the same problem: $ /stage/linux/fuse-2.7.4/example/fusexmp_fh /tmp/t ; ls /tmp/t backup/ boot/ etc/ lib64/ media/ pccyber/ sbin/ stage/ usr/ backup2/ db/ home/ lost+found/ mnt/ proc/ selinux/ sys/ var/ bin/ dev/ lib/ mail/ opt/ root/ srv/ tmp/ www/ It works immediately. Compare this to: [root@wcarh033]~# echo hi >/tmp/t/hi [root@wcarh033]~# time /opt/glusterfs/sbin/glusterfs --volfile=/etc/glusterfs/gluster-data.vol /tmp/t ; ls /tmp/t ; sleep 1 ; ls /tmp/t /opt/glusterfs/sbin/glusterfs --volfile=/etc/glusterfs/gluster-data.vol /tmp/ 0.00s user 0.00s system 113% cpu 0.003 total hi 00 06.fun 15 23.fun 32 40.fun 47 55.fun 64 00.fun 07 15.fun 24 32.fun 41 47.fun 56 64.fun 01 07.fun 16 24.fun 33 41.fun 50 56.fun 65 01.fun 10 16.fun 25 33.fun 42 50.fun 57 65.fun 02 10.fun 17 25.fun 34 42.fun 51 57.fun 66 02.fun 11 17.fun 26 34.fun 43 51.fun 60 66.fun 03 11.fun 20 26.fun 35 43.fun 52 60.fun 67 03.fun 12 20.fun 27 35.fun 44 52.fun 61 67.fun 04 12.fun 21 27.fun 36 44.fun 53 61.fun lost+found 04.fun 13 21.fun 30 36.fun 45 53.fun 62 05 13.fun 22 30.fun 37 45.fun 54 62.fun 05.fun 14 22.fun 31 37.fun 46 54.fun 63 06 14.fun 23 31.fun 40 46.fun 55 63.fun Note that the first 'ls' returns 'hi', and a second later, 'ls' returns the glusterfs content. For fusexmp, it appears to complete the mount before it returns. For glusterfs, it seems to complete the mount a short time after it completes. I think this is where autofs is getting confused, and serving the handle to the directory to the client too early. It thinks glusterfs is done mounting, and gives the handle to the client, but this handle is broken and fails. Glusterfs completes the mount, and a short time later the lookups succeed. Adding 'sleep' in mount.glusterfs do not seem to be good enough - as 'sleep 1' and 'sleep 20' do not change the frequency. The existing 'sleep 3' in /sbin/mount.glusterfs should be completely unnecessary. Instead, we should figure out why GlusterFS cannot ensure the mount is in place before it returns? |