| Summary: | readdir function can't get all files ,when the directory has huge files | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | rick <donyren> | ||||||||
| Component: | distribute | Assignee: | shishir gowda <sgowda> | ||||||||
| Status: | CLOSED WONTFIX | QA Contact: | |||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | low | ||||||||||
| Version: | 3.1.0 | CC: | aavati, amarts, gluster-bugs, nsathyan | ||||||||
| Target Milestone: | --- | ||||||||||
| Target Release: | --- | ||||||||||
| Hardware: | i386 | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2012-04-27 05:33:37 UTC | Type: | --- | ||||||||
| Regression: | --- | Mount Type: | fuse | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Attachments: |
|
||||||||||
|
Description
rick
2010-11-29 01:39:04 UTC
Hi Rick, Can you please give more info on how to reproduce the issue? The description of the bug doesn't clearly define it. you were using 'readdir()' call? or /bin/ls command? also, it doesn't tell, where are you running 'readdir()'.. Created attachment 393 The attachment have my test code ,makefile,test process graph and volume file. (In reply to comment #1) > Hi Rick, > > Can you please give more info on how to reproduce the issue? The description of > the bug doesn't clearly define it. you were using 'readdir()' call? or /bin/ls > command? also, it doesn't tell, where are you running 'readdir()'.. Hi Amar Tumballi Oh yes,the attachment have my test code ,makefile,test process graph and volume file.I execute "glusterfs -f t-volume-fuse.vol /glusterfs" ,the volume was mounted on /glusterfs.then, then run "./fsopt creat /glusterfs 2 4094 8192 10000" to create file.it's mean startup 2 threads to create files in /glusterfs directory ,per thread create 10000 files and the file size between 4096 and 8192 bytes. run "ls /glusterfs/thread0/group0/ | wc -l" ,the output is 10000 ,this is right.but i run "/root/fsopt/readfiles /glusterfs/thread0/group0",just get 649 files. the program readfiles call readdir api to get the files in directory. Are you using unify? Can you please give us your t-volume-fuse.vol file? Avati (In reply to comment #3) > (In reply to comment #1) > > Hi Rick, > > > > Can you please give more info on how to reproduce the issue? The description of > > the bug doesn't clearly define it. you were using 'readdir()' call? or /bin/ls > > command? also, it doesn't tell, where are you running 'readdir()'.. > > Hi Amar Tumballi > > Oh yes,the attachment have my test code ,makefile,test process graph and > volume file.I execute "glusterfs -f t-volume-fuse.vol /glusterfs" ,the volume > was mounted on /glusterfs.then, then run "./fsopt creat /glusterfs 2 4094 8192 > 10000" to create file.it's mean startup 2 threads to create files in /glusterfs > directory ,per thread create 10000 files and the file size between 4096 and > 8192 bytes. run "ls /glusterfs/thread0/group0/ | wc -l" ,the output is 10000 > ,this is right.but i run "/root/fsopt/readfiles /glusterfs/thread0/group0",just > get 649 files. the program readfiles call readdir api to get the files in > directory. vol file
volume t-volume-client-0
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdb
option transport-type tcp
end-volume
volume t-volume-client-1
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdc
option transport-type tcp
end-volume
volume t-volume-client-2
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdd
option transport-type tcp
end-volume
volume t-volume-client-3
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sde
option transport-type tcp
end-volume
volume t-volume-client-4
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdf
option transport-type tcp
end-volume
volume t-volume-client-5
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdg
option transport-type tcp
end-volume
volume t-volume-client-6
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdh
option transport-type tcp
end-volume
volume t-volume-client-7
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdi
option transport-type tcp
end-volume
volume t-volume-client-8
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdj
option transport-type tcp
end-volume
volume t-volume-client-9
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdk
option transport-type tcp
end-volume
volume t-volume-client-10
type protocol/client
option remote-host 192.168.13.146
option remote-subvolume /mnt/sdl
option transport-type tcp
end-volume
volume t-volume-client-11
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdb
option transport-type tcp
end-volume
volume t-volume-client-12
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdc
option transport-type tcp
end-volume
volume t-volume-client-13
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdd
option transport-type tcp
end-volume
volume t-volume-client-14
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sde
option transport-type tcp
end-volume
volume t-volume-client-15
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdf
option transport-type tcp
end-volume
volume t-volume-client-16
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdg
option transport-type tcp
end-volume
volume t-volume-client-17
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdh
option transport-type tcp
end-volume
volume t-volume-client-18
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdi
option transport-type tcp
end-volume
volume t-volume-client-19
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdj
option transport-type tcp
end-volume
volume t-volume-client-20
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdk
option transport-type tcp
end-volume
volume t-volume-client-21
type protocol/client
option remote-host 192.168.13.147
option remote-subvolume /mnt/sdl
option transport-type tcp
end-volume
volume t-volume-client-22
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdb
option transport-type tcp
end-volume
volume t-volume-client-23
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdc
option transport-type tcp
end-volume
volume t-volume-client-24
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdd
option transport-type tcp
end-volume
volume t-volume-client-25
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sde
option transport-type tcp
end-volume
volume t-volume-client-26
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdf
option transport-type tcp
end-volume
volume t-volume-client-27
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdg
option transport-type tcp
end-volume
volume t-volume-client-28
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdh
option transport-type tcp
end-volume
volume t-volume-client-29
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdi
option transport-type tcp
end-volume
volume t-volume-client-30
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdj
option transport-type tcp
end-volume
volume t-volume-client-31
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdk
option transport-type tcp
end-volume
volume t-volume-client-32
type protocol/client
option remote-host 192.168.13.145
option remote-subvolume /mnt/sdl
option transport-type tcp
end-volume
volume t-volume-client-33
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdb
option transport-type tcp
end-volume
volume t-volume-client-34
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdc
option transport-type tcp
end-volume
volume t-volume-client-35
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdd
option transport-type tcp
end-volume
volume t-volume-client-36
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sde
option transport-type tcp
end-volume
volume t-volume-client-37
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdf
option transport-type tcp
end-volume
volume t-volume-client-38
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdg
option transport-type tcp
end-volume
volume t-volume-client-39
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdh
option transport-type tcp
end-volume
volume t-volume-client-40
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdi
option transport-type tcp
end-volume
volume t-volume-client-41
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdj
option transport-type tcp
end-volume
volume t-volume-client-42
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdk
option transport-type tcp
end-volume
volume t-volume-client-43
type protocol/client
option remote-host 192.168.13.148
option remote-subvolume /mnt/sdl
option transport-type tcp
end-volume
volume t-volume-dht
type cluster/distribute
subvolumes t-volume-client-0 t-volume-client-1 t-volume-client-2 t-volume-client-3 t-volume-client-4 t-volume-client-5 t-volume-client-6 t-volume-client-7 t-volume-client-8 t-volume-client-9 t-volume-client-10 t-volume-client-11 t-volume-client-12 t-volume-client-13 t-volume-client-14 t-volume-client-15 t-volume-client-16 t-volume-client-17 t-volume-client-18 t-volume-client-19 t-volume-client-20 t-volume-client-21 t-volume-client-22 t-volume-client-23 t-volume-client-24 t-volume-client-25 t-volume-client-26 t-volume-client-27 t-volume-client-28 t-volume-client-29 t-volume-client-30 t-volume-client-31 t-volume-client-32 t-volume-client-33 t-volume-client-34 t-volume-client-35 t-volume-client-36 t-volume-client-37 t-volume-client-38 t-volume-client-39 t-volume-client-40 t-volume-client-41 t-volume-client-42 t-volume-client-43
end-volume
volume t-volume-write-behind
type performance/write-behind
subvolumes t-volume-dht
end-volume
volume t-volume-read-ahead
type performance/read-ahead
subvolumes t-volume-write-behind
end-volume
volume t-volume-io-cache
type performance/io-cache
subvolumes t-volume-read-ahead
end-volume
volume t-volume-quick-read
type performance/quick-read
subvolumes t-volume-io-cache
end-volume
volume t-volume
type debug/io-stats
subvolumes t-volume-quick-read
end-volume
Created attachment 417 Hi,
I tried to reproduce this issue on my setup using the same fsopt tool.
Created 10000 file on distributed setup, both ls and readdir tool showing the correct results. Attached the screenshot. See volfile below.
volume vol_dht-client-0
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol1
option transport-type tcp
end-volume
volume vol_dht-client-1
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol2
option transport-type tcp
end-volume
volume vol_dht-client-2
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol3
option transport-type tcp
end-volume
volume vol_dht-client-3
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol4
option transport-type tcp
end-volume
volume vol_dht-client-4
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol5
option transport-type tcp
end-volume
volume vol_dht-client-5
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol6
option transport-type tcp
end-volume
volume vol_dht-client-6
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol7
option transport-type tcp
end-volume
volume vol_dht-client-7
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol8
option transport-type tcp
end-volume
volume vol_dht-client-8
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol9
option transport-type tcp
end-volume
volume vol_dht-client-9
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol10
option transport-type tcp
end-volume
volume vol_dht-client-10
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol11
option transport-type tcp
end-volume
volume vol_dht-client-11
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol12
option transport-type tcp
end-volume
volume vol_dht-client-12
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol13
option transport-type tcp
end-volume
volume vol_dht-client-13
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol14
option transport-type tcp
end-volume
volume vol_dht-client-14
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol15
option transport-type tcp
end-volume
volume vol_dht-client-15
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol16
option transport-type tcp
end-volume
volume vol_dht-client-16
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol17
option transport-type tcp
end-volume
volume vol_dht-client-17
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol18
option transport-type tcp
end-volume
volume vol_dht-client-18
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol19
option transport-type tcp
end-volume
volume vol_dht-client-19
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol20
option transport-type tcp
end-volume
volume vol_dht-client-20
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol21
option transport-type tcp
end-volume
volume vol_dht-client-21
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol22
option transport-type tcp
end-volume
volume vol_dht-client-22
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol23
option transport-type tcp
end-volume
volume vol_dht-client-23
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol24
option transport-type tcp
end-volume
volume vol_dht-client-24
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol25
option transport-type tcp
end-volume
volume vol_dht-client-25
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol26
option transport-type tcp
end-volume
volume vol_dht-client-26
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol27
option transport-type tcp
end-volume
volume vol_dht-client-27
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol28
option transport-type tcp
end-volume
volume vol_dht-client-28
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol29
option transport-type tcp
end-volume
volume vol_dht-client-29
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol30
option transport-type tcp
end-volume
volume vol_dht-client-30
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol31
option transport-type tcp
end-volume
volume vol_dht-client-31
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol32
option transport-type tcp
end-volume
volume vol_dht-client-32
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol33
option transport-type tcp
end-volume
volume vol_dht-client-33
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol34
option transport-type tcp
end-volume
volume vol_dht-client-34
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol35
option transport-type tcp
end-volume
volume vol_dht-client-35
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol36
option transport-type tcp
end-volume
volume vol_dht-client-36
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol37
option transport-type tcp
end-volume
volume vol_dht-client-37
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol38
option transport-type tcp
end-volume
volume vol_dht-client-38
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol39
option transport-type tcp
end-volume
volume vol_dht-client-39
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol40
option transport-type tcp
end-volume
volume vol_dht-client-40
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol41
option transport-type tcp
end-volume
volume vol_dht-client-41
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol42
option transport-type tcp
end-volume
volume vol_dht-client-42
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol43
option transport-type tcp
end-volume
volume vol_dht-client-43
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol44
option transport-type tcp
end-volume
volume vol_dht-client-44
type protocol/client
option remote-host gaurav-Vostro-3400
option remote-subvolume /home/gaurav/vol45
option transport-type tcp
end-volume
volume vol_dht-dht
type cluster/distribute
subvolumes vol_dht-client-0 vol_dht-client-1 vol_dht-client-2 vol_dht-client-3 vol_dht-client-4 vol_dht-client-5 vol_dht-client-6 vol_dht-client-7 vol_dht-client-8 vol_dht-client-9 vol_dht-client-10 vol_dht-client-11 vol_dht-client-12 vol_dht-client-13 vol_dht-client-14 vol_dht-client-15 vol_dht-client-16 vol_dht-client-17 vol_dht-client-18 vol_dht-client-19 vol_dht-client-20 vol_dht-client-21 vol_dht-client-22 vol_dht-client-23 vol_dht-client-24 vol_dht-client-25 vol_dht-client-26 vol_dht-client-27 vol_dht-client-28 vol_dht-client-29 vol_dht-client-30 vol_dht-client-31 vol_dht-client-32 vol_dht-client-33 vol_dht-client-34 vol_dht-client-35 vol_dht-client-36 vol_dht-client-37 vol_dht-client-38 vol_dht-client-39 vol_dht-client-40 vol_dht-client-41 vol_dht-client-42 vol_dht-client-43 vol_dht-client-44
end-volume
volume vol_dht-write-behind
type performance/write-behind
subvolumes vol_dht-dht
end-volume
volume vol_dht-read-ahead
type performance/read-ahead
subvolumes vol_dht-write-behind
end-volume
volume vol_dht-io-cache
type performance/io-cache
subvolumes vol_dht-read-ahead
end-volume
volume vol_dht-quick-read
type performance/quick-read
subvolumes vol_dht-io-cache
end-volume
volume vol_dht-stat-prefetch
type performance/stat-prefetch
subvolumes vol_dht-quick-read
end-volume
volume vol_dht
type debug/io-stats
subvolumes vol_dht-stat-prefetch
end-volume
Please check with latest build, we can close it if not reproducible.
Also give log files if it is reproduced. (In reply to comment #8) > Also give log files if it is reproduced. hi, I first find this issue is on 3.1.0 version,today i reproduce this issue on build of 3.1.1.i add subvolume of debug/trace in order to get detail log. Created attachment 421 ls is also calling readdir, but its succeding. We can clearly differentiate from client logs that, both ls and readfiles tool calling readdir, but readfiles stop after offset=94489280481. I am doubting its a readdir problem, otherwise ls should also fail. it's very strange,you can't reproduce this issue on your setup,but here ,it's very easy to reproduce. result is correct on local file system in some test way. i also find that "ls" run time is shorter than "readfiles". priority will be re-addressed after 3.3.0 GA We are not able to reproduce this issue in-house. please check with 3.3.0beta3. If reproduced, please re-open the bug. |