Description of problem: Reading is extremely slow on an ext3 filesystem. It is quite noticeable as stuttering while playing HD movie files when the data does not get read quick enough on scenes with lots of movement. I did not have this problem with Fedora 10 i686 on the same system. Increasing player read buffer size mitigates the problem. Version-Release number of selected component (if applicable): [max@truth ~]$ uname -a Linux truth 2.6.29.6-213.fc11.x86_64 #1 SMP Tue Jul 7 21:02:57 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux How reproducible: Always. Steps to Reproduce: 1. Read 1Gb of data from a file on an ext3 filesystem and note the read rate. 2. Read 1Gb of data from the block device of the same ext3 filesystem and note the read rate. 3. Compare the two rates. They should be close. Actual results: [max@truth ~]$ sudo hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 310 MB in 3.02 seconds = 102.68 MB/sec [max@truth ~]$ dd if=~/Videos/Home.2009.Bluray.720p.AC3.x264-CHD.mkv of=/dev/null bs=1M count=1k 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 250.262 s, 4.3 MB/s [max@truth ~]$ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 33025272 6495036 24852652 21% / /dev/sda5 309553864 281675012 11900608 96% /home tmpfs 1019448 14228 1005220 2% /dev/shm [max@truth ~]$ sudo dd if=/dev/sda5 of=/dev/null bs=1M count=1k 1024+0 records in 1024+0 records out 1073741824 bytes (1.1 GB) copied, 11.392 s, 94.3 MB/s Expected results: I expect the file read rate to be much higher than 4.3 Mb/s, since I can read 94.3 MB/s from the same block device. Additional info: Can provide if requested.
If /home is 96% full the file may be highly fragmented.
point filefrag [-v] at the file you're testing, we can see how fragmented it is. -Eric
(In reply to comment #2) > point filefrag [-v] at the file you're testing, we can see how fragmented it > is. Whoa, the file seems to be fragmented badly: [max@truth ~]$ sudo filefrag ~/Videos/Home.2009.Bluray.720p.AC3.x264-CHD.mkv /home/max/Videos/Home.2009.Bluray.720p.AC3.x264-CHD.mkv: 449868 extents found, perfection would be 41 extents I tried reading a non-fragmented file: [max@truth ~]$ sudo filefrag ~/Videos/TOM.AVI /home/max/Videos/TOM.AVI: 7 extents found [max@truth ~]$ dd if=~/Videos/TOM.AVI of=/dev/null bs=1M count=728 690+1 records in 690+1 records out 723529728 bytes (724 MB) copied, 11.0531 s, 65.5 MB/s So, the issue seems to be high fragmentation. Thank you guys. Max p.s.: I probably need to make eMule preallocate the whole file! ;)
Yup I was going to ask whether it was a torrent ;) Some torrent clients will do nice preallocation on ext4 or xfs, which support fast preallocation; something you might consider. -Eric
(In reply to comment #4) > Yup I was going to ask whether it was a torrent ;) Correction: I use aMule, not eMule. I use Transmission too, but BitTorrent being the hottest topic nowadays I'd like to keep under radars sometimes by using eDonkey network. And I love the integrated search. offtopic: Having mentioned BitTorrent, I think there is at least one good idea how to make seeders anonymous (which solves legal issues in some states), so if anyone cares here is a link http://max0x7ba.blogspot.com/2009/07/how-to-build-anonymous-peer-to-peer.html I hope one smart hacker integrates something like that in BitTorrent transfer protocol. > Some torrent clients will do nice preallocation on ext4 or xfs, which support > fast preallocation; something you might consider. I now definitely will! By any chance, do you know of a tool that generates some sort of a filesystem fragmentation report (apart from manual `find <folder> -type f -exec filefrag {} \; | <some-formatter-script>)`? And, while we are at it, is there any recommended filesystem (ext3 in my case) defragmenter tool? (I did a quick google search on "ext3 defragment" and there are no hot hits, and the source of all knowledge (Wikipedia) says "... A true defragmentation tool does not exist for ext3..." Max
(In reply to comment #5) > (In reply to comment #4) ... > > Some torrent clients will do nice preallocation on ext4 or xfs, which support > > fast preallocation; something you might consider. > > I now definitely will! > > By any chance, do you know of a tool that generates some sort of a filesystem > fragmentation report (apart from manual `find <folder> -type f -exec filefrag > {} \; | <some-formatter-script>)`? latest e2fsck with some degree of "-v" gives a total fragmentation info. git e2fsprogs has an "e2freefrag" tool for freespace xfs_db can do "frag" and "freesp" to get some idea of file & freespace fragmentation. > And, while we are at it, is there any recommended filesystem (ext3 in my case) > defragmenter tool? (I did a quick google search on "ext3 defragment" and there > are no hot hits, and the source of all knowledge (Wikipedia) says "... A true > defragmentation tool does not exist for ext3..." that's true. ext4 will get e4defrag eventually, a rough cut is in git. xfs has xfs_fsr, which works. -Eric