Bug 87493 - e2fsck thinks driectories with enormous numbers of files are bad
Summary: e2fsck thinks driectories with enormous numbers of files are bad
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Red Hat Linux
Classification: Retired
Component: e2fsprogs
Version: 8.0
Hardware: i386
OS: Linux
medium
high
Target Milestone: ---
Assignee: Florian La Roche
QA Contact: Jay Turner
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2003-03-27 18:41 UTC by Jon Burgess
Modified: 2015-01-08 00:04 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2003-06-03 12:37:18 UTC
Embargoed:


Attachments (Terms of Use)

Description Jon Burgess 2003-03-27 18:41:14 UTC
From Bugzilla Helper:
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3) Gecko/20030314

Description of problem:
I installed a CDDB catalogue on my machine & then when it came to check the disk
it through up a ton of errors and was still trying to fix them 6 hours later.

I have since repeated the test on a different machine with a loopback mounted fs
at it seems to indicate that in my case having >260k files in one directory
causes a problem (I think the limit is 512x512)

I then tried it with the latest e2fsprogs-1.32-6 from Phoebe-8.0.94 at it does
not report any errors on the filesystem.

Version-Release number of selected component (if applicable):
e2fsprogs-1.27-9

How reproducible:
Always

Steps to Reproduce:
1. Create an ext3 filesystem of 5Gb or more using default mke2fs parameters: (I
used a loopback device, but this is easier to explain)
# mke2fs /dev/sda1
2. Create ext3 journal (tune2fs -j /dev/sda1)
3. Mount this partition somewhere as ext3
4. Extract the CDDB database freedb-complete-20030302.tar.bz2 from
www.freedb.org to the new partition
5. Wait a _very_ long time like 6+ hours since handling directories this large
is very bad on ext2/3
6. Unmount partition and run "e2fsck -f -d -r /dev/sda1" 
(I used the "read-only" e2fsck mode so it didn't make any changes)

Actual Results:
  [root@cele tmp]# e2fsck -f -n -d /dev/loop0
e2fsck 1.27 (8-Mar-2002)
Pass 1: Checking inodes, blocks, and sizes
Inode 131073 is too big.  Truncate? no

Block #513 (279587) causes directory to be too big.  IGNORED.
Block #514 (279588) causes directory to be too big.  IGNORED.
Block #515 (279589) causes directory to be too big.  IGNORED.
Block #516 (279590) causes directory to be too big.  IGNORED.
Block #517 (279591) causes directory to be too big.  IGNORED.
Block #518 (279592) causes directory to be too big.  IGNORED.
Block #519 (279593) causes directory to be too big.  IGNORED.
Block #520 (279594) causes directory to be too big.  IGNORED.
Block #521 (279595) causes directory to be too big.  IGNORED.
Block #522 (279596) causes directory to be too big.  IGNORED.
Block #523 (279597) causes directory to be too big.  IGNORED.
Too many illegal blocks in inode 131073.
Clear inode? no

Suppress messages? no
...


Expected Results:  I then repeated the test with e2fsprogs-1.32-6 from
Phoebe-8.0.94 on exactly the same filesystem
[root@cele mount]# e2fsck -V
e2fsck 1.32 (09-Nov-2002)
        Using EXT2FS Library version 1.32, 09-Nov-2002
[root@cele tmp]# e2fsck -f -n -d /dev/loop0
e2fsck 1.32 (09-Nov-2002)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/loop0: 417529/655360 files (0.0% non-contiguous), 448765/1310720 blocks




Additional info:

The e2fsprogs complains about inode 131073, which is the "misc" directory:
[root@cele mount]# ls -li
total 6616
  32769 drwxr-xr-x    2 root     root        81920 Mar 27 10:27 data
  65537 drwxr-xr-x    2 1007     1007       847872 Mar  2 11:00 folk
  98305 drwxr-xr-x    2 1007     1007       913408 Mar  2 12:00 jazz
     11 drwx------    2 root     root        16384 Mar 27 10:18 lost+found
 131073 drwxr-xr-x    2 1007     1007      4468736 Mar  2 12:00 misc
 425985 drwxr-xr-x    2 root     root       417792 Mar 27 17:37 rock

"misc" contains 279042 files:
[root@cele mount]# for i in *; do echo -n $i; find $i | wc -l ; done
mount/data   5093
mount/folk  52875
mount/jazz  57046
mount/lost+found      1
mount/misc 279042
mount/rock  26026
[ There are actually a few more files in the archive, but I couldn't be bothered
to wait hours for them all to be extracted ]

I'm guessing that the limit is 512x512 = 260k files, this is based on the "512
inode blocks per group" shown below:

[root@cele tmp]# tune2fs -l /dev/loop0
tune2fs 1.32 (09-Nov-2002)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          34ccf580-25f3-40ab-b768-34ff56fed1e5
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery sparse_super
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              655360
Block count:              1310720
Reserved block count:     65536
Free blocks:              861956
Free inodes:              237832
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Filesystem created:       Thu Mar 27 10:18:18 2003
Last mount time:          Thu Mar 27 10:20:14 2003
Last write time:          Thu Mar 27 10:20:14 2003
Mount count:              1
Maximum mount count:      36
Last checked:             Thu Mar 27 10:20:04 2003
Check interval:           15552000 (6 months)
Next check after:         Tue Sep 23 11:20:04 2003
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal UUID:             <none>
Journal inode:            8
Journal device:           0x0000
First orphan inode:       0

I (now) know that ext2/3 isn't a good choice for >200k files in one directory
and i'll use reiserfs next time. Unfortunately it has brought this PC /
filesystem down for a day or so until I can sort out this mess. I'm still not
sure that I can clean up this partially e2fsck'd filesystem.

Perhaps it is worth updating to the newest e2fsprogs in the RH7.3/8.0/AS product
updates?

Comment 1 Florian La Roche 2003-06-03 12:37:18 UTC
I have updated e2fsprogs to a newer release, but won't releasse this as errata
for older RHL releases.

Thanks for your report,

Florian La Roche



Note You need to log in before you can comment on or make changes to this bug.