Red Hat Bugzilla – Bug 37211
Installer (incorrectly) reports "not enough file nodes"
Last modified: 2007-04-18 12:32:48 EDT
From Bugzilla Helper:
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT)
Trying to upgrade 7.0 -> 7.1; installation proceeds nicely to the part
where it checks that there's enough disk space for the selected packages.
I have 68MB free on "/", 580MB free in "/usr" and 3GB free on "/home" so
I'm not worried. The installer comes back telling me I don't have enough
file nodes on the "/" partition, I need a few hundred more.
Steps to Reproduce:
1. Upgrade of existing RH7.0 system to RH7.1.
Actual Results: Upgrade fails at part of process where disk space checks
Expected Results: Installation should have proceeded normally.
This problem has happened on three of the four systems I've tried to
The first upgrade, on a system with fairly limited available disk space,
For the second (the first to fail), on my small firewall machine with very
few packages and even less disk space, there were sufficiently few
packages installed so that I could do the upgrade by just manually
upgrading all of the installed packages, plus adding whatever new ones
were required to satisfy dependencies. None of the new packages caused any
problems when upgrading.
The third upgrade was the machine with loads of available disk space, as
mentioned earlier. Since there were a very large number of packages to
install, I decided to leave the upgrade for a while after it failed.
The last upgrade was a system with very limited disk space. Here I found
an important clue. After the installer had failed I was trying a manual
upgrade of the individual packages. The "dev" package refused to go in,
rpm claiming that there were not enough file nodes, i.e. the same message
as the installer. I cleared out plenty more disk space but still it
complained. I had over 8000 free inodes on the root partition so I was
convinced that it should go in. I tried using the "--ignoresize" option on
rpm to force the upgrade to apply. It went in OK, leaving plenty of free
inodes. Hence the problem would appear to be with rpm 4.0.2 (I'd already
upgraded the rpm package) or with the "dev" package, or possibly some
combination of the two.
To test my theory, I returned to my third machine, which had not yet been
upgraded. I manually upgraded the "dev" package and then ran the RH7.1
installer. The upgrade applied with no hitches.
Hence I believe the problem lies with rpm 4.0.2 (which is linked with the
installer I believe), or the "dev" package in RH7.1. I've put it in
the "anaconda" category because it was during installation that I hit the
problem, and that's probably where anyone else who hits it will probably
look for existing bug reports.
Did you try supplying a "few hundred more" inodes? The disk accounting
check in rpm checks both blocks and inodes, and AFAIK (and many, many
installs show) that the algorithm is correct, but it's impossible to do
the check precisely because of intermediate temp files and such ...
At the time I had the original problem I didn't know for sure that it was
inodes that it was complaining about, so no, I didn't try freeing up more of
them. I did, however, have 68MB of free space on my root partition, which was a
plain vanilla ext2fs with a "clean" RH7 install on it, so I wasn't inclined to
believe that I didn't actually have enough free inodes. Since the "dev" package
on its own installed cleanly with no complaints from rpm, and subsequently the
full upgrade completed with no complaints, I suspected a problem with the
I'm sure that you've done many more installs that I have, and presumably
haven't come across anything like this before, but I *have* seen this on 3 of 4
attempted upgrades. Since I have now completed those upgrades, I'll only be
able to provide more information if the problem crops up again with other
machines (I still have a few to do).
Is there any way I can get rpm to tell me how many free inodes it thinks a
given package upgrade will need? I can then compare that with the output
of "df -i", which presumably would be useful to you?
1) the number of inodes available correlates with disk free space
only because there are heuristics in mke2fs (and friends)
that attempts to scale the two together. Truly, the two limits
are different ...
2) rpm uses the results of statvfs() call as gospel. That
may or may not be what "df -i" uses (an strace shows activity
in /proc, not a stafs(2) call, but that may be the way that statfs
is implemented in glibc.) Certainly the two results should be
very similar, but that's not the same as identical. Write a short
program to print out the return of statvfs() if you want to see
see the values that rpm is using.
3) No matter what, rpm is not going to be able to better than nominal
(i.e. 1% or so ) accuracy, as there's no attempt (nor should there be)
to track each and every file created during an install. Even for disk
blocks, rpm cannot Get It Right because of the root reserved limit (rpm
adds a 5% reserved for root heuristic, not right in general).
So, with file systems close to the edge, there may still be
failures, possibly even intermittent non-reproducible failures.
4) Certain file system types do not return accurate information.
Both vfat and reiserfs are known to fall in this category, there
are other (ahem) "friends" of e2fs that will break. FWIW, rpm
detects reiserfs/vfat breakage and disables inode checks, but
there's bound to be more breakage ahead.
Regarding your points:
2. I've written a program as suggested; the results from statvfs() line up
exactly with "df" and "df -i", at least with the filesystems I've tried.
3. Can't rpm use the "f_bfree" part of the statvfs structure instead of
the "f_bavail" part to determine available space if it wants to use the root
So now I know how much space and inodes are actually available, how do I know
how many rpm thinks it's going to need?
For a system upgrade, does rpm add together the sum of the temporary inodes
it'll need in each filesystem for each package, or does it just take the worst
case (biggest) number from all of the packages? e.g. if (say) the "dev" package
needs 5000 temporary inodes and the "dev86" package needs 500, does rpm
consider that an upgrade of both packages requires 5000 or 5500? I ask this
because of the case where a full upgrade failed but later passed after a manual
upgrade of the "dev" package...
This is an rpm problem, fixed in rpm CVS, will be in rpm-4.0.3 final.