Red Hat Bugzilla – Bug 178266
VFS: file-max limit 50905 reached
Last modified: 2015-01-04 17:24:29 EST
From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8) Gecko/20051201 Fedora/1.5-1.1.fc4.nr Firefox/1.5 Description of problem: When I run make on a fairly large C++ program, I get an error "Too many open files in system", accompanied by the kernel message "VFS: file-max limit 50905 reached". The error occurs at somewhat random points during the build, but usually occurs during linking, which presumably causes many library files to be opened simultaneously. I can't imagine that ld actually opens anywhere near 50000 files simultaneously, however. I even tried increasing the maximum file descriptors by running "echo 100000 >/proc/sys/fs/file-max". The build runs a bit longer, but I soon get the same error. This indicates to me that the kernel is somehow leaking fds internally. Version-Release number of selected component (if applicable): kernel-2.6.14-1.1656_FC4 How reproducible: Always Steps to Reproduce: The problem is easy for me to reproduce, but unfortunately I cannot distribute the program I am attempting to build. I will try to reproduce the problem by building some open-source software package, and will update this bug with more details. Additional info: The problem occurs every time on kernel-2.6.14-1.1656_FC4 and kernel-2.6.14-1.1644_FC4. It does not occur at all on kernel-2.6.13-1.1532_FC4 or kernel-2.6.11-1.1369_FC4. I am running Fedora Core 4 within qemu-0.7.2 configured with 512 MB of memory. (The host machine is a 3 GHz dual-core Pentium D with 2 GB of RAM, also running FC4.)
The problem also occurs on kernel-2.6.15-1.1824_FC4 (testing).
This thread on the linux-kernel mailing list seems relevant: http://groups.google.com/group/fa.linux.kernel/browse_thread/thread/43eef3f1d89a4ef2/ My vague understanding of this thread: It seems that the way the kernel frees up closed file descriptors was changed in 2.6.14. It now uses a mechanism called an RCU callback, which ksoftirqd runs every so often, and can process a certain number of tasks each time it runs, limited by a configurable parameter (rcupdate.maxbatch). If for whatever reason the callback does not run frequently enough, closed-but-not-freed file descriptors start to pile up until the maximum is reached. (This situation seems quite possible given that ksoftirqd runs at nice +19.) My guess is that this change has made the kernel much more sensitive to the weird realtime clock behavior that is common when running on a virtual machine like qemu, where the frequency of timer interrupts might be off by a factor of 10 depending on how heavily loaded the host machine is. Obviously I don't understand all the issues here, but perusing the linux-kernel thread doesn't inspire confidence that every corner case has been accounted for.
I can reproduce this problem by building the Boost library (http://prdownloads.sourceforge.net/boost/boost_1_33_1.tar.bz2): $ tar jxvf boost_1_32_0.tar.bz2 $ cd boost_1_32_0/tools/build/jam_src $ ./build.sh $ cd ../../.. $ tools/build/jam_src/bin.linuxx86/bjam A few minutes into the build, I get the error: /usr/include/features.h:314:25: error: /usr/lib/gcc/i386-redhat-linux/4.0.2/../../../../include/c++/4.0.2/sys/cdefs.h: Too many open files in system with the following in dmesg: VFS: file-max limit 50905 reached
This is a mass-update to all currently open kernel bugs. A new kernel update has been released (Version: 2.6.15-1.1830_FC4) based upon a new upstream kernel release. Please retest against this new kernel, as a large number of patches go into each upstream release, possibly including changes that may address this problem. This bug has been placed in NEEDINFO_REPORTER state. Due to the large volume of inactive bugs in bugzilla, if this bug is still in this state in two weeks time, it will be closed. Should this bug still be relevant after this period, the reporter can reopen the bug at any time. Any other users on the Cc: list of this bug can request that the bug be reopened by adding a comment to the bug. If this bug is a problem preventing you from installing the release this version is filed against, please see bug 169613. Thank you.
Closing due to lack of response.