Kernel running out of system file descriptors when using memory
mapping.

Platform intel 686, Redhat 6.1.

We have ported our Database management tool to redhat linux from
NCR SVR4, IBM AIX 4.2, Windows NT 4.0, Sco Open Server 5.4,
Unixware 2.1 & Unixware 7.1 etc.

The system is designed around memory mapping and uses the follow
formula to access the database.

The database is split up into 1 Megabyte files (pools) and can
only open upto eleven pools, each pool is mapped in 4k pages,
giving us a total of 512 pages per pool, we map upto a maximum
of 2000 pages this is our high water mark and once reached we
unmap 1000 pages.

The problem seems to be that when a pool is closed and there are
still mapped pages linked to that pool Linux copies or duplicates
the file descriptor, this file descriptor is deducted from the
system table of file descriptors and therefore causing the system
to lose upto 2000 file descriptors per user from it's default
of 4096.

This software works on all the other operating systems we have
ported too, for example:-
we currently have a very large user (1000 users and growing)
running on NCR's SVR4 unix and our application, this user wants
to switch from his NCR svr4 system to a DELL machine running
RedHat Linux, in total this user will switch 3000 networked users
from NCR to RedHat linux this year.

All this depends on this problem been resolved.

I have attacted an example program that needs to be run from root,
that shows the problem.

Source Program.


Program Name : "t"


#include <fstream.h>
#include <sys/mman.h>
#include "sys/time.h"
#include "sys/resource.h"
#include "unistd.h"
#include "sys/types.h"
#include "sys/stat.h"
#include "fcntl.h"
#include "iostream.h"
#include "stdio.h"
#include "error.h"



void OpenFileTest(void) {
	const char * fn = "XXX";
	int last,fd;
	int count = 0;
	int start=open(fn, O_CREAT | O_RDONLY, S_IRWXU);
	while(1) {
		count=open(fn, O_CREAT | O_RDONLY, S_IRWXU);
		if ( count == -1 	) {
			perror("dup failed!");
			cerr << "Target set to " << last << endl;
			break;
		}
		last=count;
	}
	for (int loop=start;loop <= last; loop++) {
		fd =close(loop);
		if ( fd == -1 	) {
		   cerr << "<" << loop << "> " ;
			perror("Close failed!");
			break;
		}
	}
};

void main(void) {
	int fd;
	struct rlimit *rlim;
	fd = getrlimit(RLIMIT_NOFILE,rlim);
	rlim->rlim_max=15120;
	rlim->rlim_cur=15120;
	fd = setrlimit(RLIMIT_NOFILE,rlim);
	if ( fd == -1 	)
	{
		perror("Setrlimit failed!");
		exit;
	}
	fd = getrlimit(RLIMIT_NOFILE,rlim);
	cerr << " rlim data,  current <" << rlim->rlim_cur << "> ";
	cerr << " max <" << rlim->rlim_max << ">." << endl;
	while(1) {
		const char * Path="ZZZZ";
		fstream F(Path, ios::in | ios::out, filebuf::openprot);
		fd = F.rdbuf()->fd();
		cerr << "fd:" << fd << endl;
		caddr_t Addr;
		const int Length = 4096;
		int Offset = 0;
	if ((Addr = (caddr_t) mmap(NULL, Length, PROT_READ | PROT_WRITE,
			MAP_SHARED, fd, Offset)) == MAP_FAILED) {
		perror("Map1 failed");
		exit;
	}
	Offset += Length;
	if ((Addr = (caddr_t) mmap(NULL, Length, PROT_READ | PROT_WRITE,
			MAP_SHARED, fd, Offset)) == MAP_FAILED) {
		perror("Map2 failed");
		exit;
	}

	OpenFileTest();
	F.close();
	cin.get();
	}
};

login as root

compile line :-
  g++ t.C -o t

touch ZZZZ

to execute :-
./t


The fd count is reduced by 1 with each loop.

This is a major problem and is stopping us from releasing our software on
RedHat Linux.

Regards
       Keith Ansell.