gdm 2.0beta2 (installed first as the Red Hat 6.2 shipped package and then upgraded to gdm-2.0beta2-26 from the package on the Red Hat 6.2 errata mirrors) appears to be leaking file descriptors. The first sign was when the server ran out of file handles and the kernel reported via dmesg: "VFS: file-max limit 4096 reached". At the time no unusual processes were being run, just five standard GNOME login sessions via XDM from X-terminals and PCs running Exceed. I increased the file descriptors using /proc/sys/fs/file-max to solve the immediate problem and then used lsof to track down the user of the files. This appeared to be gdm: [root@molniya /root]# lsof | grep gdm | wc -l 1856 In other words, the combined gdm and gdmlogin (matches 'grep gdm') processes had over 1800 files open between them. Looking more closely at the master gdm and the files it held with lsof, I noticed that it was holding multiple file descriptors for the gdm log files. [root@molniya /root]# lsof -p 3230 | grep '/var/gdm/' | sort +8 | less <snip> gdm 3230 root 50w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 51w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 52w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 56w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 58w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 61w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 69w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 73w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 77w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 79w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log gdm 3230 root 83w REG 8,7 0 26142 /var/gdm/sscpc067.ee.surrey.ac.uk:0.log <snip> This is a part of the output but it is replicated for other displays. It seems to show that the master gdm process holds multiple file descriptors open for the same log file and does this for every display. Other gdm child processes appear to do the same thing. The combined effect of this possible file descriptor leak is that the kernel runs out of file descriptors quickly. I have yet to look at the latest gdm 2.0beta4 to see if the same problem is present or to check the source to see what is happening. However it seemed worth reporting to you as others may be seeing the same problem.
I have had a look at the source of gdm 2.0beta4. There is nothing in the changelog to show this has been noted or fixed. Looking in the source in file daemon/xdmcp.c, the log file is opened on line 773 with handle 'logfd'. However it does not look like the log file is closed again at any point, at least there is no obvious close call on logfd.
I've added a close(logfd) in the latest version, should solve the problem.
*** Bug 22794 has been marked as a duplicate of this bug. ***