RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 618995 - File descriptor leak on pvdisplay invocation in single user mode
Summary: File descriptor leak on pvdisplay invocation in single user mode
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: upstart
Version: 6.0
Hardware: All
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Petr Lautrbach
QA Contact: Martin Cermak
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-07-28 10:08 UTC by Alexander Todorov
Modified: 2018-11-14 13:38 UTC (History)
14 users (show)

Fixed In Version: upstart-0.6.5-8.el6
Doc Type: Bug Fix
Doc Text:
Upstart sends jobs to the shell via pipe represented by /proc/self/fd/<fd> file in /proc filesystem. This file descriptor was not, which resulted in a message about a file descriptor leak when operating at run level 1. This file descriptor is now closed correctly.
Clone Of:
Environment:
Last Closed: 2011-05-19 12:40:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
send close command for pipe fd to shell (675 bytes, patch)
2010-10-11 10:20 UTC, Petr Lautrbach
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2011:0531 0 normal SHIPPED_LIVE upstart bug fix update 2011-05-18 17:57:48 UTC

Description Alexander Todorov 2010-07-28 10:08:31 UTC
Description of problem:
In single user mode with latest RHEL6 snapshot there's a message about file descriptor leak when running pvdisplay:

# pvdisplay
File descriptor 10 (pipe:[16563]) leaked on pvdisplay invocation. Parent PID 6297: /bin/bash
 --- Physical Volume ---
  PV info follows

Version-Release number of selected component (if applicable):
lvm2-2.02.69-2.el6

How reproducible:
always

Steps to Reproduce:
1. Perform default install where / and swap is on LVM
2. Reboot the system
3. Login as root and execute init 1 
  
Actual results:
when pvdisplay is executed in runlevel 3 there's no message about descriptor leaks. When executed in runlevel 1 there is such a message. 

Expected results:


Additional info:

Comment 1 Alasdair Kergon 2010-07-28 10:17:14 UTC
Run pstree -p to get a list of all the parent processes back up to init, then run lsof on all those processes (so lsof -p 1;... lsof -p 6297)  and see where that fd is coming from.

Comment 2 Alasdair Kergon 2010-07-28 10:18:29 UTC
(or get the same info by browsing /proc)

Comment 3 Alexander Todorov 2010-07-29 13:41:58 UTC
All commands executed in runlevel 1 right after boot.

# pvs
File descriptor 9 (pipe:[10526]) leaked on pvs invocation. Parent PID 996: /bin/bash
  PV         VG       Fmt  Attr PSize PFree
  /dev/vda2  vg_one   lvm2 a-   9.76g    0 
  /dev/vda5  VolGroup lvm2 a-   9.25g    0 

[root@localhost /]# pstree -p
init(1)─┬─bash(996)───pstree(1025)
        └─udevd(494)─┬─udevd(893)
                     └─udevd(894)

[root@localhost /]# lsof -p 996
COMMAND PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
bash    996 root  cwd    DIR  253,0     4096      2 /
bash    996 root  rtd    DIR  253,0     4096      2 /
bash    996 root  txt    REG  253,0   943248   5149 /bin/bash
bash    996 root  mem    REG  253,0   150672  15098 /lib64/ld-2.12.so
bash    996 root  mem    REG  253,0    22536  15100 /lib64/libdl-2.12.so
bash    996 root  mem    REG  253,0  1838296  15099 /lib64/libc-2.12.so
bash    996 root  mem    REG  253,0   138280   4952 /lib64/libtinfo.so.5.7
bash    996 root  mem    REG  253,0    26050 131015 /usr/lib64/gconv/gconv-modules.cache
bash    996 root  mem    REG  253,0 99158752   2463 /usr/lib/locale/locale-archive
bash    996 root  mem    REG  253,0    61624   2490 /lib64/libnss_files-2.12.so
bash    996 root    0u   CHR    5,1      0t0   4821 /dev/console
bash    996 root    1u   CHR    5,1      0t0   4821 /dev/console
bash    996 root    2u   CHR    5,1      0t0   4821 /dev/console
bash    996 root    9r  FIFO    0,8      0t0  10526 pipe
bash    996 root  255u   CHR    5,1      0t0   4821 /dev/console

[root@localhost /]# lsof -p 1
COMMAND PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
init      1 root  cwd    DIR              253,0     4096      2 /
init      1 root  rtd    DIR              253,0     4096      2 /
init      1 root  txt    REG              253,0   137808 259553 /sbin/init
init      1 root  mem    REG              253,0   150672  15098 /lib64/ld-2.12.so
init      1 root  mem    REG              253,0   106360   3274 /lib64/libnih.so.1.0.0
init      1 root  mem    REG              253,0  1838296  15099 /lib64/libc-2.12.so
init      1 root  mem    REG              253,0   145672  15116 /lib64/libpthread-2.12.so
init      1 root  mem    REG              253,0    40080   8143 /lib64/libnih-dbus.so.1.0.0
init      1 root  mem    REG              253,0    47064  15123 /lib64/librt-2.12.so
init      1 root  mem    REG              253,0   268200  15138 /lib64/libdbus-1.so.3.4.0
init      1 root  mem    REG              253,0    93320  15142 /lib64/libgcc_s-4.4.4-20100713.so.1
init      1 root  mem    REG              253,0    61624   2490 /lib64/libnss_files-2.12.so
init      1 root    0u   CHR                1,3      0t0   3434 /dev/null
init      1 root    1u   CHR                1,3      0t0   3434 /dev/null
init      1 root    2u   CHR                1,3      0t0   3434 /dev/null
init      1 root    3r  FIFO                0,8      0t0   8154 pipe
init      1 root    4w  FIFO                0,8      0t0   8154 pipe
init      1 root    5r   DIR               0,10        0      1 inotify
init      1 root    6r   DIR               0,10        0      1 inotify
init      1 root    7u  unix 0xffff8800379f4680      0t0   8155 socket

Comment 4 Milan Broz 2010-07-29 16:48:42 UTC
This is not lvm2 bug.

See man page:
On invocation, lvm requires that only the standard file descriptors stdin, stdout and stderr are available.  If others are found, they get closed and messages are issued warning about the leak.

So there is bug in some runlevel script which invoke bash after switching to runlevel 1 but forgot to close all unused descriptors.

If you run "pvdisplay 9>&-" to close it (where 9 is the reported leaked descriptor), no warning is printed.

Reassigning to upstart, but it is possible that it is some intermediate script which leaks it...

(and this should not be blocker IMHO)

Comment 6 Petr Lautrbach 2010-08-03 12:01:40 UTC
Upstart sends job scripts to shell via pipe represented by /dev/fd/<fd> (/proc/self/fd/) file in proc filesystem:

/bin/sh -e /dev/fd/9

This fd stays open and pvdisplay complaints about that. If it is closed on exec then /dev/fd/9 won't exist and shell won't have place to read from. 

Possible solution would be send first closing pipe fd command to shell:

+		close_fd = NIH_MUST (nih_sprintf (NULL, "<&%d- ;", fds[0]));
+		NIH_ZERO (nih_io_write (io, close_fd, strlen (close_fd)));
+
 		NIH_ZERO (nih_io_write (io, script, strlen (script)));
 		nih_io_shutdown (io);

Comment 7 Petr Lautrbach 2010-10-11 10:20:12 UTC
Created attachment 452680 [details]
send close command for pipe fd to shell

This is upstart workaround patch which first sends close fd command to shell before script defined in job.

Comment 8 Petr Lautrbach 2010-10-11 10:24:32 UTC
I can reproduce this problem without upstart using named fifo as script source instead of file:

# bash <( echo pvdisplay)
File descriptor 63 (pipe:[20578493]) leaked on pvdisplay invocation. Parent PID 27181: bash
...

# bash <( echo lsof -p \$\$)
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF     NODE NAME
bash    27310 root  cwd    DIR  253,0     4096    90113 /root
bash    27310 root  rtd    DIR  253,0     4096        2 /
bash    27310 root  txt    REG  253,0   943248     6347 /bin/bash
bash    27310 root  mem    REG  253,0   150672     5069 /lib64/ld-2.12.so
bash    27310 root  mem    REG  253,0  1838296     7190 /lib64/libc-2.12.so
bash    27310 root  mem    REG  253,0    22536    10714 /lib64/libdl-2.12.so
bash    27310 root  mem    REG  253,0   138280    11019 /lib64/libtinfo.so.5.7
bash    27310 root  mem    REG  253,0 99158752   141962 /usr/lib/locale/locale-archive
bash    27310 root  mem    REG  253,0    26050   141827 /usr/lib64/gconv/gconv-modules.cache
bash    27310 root    0u   CHR  136,6      0t0        9 /dev/pts/6
bash    27310 root    1u   CHR  136,6      0t0        9 /dev/pts/6
bash    27310 root    2u   CHR  136,6      0t0        9 /dev/pts/6
bash    27310 root   63r  FIFO    0,8      0t0 20697063 pipe
^^^^^^^ shouldn't be here
bash    27310 root  255r  FIFO    0,8      0t0 20697063 pipe

# cat test_fd.sh
lsof -p $$

# bash test_fd.sh
COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF   NODE NAME
bash    27357 root  cwd    DIR  253,2     4096 122882 /home/plautrba/tmp
bash    27357 root  rtd    DIR  253,0     4096      2 /
bash    27357 root  txt    REG  253,0   943248   6347 /bin/bash
bash    27357 root  mem    REG  253,0   150672   5069 /lib64/ld-2.12.so
bash    27357 root  mem    REG  253,0  1838296   7190 /lib64/libc-2.12.so
bash    27357 root  mem    REG  253,0    22536  10714 /lib64/libdl-2.12.so
bash    27357 root  mem    REG  253,0   138280  11019 /lib64/libtinfo.so.5.7
bash    27357 root  mem    REG  253,0 99158752 141962 /usr/lib/locale/locale-archive
bash    27357 root  mem    REG  253,0    26050 141827 /usr/lib64/gconv/gconv-modules.cache
bash    27357 root    0u   CHR  136,6      0t0      9 /dev/pts/6
bash    27357 root    1u   CHR  136,6      0t0      9 /dev/pts/6
bash    27357 root    2u   CHR  136,6      0t0      9 /dev/pts/6
bash    27357 root  255r   REG  253,2       21   2188 /home/plautrba/tmp/test_fd.sh

# rpm -q bash
bash-4.1.2-3.el6.x86_64

Comment 9 Roman Rakus 2010-10-11 14:46:20 UTC
It is standard behaviour of bash. Bash keeps input file open; in this case pipe, which is fd 63.
What about to use named pipe (FIFO)?

Comment 10 Petr Lautrbach 2010-10-12 09:05:59 UTC
In case of pipe, it's opened twice
bash    27310 root   63r  FIFO    0,8      0t0 20697063 pipe
bash    27310 root  255r  FIFO    0,8      0t0 20697063 pipe

but regular file is opened only once
bash    27357 root  255r   REG  253,2       21   2188 /home/plautrba/tmp/test_fd.sh


This is reproducer using /dev/fd/<fd>:

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>

int main() {
	char buf[] = "pvdisplay; lsof -p $$";
	char fd_name[256];
	int fd[2], s;

	pipe(fd);

	if (fork()) {
		close(fd[0]);
		write(fd[1], buf, sizeof(buf));
		close(fd[1]);
		wait(&s);
		return 0;
	}

	close(fd[1]);
	sprintf(fd_name, "/dev/fd/%d", fd[0]);
	execl("/bin/bash", "bash", fd_name, NULL);
}

Comment 11 Petr Lautrbach 2010-10-12 10:08:29 UTC
# mkfifo test_fd; echo 'pvdisplay; lsof -p $$' > test_fd & bash test_fd

works ok

> bash    27310 root   63r  FIFO    0,8      0t0 20697063 pipe
> bash    27310 root  255r  FIFO    0,8      0t0 20697063 pipe

here comes opened fd 63 from parent process. bash (or child generally) probably shouldn't close opened fd from parent process as it can be intended.

Comment 16 Martin Cermak 2011-01-04 14:29:48 UTC
Comments #15, #16 => VERIFIED

Comment 18 Laura Bailey 2011-05-03 04:26:02 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Upstart sends jobs to the shell via file descriptors (/dev/fd files) in the /proc file system. One of these file descriptors was not closed, which resulted in a message about a file descriptor leak when operating at run level 1. This file descriptor is now closed correctly.

Comment 19 Petr Lautrbach 2011-05-06 13:48:30 UTC
    Technical note updated. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    Diffed Contents:
@@ -1 +1 @@
-Upstart sends jobs to the shell via file descriptors (/dev/fd files) in the /proc file system. One of these file descriptors was not closed, which resulted in a message about a file descriptor leak when operating at run level 1. This file descriptor is now closed correctly.+Upstart sends jobs to the shell via pipe represented by /proc/self/fd/<fd> file in /proc filesystem. This file descriptor was not, which resulted in a message about a file descriptor leak when operating at run level 1. This file descriptor is now closed correctly.

Comment 21 errata-xmlrpc 2011-05-19 12:40:40 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-0531.html


Note You need to log in before you can comment on or make changes to this bug.