Hide Forgot
Description of problem: File descriptor leaks on libvirt daemon. Version-Release number of selected component (if applicable): # rpm -q libvirt libvirt-0.9.8-1.el6.x86_64 How reproducible: always Steps to Reproduce: 1. service libvirtd stop 2. algrind -v --track-fds=yes /usr/sbin/libvirtd --daemon Actual results: <snip> ==16804== FILE DESCRIPTORS: 7 open at exit. ==16804== Open file descriptor 7: ==16804== at 0x321FAD8B87: pipe (in /lib64/libc-2.12.so) ==16804== by 0x41F34D: daemonForkIntoBackground (libvirtd.c:186) ==16804== by 0x4207A0: main (libvirtd.c:1420) ==16804== ==16804== Open file descriptor 5: /var/log/libvirt/libvirtd.log ==16804== at 0x322020ED10: __open_nocancel (in /lib64/libpthread-2.12.so) ==16804== by 0x3E9924D0F9: virLogParseOutputs (fcntl2.h:54) ==16804== by 0x41FE79: main (libvirtd.c:792) ==16804== ==16804== Open file descriptor 4: /dev/urandom ==16804== at 0x322020ED10: __open_nocancel (in /lib64/libpthread-2.12.so) ==16804== by 0x3231247771: ??? (in /lib64/libgcrypt.so.11.5.3) ==16804== by 0x3231247AF7: ??? (in /lib64/libgcrypt.so.11.5.3) ==16804== by 0x3231244DCF: ??? (in /lib64/libgcrypt.so.11.5.3) ==16804== by 0x32312460AB: ??? (in /lib64/libgcrypt.so.11.5.3) ==16804== by 0x323124512C: ??? (in /lib64/libgcrypt.so.11.5.3) ==16804== by 0x323523D2F6: ??? (in /usr/lib64/libgnutls.so.26.14.12) ==16804== by 0x323523B59B: ??? (in /usr/lib64/libgnutls.so.26.14.12) ==16804== by 0x323522CB05: gnutls_global_init (in /usr/lib64/libgnutls.so.26.14.12) ==16804== by 0x3E992C47DB: virInitialize (libvirt.c:412) ==16804== by 0x41F7EE: main (libvirtd.c:1289) </snip> Expected results: fix fd leaks. Additional info: I think fd 4,5 leaks doesn't belong to libvirt, so I haven't fixed them in this patch.
Patch for upstream: https://www.redhat.com/archives/libvir-list/2011-December/msg00959.html
Patch on upstream: https://www.redhat.com/archives/libvir-list/2011-December/msg01026.html And patch has been ACKed and pushed on upstream: commit e957b670613cdc2de5d84b806bb0432b63c990c0 Author: Eric Blake <eblake> Date: Thu Dec 22 16:28:04 2011 +0800 daemon: clean up daemonization Valgrind detected a pipe fd leak before the parent exits on success, introduced in commit 4296cea; by itself, the leak is not bad, since we immediately called _exit(), but we might as well be clean to make valgrind analysis easier. Meanwhile, if the daemon grandchild detects an error, the parent failed to flush the error message before exiting. Also, we had the possibility of both parent and child returning to the caller, such that the user could see duplicated reports of failure from the two return paths. And we might as well be robust to the (unlikely) situation of being started with stdin closed. * daemon/libvirtd.c (daemonForkIntoBackground): Use exit if an error message was generated, avoid fd leaks for valgrind's sake, avoid returning to caller in both parent and child, and don't close a just-dup'd stdin. Based on a report by Alex Jia. However, I want to open the bug to trace more FD leaks in libvirt daemon instead of moving to 'POST', because I found new FD leaks in libvirtd again, and I will append new issue and patch in the bug.
More FD leaks on libvirt daemon: ==29248== Open file descriptor 24: ==29248== at 0x3B726D8BB7: pipe2 (in /lib64/libc-2.12.so) ==29248== by 0x3B8B363B47: rpl_pipe2 (pipe2.c:61) ==29248== by 0x3B8B23CA0D: virCommandRequireHandshake (command.c:2325) ==29248== by 0x485EA3: qemuProcessStart (qemu_process.c:3168) ==29248== by 0x45B2F5: qemuDomainObjStart (qemu_driver.c:4624) ==29248== by 0x45B8B1: qemuDomainStartWithFlags (qemu_driver.c:4681) ==29248== by 0x3B8B2CB385: virDomainCreate (libvirt.c:7888) ==29248== by 0x436D11: remoteDispatchDomainCreateHelper (remote_dispatch.h:794) ==29248== by 0x3B8B306334: virNetServerProgramDispatch (virnetserverprogram.c:416) ==29248== by 0x3B8B3075C0: virNetServerHandleJob (virnetserver.c:164) ==29248== by 0x3B8B25758B: virThreadPoolWorker (threadpool.c:144) ==29248== by 0x3B8B256EA1: virThreadHelper (threads-pthread.c:157) ==29248== ==29248== Open file descriptor 21: /dev/net/tun ==29248== at 0x3B72A0ED2D: ??? (in /lib64/libpthread-2.12.so) ==29248== by 0x3B8B265286: virNetDevTapCreate (fcntl2.h:54) ==29248== by 0x3B8B265658: virNetDevTapCreateInBridgePort (virnetdevtap.c:270) ==29248== by 0x46D98F: qemuNetworkIfaceConnect (qemu_command.c:249) ==29248== by 0x470EAC: qemuBuildCommandLine (qemu_command.c:4501) ==29248== by 0x485CD7: qemuProcessStart (qemu_process.c:3114) ==29248== by 0x45B2F5: qemuDomainObjStart (qemu_driver.c:4624) ==29248== by 0x45B8B1: qemuDomainStartWithFlags (qemu_driver.c:4681) ==29248== by 0x3B8B2CB385: virDomainCreate (libvirt.c:7888) ==29248== by 0x436D11: remoteDispatchDomainCreateHelper (remote_dispatch.h:794) ==29248== by 0x3B8B306334: virNetServerProgramDispatch (virnetserverprogram.c:416) ==29248== by 0x3B8B3075C0: virNetServerHandleJob (virnetserver.c:164) ==29248== ==29248== Open file descriptor 2: /var/log/libvirt/qemu/vr-rhel6u2-x86_64-kvm.log ==29248== at 0x3B726D8B27: dup2 (in /lib64/libc-2.12.so) ==29248== by 0x3B8B23CC4C: prepareStdFd (command.c:279) ==29248== by 0x3B8B23EBE1: virExecWithHook.clone.0 (command.c:452) ==29248== by 0x3B8B23F7BB: virCommandRunAsync (command.c:2065) ==29248== by 0x3B8B23FC31: virCommandRun (command.c:1858) ==29248== by 0x485EAF: qemuProcessStart (qemu_process.c:3170) ==29248== by 0x45B2F5: qemuDomainObjStart (qemu_driver.c:4624) ==29248== by 0x45B8B1: qemuDomainStartWithFlags (qemu_driver.c:4681) ==29248== by 0x3B8B2CB385: virDomainCreate (libvirt.c:7888) ==29248== by 0x436D11: remoteDispatchDomainCreateHelper (remote_dispatch.h:794) ==29248== by 0x3B8B306334: virNetServerProgramDispatch (virnetserverprogram.c:416) ==29248== by 0x3B8B3075C0: virNetServerHandleJob (virnetserver.c:164) ==22824== ==22824== Open file descriptor 22: /dev/vhost-net ==22824== at 0x3B72A0ED2D: ??? (in /lib64/libpthread-2.12.so) ==22824== by 0x465F64: qemuOpenVhostNet (fcntl2.h:54) ==22824== by 0x470E34: qemuBuildCommandLine (qemu_command.c:4533) ==22824== by 0x485CD7: qemuProcessStart (qemu_process.c:3114) ==22824== by 0x45B2F5: qemuDomainObjStart (qemu_driver.c:4624) ==22824== by 0x45B8B1: qemuDomainStartWithFlags (qemu_driver.c:4681) ==22824== by 0x3B8B2CB385: virDomainCreate (libvirt.c:7888) ==22824== by 0x436D11: remoteDispatchDomainCreateHelper (remote_dispatch.h:794) ==22824== by 0x3B8B306334: virNetServerProgramDispatch (virnetserverprogram.c:416) ==22824== by 0x3B8B3075C0: virNetServerHandleJob (virnetserver.c:164) ==22824== by 0x3B8B25758B: virThreadPoolWorker (threadpool.c:144) ==22824== by 0x3B8B256EA1: virThreadHelper (threads-pthread.c:157) ==27081== Open pf-16 socket 16: ==27081== at 0x3B726E67B7: socket (in /lib64/libc-2.12.so) ==27081== by 0x3B7F208814: udev_monitor_new_from_netlink (in /lib64/libudev.so.0.5.1) ==27081== by 0x4CCFBE: udevDeviceMonitorStartup (node_device_udev.c:1667) ==27081== by 0x3B8B2C55DF: virStateInitialize (libvirt.c:852) ==27081== by 0x41F8A0: daemonRunStateInit (libvirtd.c:1177) ==27081== by 0x3B8B256EA1: virThreadHelper (threads-pthread.c:157) ==27081== by 0x3B72A077F0: start_thread (in /lib64/libpthread-2.12.so) ==27081== by 0x3B726E570C: clone (in /lib64/libc-2.12.so) ==27081== Open file descriptor 15: /proc/mtrr ==27081== at 0x3B72A0ED2D: ??? (in /lib64/libpthread-2.12.so) ==27081== by 0x3B74205F83: ??? (in /usr/lib64/libpciaccess.so.0.10.8) ==27081== by 0x4CCF11: udevDeviceMonitorStartup (node_device_udev.c:1619) ==27081== by 0x3B8B2C55DF: virStateInitialize (libvirt.c:852) ==27081== by 0x41F8A0: daemonRunStateInit (libvirtd.c:1177) ==27081== by 0x3B8B256EA1: virThreadHelper (threads-pthread.c:157) ==27081== by 0x3B72A077F0: start_thread (in /lib64/libpthread-2.12.so) ==27081== by 0x3B726E570C: clone (in /lib64/libc-2.12.so) ==27081== Open pf-16 socket 12: ==27081== at 0x3B726E67B7: socket (in /lib64/libc-2.12.so) ==27081== by 0x3B80E0488B: audit_open (in /lib64/libaudit.so.1.0.0) ==27081== by 0x3B8B25BFD9: virAuditOpen (viraudit.c:59) ==27081== by 0x4208D4: main (libvirtd.c:1549)
Like for 769517, it would be good to get a new set of fd leak results with a recent version like 0.9.1-4, thanks ! Daniel
(In reply to comment #5) > Like for 769517, it would be good to get a new set of fd leak results > with a recent version like 0.9.1-4, > > thanks ! > > Daniel Hello Daniel, There are many FD leaks still exist on libvirt-0.9.10-4.el6.x86_64: ==19661== Open file descriptor 7: ==19661== at 0x3EB70D8B87: pipe (in /lib64/libc-2.12.so) ==19661== by 0x420ADB: daemonForkIntoBackground (libvirtd.c:187) ==19661== by 0x42205B: main (libvirtd.c:1442) ==19661== Open file descriptor 2: /dev/null ==19661== at 0x3EB70D8B27: dup2 (in /lib64/libc-2.12.so) ==19661== by 0x420B8D: daemonForkIntoBackground (libvirtd.c:209) ==19661== by 0x42205B: main (libvirtd.c:1442) ==20943== FILE DESCRIPTORS: 6 open at exit. ==20943== Open file descriptor 25: ==20943== at 0x3EB70D8BB7: pipe2 (in /lib64/libc-2.12.so) ==20943== by 0x4F89797: rpl_pipe2 (pipe2.c:59) ==20943== by 0x4E58A5D: virCommandRequireHandshake (command.c:2441) ==20943== by 0x48CC4C: qemuProcessStart (qemu_process.c:3355) ==20943== by 0x45E2A5: qemuDomainObjStart (qemu_driver.c:4720) ==20943== by 0x45E861: qemuDomainStartWithFlags (qemu_driver.c:4777) ==20943== by 0x4EEEEB5: virDomainCreate (libvirt.c:8062) ==20943== by 0x439291: remoteDispatchDomainCreateHelper (remote_dispatch.h:852) ==20943== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==20943== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) ==20943== by 0x4E7266B: virThreadPoolWorker (threadpool.c:144) ==20943== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==20943== ==20943== Open file descriptor 24: ==20943== at 0x3EB70D8BB7: pipe2 (in /lib64/libc-2.12.so) ==20943== by 0x4F89797: rpl_pipe2 (pipe2.c:59) ==20943== by 0x4E58A44: virCommandRequireHandshake (command.c:2437) ==20943== by 0x48CC4C: qemuProcessStart (qemu_process.c:3355) ==20943== by 0x45E2A5: qemuDomainObjStart (qemu_driver.c:4720) ==20943== by 0x45E861: qemuDomainStartWithFlags (qemu_driver.c:4777) ==20943== by 0x4EEEEB5: virDomainCreate (libvirt.c:8062) ==20943== by 0x439291: remoteDispatchDomainCreateHelper (remote_dispatch.h:852) ==20943== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==20943== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) ==20943== by 0x4E7266B: virThreadPoolWorker (threadpool.c:144) ==20943== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==20943== ==20943== Open file descriptor 22: /dev/net/tun ==20943== at 0x3EB740ED2D: ??? (in /lib64/libpthread-2.12.so) ==20943== by 0x4E83C96: virNetDevTapCreate (fcntl2.h:54) ==20943== by 0x4E84068: virNetDevTapCreateInBridgePort (virnetdevtap.c:270) ==20943== by 0x4739C8: qemuNetworkIfaceConnect (qemu_command.c:249) ==20943== by 0x476A84: qemuBuildCommandLine (qemu_command.c:4819) ==20943== by 0x48CA05: qemuProcessStart (qemu_process.c:3294) ==20943== by 0x45E2A5: qemuDomainObjStart (qemu_driver.c:4720) ==20943== by 0x45E861: qemuDomainStartWithFlags (qemu_driver.c:4777) ==20943== by 0x4EEEEB5: virDomainCreate (libvirt.c:8062) ==20943== by 0x439291: remoteDispatchDomainCreateHelper (remote_dispatch.h:852) ==20943== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==20943== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) ==20943== Open file descriptor 0: /dev/null ==20943== at 0x3EB70D8B27: dup2 (in /lib64/libc-2.12.so) ==20943== by 0x4E583CC: prepareStdFd (command.c:376) ==20943== by 0x4E5A662: virExecWithHook.clone.0 (command.c:544) ==20943== by 0x4E5B271: virCommandRunAsync (command.c:2181) ==20943== by 0x4E5B6B6: virCommandRun (command.c:1974) ==20943== by 0x48CC58: qemuProcessStart (qemu_process.c:3357) ==20943== by 0x45E2A5: qemuDomainObjStart (qemu_driver.c:4720) ==20943== by 0x45E861: qemuDomainStartWithFlags (qemu_driver.c:4777) ==20943== by 0x4EEEEB5: virDomainCreate (libvirt.c:8062) ==20943== by 0x439291: remoteDispatchDomainCreateHelper (remote_dispatch.h:852) ==20943== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==20943== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) I will update other FD leaks later. Regards, Alex
FD leaks on 0.9.10-4(cont 1): ==26934== Open file descriptor 26: /dev/vhost-net ==26934== at 0x3EB740ED2D: ??? (in /lib64/libpthread-2.12.so) ==26934== by 0x46B764: qemuOpenVhostNet (fcntl2.h:54) ==26934== by 0x476ADC: qemuBuildCommandLine (qemu_command.c:4851) ==26934== by 0x48CA05: qemuProcessStart (qemu_process.c:3294) ==26934== by 0x45E2A5: qemuDomainObjStart (qemu_driver.c:4720) ==26934== by 0x45E861: qemuDomainStartWithFlags (qemu_driver.c:4777) ==26934== by 0x4EEEEB5: virDomainCreate (libvirt.c:8062) ==26934== by 0x439291: remoteDispatchDomainCreateHelper (remote_dispatch.h:852) ==26934== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==26934== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) ==26934== by 0x4E7266B: virThreadPoolWorker (threadpool.c:144) ==26934== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==25975== Open pf-16 socket 20: ==25975== at 0x3EB70E67B7: socket (in /lib64/libc-2.12.so) ==25975== by 0x3EC1C1C9CD: nl_connect (in /lib64/libnl.so.1.1) ==25975== by 0x3EC380846D: ??? (in /usr/lib64/libnetcf.so.1.4.0) ==25975== by 0x3EC3806FEE: ??? (in /usr/lib64/libnetcf.so.1.4.0) ==25975== by 0x4D1E88: interfaceOpenInterface (netcf_driver.c:144) ==25975== by 0x4EE78EB: do_open (libvirt.c:1233) ==25975== by 0x4EE87C5: virConnectOpen (libvirt.c:1350) ==25975== by 0x439AA0: remoteDispatchOpenHelper (remote.c:628) ==25975== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==25975== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) ==25975== by 0x4E7266B: virThreadPoolWorker (threadpool.c:144) ==25975== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==25975== ==25975== Open AF_INET socket 19: <unbound> <-> unbound ==25975== at 0x3EB70E67B7: socket (in /lib64/libc-2.12.so) ==25975== by 0x3EC3809119: ??? (in /usr/lib64/libnetcf.so.1.4.0) ==25975== by 0x3EC3806FD8: ??? (in /usr/lib64/libnetcf.so.1.4.0) ==25975== by 0x4D1E88: interfaceOpenInterface (netcf_driver.c:144) ==25975== by 0x4EE78EB: do_open (libvirt.c:1233) ==25975== by 0x4EE87C5: virConnectOpen (libvirt.c:1350) ==25975== by 0x439AA0: remoteDispatchOpenHelper (remote.c:628) ==25975== by 0x4F2E1F4: virNetServerProgramDispatch (virnetserverprogram.c:416) ==25975== by 0x4F2CFF0: virNetServerHandleJob (virnetserver.c:164) ==25975== by 0x4E7266B: virThreadPoolWorker (threadpool.c:144) ==25975== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==25975== by 0x3EB74077F0: start_thread (in /lib64/libpthread-2.12.so) ==25975== ==25975== Open AF_UNIX socket 18: /var/run/libvirt/libvirt-sock ==25975== at 0x3EB740E7ED: ??? (in /lib64/libpthread-2.12.so) ==25975== by 0x4F35A97: virNetSocketAccept (virnetsocket.c:1261) ==25975== by 0x4F2E6AB: virNetServerServiceAccept (virnetserverservice.c:60) ==25975== by 0x4E61B1E: virEventPollRunOnce (event_poll.c:490) ==25975== by 0x4E608C6: virEventRunDefaultImpl (event.c:247) ==25975== by 0x4F2C5BC: virNetServerRun (virnetserver.c:736) ==25975== by 0x421D65: main (libvirtd.c:1609) ==25975== ==25975== Open pf-16 socket 17: ==25975== at 0x3EB70E67B7: socket (in /lib64/libc-2.12.so) ==25975== by 0x3EBC008814: udev_monitor_new_from_netlink (in /lib64/libudev.so.0.5.1) ==25975== by 0x4D574E: udevDeviceMonitorStartup (node_device_udev.c:1667) ==25975== by 0x4EE88DF: virStateInitialize (libvirt.c:852) ==25975== by 0x4209D0: daemonRunStateInit (libvirtd.c:1178) ==25975== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==25975== by 0x3EB74077F0: start_thread (in /lib64/libpthread-2.12.so) ==25975== by 0x3EB70E570C: clone (in /lib64/libc-2.12.so) ==25975== ==25975== Open file descriptor 16: /proc/mtrr ==25975== at 0x3EB740ED2D: ??? (in /lib64/libpthread-2.12.so) ==25975== by 0x332CC05F83: ??? (in /usr/lib64/libpciaccess.so.0.10.8) ==25975== by 0x4D56A1: udevDeviceMonitorStartup (node_device_udev.c:1619) ==25975== by 0x4EE88DF: virStateInitialize (libvirt.c:852) ==25975== by 0x4209D0: daemonRunStateInit (libvirtd.c:1178) ==25975== by 0x4E71F88: virThreadHelper (threads-pthread.c:161) ==25975== by 0x3EB74077F0: start_thread (in /lib64/libpthread-2.12.so) ==25975== by 0x3EB70E570C: clone (in /lib64/libc-2.12.so) ==25975== ==25975== Open AF_UNIX socket 15: <unknown> ==25975== at 0x3EB70E67B7: socket (in /lib64/libc-2.12.so) ==25975== by 0x332B42CF06: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B42D548: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B424896: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B424A50: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B423144: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B40F920: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B40B682: ??? (in /lib64/libdbus-1.so.3.4.0) ==25975== by 0x332B0047A9: avahi_client_new (in /usr/lib64/libavahi-client.so.3.2.5) ==25975== by 0x4F2B281: virNetServerMDNSStart (virnetservermdns.c:452) ==25975== by 0x4F2C4EA: virNetServerRun (virnetserver.c:699) ==25975== by 0x421D65: main (libvirtd.c:1609) ==25975== ==25975== Open pf-16 socket 12: ==25975== at 0x3EB70E67B7: socket (in /lib64/libc-2.12.so) ==25975== by 0x332AC0488B: audit_open (in /lib64/libaudit.so.1.0.0) ==25975== by 0x4E77099: virAuditOpen (viraudit.c:59) ==25975== by 0x421A06: main (libvirtd.c:1550) ==25975== ==25975== Open file descriptor 9: ==25975== at 0x3EB70D8BB7: pipe2 (in /lib64/libc-2.12.so) ==25975== by 0x4F89797: rpl_pipe2 (pipe2.c:59) ==25975== by 0x4E61F45: virEventPollInit (event_poll.c:675) ==25975== by 0x4E60916: virEventRegisterDefaultImpl (event.c:208) ==25975== by 0x4F2D538: virNetServerNew (virnetserver.c:381) ==25975== by 0x421896: main (libvirtd.c:1485) And will continue to add others in here.
Alex, since it has been a long time I guess many fd leaks have been fixed. If you happen to find a new ones, please open a new bug. I am closing this as WORKSFORME.
(In reply to comment #11) > Alex, > > since it has been a long time I guess many fd leaks have been fixed. If you > happen to find a new ones, please open a new bug. I am closing this as > WORKSFORME. Okay, I will check it again, if I find FD leaks then will file a new bug.