Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1164234

Summary: liblvm2app leaking socket when using lvmetad
Product: Red Hat Enterprise Linux 7 Reporter: Marc Fournier <rh>
Component: lvm2Assignee: Alasdair Kergon <agk>
lvm2 sub component: LVM Metadata / lvmetad QA Contact: Cluster QE <mspqa-list>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: urgent CC: agk, cmarthal, heinzm, jbrassow, msnitzer, prajnoha, prockai, rh, zkabelac
Version: 7.1Keywords: Triaged
Target Milestone: rc   
Target Release: 7.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: lvm2-2.02.114-2.el7 Doc Type: Bug Fix
Doc Text:
Cause: lvmetad socket not closed after use while using lvm2app in long-running processes. Consequence: Possible process failure due to high number of open files (lvmetad socket) that accumulate over time. Fix: lvmetad socket is always released properly after use while using lvm2app library. Result: The failure in processes using lvm2app library caused by too many open files (caused by not releasing lvmetad socket after use) does not occur anymore.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-03-05 13:10:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Marc Fournier 2014-11-14 12:11:31 UTC
When /etc/lvm/lvm.conf contains "use_lvmetad = 1" and the lvmetad daemon is running, a long running process linked against liblvm2app will connect to a socket to talk with lvmetad, but doesn't close it after use. This leads the process to fail with "too many open files" at some point.

This problem seems to also occur on RHEL6, as well as other distros such as debian. But it's aggravated in RHEL7 by the fact that lvmetad is now enabled by default.

Digging a bit further, it seeems that the socket gets created when "lvm_list_vg_names()" (among other functions, I imagine) is called. And not released (as one would expect) when "lvm_quit()" is called.

The example code added with this patch reproduces the problem:
http://www.redhat.com/archives/lvm-devel/2010-September/msg00025.html

strace running this example code:

[...]
socket(PF_LOCAL, SOCK_STREAM, 0)        = 3
connect(3, {sa_family=AF_LOCAL, sun_path="/run/lvm/lvmetad.socket"}, 110) = 0
write(3, "request = \"hello\"\n", 18)   = 18
write(3, "\n##\n", 4)                   = 4
read(3, "response = \"OK\"\nprotocol = \"lvme", 32) = 32
read(3, "tad\"\nversion = 1\n\n##\n", 1024) = 21
write(3, "request=\"vg_list\"\ntoken =\"filter"..., 36) = 36
write(3, "\n##\n", 4)                   = 4
read(3, "response=\"OK\"\nvolume_groups {\n\tn", 32) = 32
read(3, "f56N6-M9UE-e4rR-jBjK-KqA4-sNme-1"..., 1024) = 64
write(3, "request=\"vg_lookup\"\nuuid =\"nf56N"..., 85) = 85
write(3, "\n##\n", 4)                   = 4
read(3, "response=\"OK\"\nname=\"vg0\"\nmetadat", 32) = 32
read(3, "a {\n\tid=\"nf56N6-M9UE-e4rR-jBjK-K"..., 1024) = 1024
read(3, "10782889\n\t\t\tsegment_count=1\n\t\t\ts"..., 1056) = 1056
read(3, "e=1410782891\n\t\t\tsegment_count=1\n"..., 2112) = 1426
stat("/sys/subsystem", 0x7fffc5060f60)  = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/sys/bus", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 4
getdents(4, /* 22 entries */, 32768)    = 608
open("/usr/lib64/gconv/gconv-modules.cache", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=26254, ...}) = 0
mmap(NULL, 26254, PROT_READ, MAP_SHARED, 5, 0) = 0x7f3318b80000
close(5)                                = 0
futex(0x7f3318436a70, FUTEX_WAKE_PRIVATE, 2147483647) = 0
getdents(4, /* 0 entries */, 32768)     = 0
close(4)                                = 0
[...]

close(3) never happens.


This issue is also discussed here: https://github.com/collectd/collectd/issues/753
Finally, I wonder if this might be related to https://bugzilla.redhat.com/show_bug.cgi?id=1095463

Thanks !

Comment 2 Ondrej Kozina 2014-11-28 14:40:49 UTC
This is related to bz: 1130245

Comment 3 Alasdair Kergon 2014-11-28 19:22:01 UTC
Clearly daemon_close() should be closing the file descriptor.

Comment 7 Corey Marthaler 2015-01-16 21:59:59 UTC
Marking this verified (Sanity only). Have not seen anything resembling this during regression testing.

3.10.0-223.el7.x86_64
lvm2-2.02.114-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
lvm2-libs-2.02.114-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
lvm2-cluster-2.02.114-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
device-mapper-1.02.92-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
device-mapper-libs-1.02.92-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
device-mapper-event-1.02.92-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
device-mapper-event-libs-1.02.92-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015
device-mapper-persistent-data-0.4.1-2.el7    BUILT: Wed Nov 12 12:39:46 CST 2014
cmirror-2.02.114-5.el7    BUILT: Wed Jan 14 08:42:28 CST 2015

Comment 9 errata-xmlrpc 2015-03-05 13:10:35 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0513.html