Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
When executing pvdisplay on systems with many disks (e.g. 1000 LUNs with 4 paths), the command fails in error:
...
/sys/dev/block/253:248/holders/: opendir failed: Too many open files
/sys/dev/block/253:249/holders/: opendir failed: Too many open files
/run/lock/lvm/V_rhel: setfscreatecon failed: Too many open files
...
Version-Release number of selected component (if applicable):
lvm2-2.02.180-10.el7_6.8.x86_64
How reproducible:
Always
Steps to Reproduce:
1. Configure the reproducer as a iSCSI client and multipath
(client)
# yum -y install iscsi-initiator-utils device-mapper-multipath
# mpathconf --enable --with_multipathd y
# echo "InitiatorName=iqn.1994-05.com.redhat:5634dd137c53" > /etc/iscsi/initiatorname.iscsi
2. Set up multiple iSCSI servers (each server is limited to 256 LUNs)
For each server <serverX> (iscsi.sh in attachment):
# yum -y install targetcli
# ./iscsi.sh | targetcli
# systemctl enable target --now
3. Attach the LUNs to the reproducer
(client)
# iscsiadm -m discovery -t st -p <serverX> -l
...
4. Execute "pvdisplay"
# pvdisplay
Actual results:
See above.
Expected results:
No error.
Additional info:
By default, root has 1024 has soft limit for number of files:
# ulimit -n
1024
Increasing the limit to 65536 solves the issue.
Until a release with the fix is available, you can work around it by raising the file limit or disabling lvmetad.
*** This bug has been marked as a duplicate of bug 1691277 ***
The fix was already delivered in 7.6.z package lvm2-2.02.180-10.el7_6.7 (bug 1695879), so this is not a duplicate and fixed package is used by the customer.
Marian, I do not think this fix is in 7.6, but maybe I'm not looking in the right place?
This fix is already in 7.7 (I'm not sure we should backport it to 7.6 because I think there are dependencies on other prior commits.)
https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=5502f72e41dceb85d11ec562819ad6c5db6758ae
lvmetad: fix pvs for many devices
When using lvmetad, 'pvs' still evaluates full filters
on all devices (lvmetad only provides info about PVs,
but pvs needs to report info about all devices, at
least sometimes.)
Because some filters read the devices, pvs still reads
every device, even with lvmetad (i.e. lvmetad is no help
for the pvs command.) Because the device reads are not
being managed by the standard label scan layer, but only
happen incidentally through the filters, there is nothing
to control and limit the bcache content and the open file
descriptors for the devices. When there are a lot of devs
on the system, the number of open fd's excedes the limit
and all opens begin failing.
The proper solution for this would be for pvs to really
use lvmetad and not scan devs, or for pvs to do a proper
label scan even when lvmetad is enabled. To avoid any
major changes to the way this has worked, just work around
this problem by dropping bcache and closing the fd after
pvs evaluates the filter on each device.
In bug 1691277, everything from comment 14 to comment 25 is related to this fix. But, I guess that bug 1691277 had already been used for the other MAX_IO fix.
It seems there are two 7.6.z streams stemming from BZ#1691277.
lvm2-2.02.185-2.el7 - 7.6 z-stream (EUS) bug #1695879
lvm2-2.02.180-10.el7_6.9 - 7.6 z-stream (EUS) bug #1731981
changelog:
* Mon Jul 22 2019 Marian Csontos <mcsontos> - 7:2.02.180-10.el7_6.9
- Fix pvs for many devices when using lvmetad.
Version shown by BZ reporter is less than the fixed release where commit 5502f72e41dceb85d11ec562819ad6c5db6758ae was added:
---
lvm2-2.02.180-10.el7_6.8.x86_64
Customer needs to update their 7.6.z lvm2 package via https://access.redhat.com/errata/RHBA-2019:2850. Closing this as NOTABUG.