Bug 650588
| Summary: | subprocess fails in select when descriptors are large (rhel6) | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Haim <hateya> |
| Component: | python | Assignee: | Dave Malcolm <dmalcolm> |
| Status: | CLOSED ERRATA | QA Contact: | Petr Šplíchal <psplicha> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 6.1 | CC: | abaron, cpelland, danken, ddumas, dgregor, dmalcolm, hateya, iheim, katzj, mgoldboi, ohudlick, plyons, psplicha, tao, yeylon |
| Target Milestone: | rc | Keywords: | ZStream |
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | python-2.6.6-4.el6 | Doc Type: | Bug Fix |
| Doc Text: |
Previously, Python programs that used "ulimit -n" to enable communication with large numbers of subprocesses could still monitor only 1024 file descriptors at a time, due to the subprocess module using the "select" system call. This could cause an exception:
ValueError: filedescriptor out of range in select()
The module now uses the "poll" system call, removing this limitation.
|
Story Points: | --- |
| Clone Of: | 609020 | Environment: | |
| Last Closed: | 2011-05-19 11:36:13 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 609020 | ||
| Bug Blocks: | 616055, 630832, 668975 | ||
*** Bug 616055 has been marked as a duplicate of this bug. ***
Technical note added. If any revisions are required, please edit the "Technical Notes" field
accordingly. All revisions will be proofread by the Engineering Content Services team.
New Contents:
Previously, Python programs that used "ulimit -n" to enable communication with large numbers of subprocesses could still monitor only 1024 file descriptors at a time, due to the subprocess module using the "select" system call. This could cause an exception:
ValueError: filedescriptor out of range in select()
The module now uses the "poll" system call, removing this limitation.
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0554.html An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2011-0554.html |
looks as if this issue wasn't resolve in RHEL6. just hit it when trying to start 50 vms over single host. had 762 processes of dd running. Thread-2548::ERROR::2010-11-07 10:50:39,579::clientIF::48::vds::(wrapper) Traceback (most recent call last): File "/usr/share/vdsm/clientIF.py", line 44, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/clientIF.py", line 887, in getVdsCapabilities self.machineCapabilities = self._getCapabilities() File "/usr/share/vdsm/clientIF.py", line 875, in _getCapabilities caps['emulatedMachines'] = getEmulatedMachines() File "/usr/share/vdsm/clientIF.py", line 794, in getEmulatedMachines ' -M ?') ][1:] File "/usr/share/vdsm/utils.py", line 576, in execAndGetOutput out, err = p.communicate() File "/usr/lib64/python2.6/subprocess.py", line 701, in communicate return self._communicate(input) File "/usr/lib64/python2.6/subprocess.py", line 1199, in _communicate rlist, wlist, xlist = select.select(read_set, write_set, []) ValueError: filedescriptor out of range in select() Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 863, in _run return fn(*args, **kargs) File "/usr/share/vdsm/storage/hsm.py", line 1468, in public_getVolumeSize apparentsize = str(volume.Volume.getVSize(sdUUID, imgUUID, volUUID, bs=1)) File "/usr/share/vdsm/storage/volume.py", line 244, in getVSize return mysd.getVolumeClass().getVSize(mysd, imgUUID, volUUID, bs) File "/usr/share/vdsm/storage/blockVolume.py", line 47, in getVSize return int(int(lvm.getLVInfo(sdobj.sdUUID, volUUID)) / bs) File "/usr/share/vdsm/storage/lvm.py", line 1122, in getLVInfo return getLV(vg, lv).size File "/usr/share/vdsm/storage/lvm.py", line 923, in getLV return _lvminfo.getLv(vg, lv) File "/usr/share/vdsm/storage/lvm.py", line 567, in getLv lvs = self._reloadlvs(vgName) File "/usr/share/vdsm/storage/lvm.py", line 429, in _reloadlvs rc, out, err = self.cmd(cmd) File "/usr/share/vdsm/storage/lvm.py", line 354, in cmd return misc.execCmd(cmd) File "/usr/share/vdsm/storage/misc.py", line 116, in execCmd env=env) File "/usr/lib64/python2.6/subprocess.py", line 633, in __init__ errread, errwrite) File "/usr/lib64/python2.6/subprocess.py", line 1049, in _execute_child self.pid = os.fork() OSError: [Errno 11] Resource temporarily unavailable File "/usr/share/vdsm/storage/storage_mailbox.py", line 667, in run self._checkForMail() File "/usr/share/vdsm/storage/storage_mailbox.py", line 626, in _checkForMail (rc, in_mail, err) = misc.execCmd(cmd, sudo=False, raw=True) File "/usr/share/vdsm/storage/misc.py", line 116, in execCmd env=env) File "/usr/lib64/python2.6/subprocess.py", line 626, in __init__ errread, errwrite) = self._get_handles(stdin, stdout, stderr) File "/usr/lib64/python2.6/subprocess.py", line 968, in _get_handles p2cread, p2cwrite = os.pipe() OSError: [Errno 24] Too many open files