Bug 650588 - subprocess fails in select when descriptors are large (rhel6)
subprocess fails in select when descriptors are large (rhel6)
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: python (Show other bugs)
6.1
All Linux
high Severity high
: rc
: ---
Assigned To: Dave Malcolm
Petr Šplíchal
: ZStream
: 616055 (view as bug list)
Depends On: 609020
Blocks: 616055 630832 668975
  Show dependency treegraph
 
Reported: 2010-11-07 03:54 EST by Haim
Modified: 2015-09-27 22:26 EDT (History)
14 users (show)

See Also:
Fixed In Version: python-2.6.6-4.el6
Doc Type: Bug Fix
Doc Text:
Previously, Python programs that used "ulimit -n" to enable communication with large numbers of subprocesses could still monitor only 1024 file descriptors at a time, due to the subprocess module using the "select" system call. This could cause an exception: ValueError: filedescriptor out of range in select() The module now uses the "poll" system call, removing this limitation.
Story Points: ---
Clone Of: 609020
Environment:
Last Closed: 2011-05-19 07:36:13 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)

  None (edit)
Comment 1 Haim 2010-11-07 03:59:17 EST
looks as if this issue wasn't resolve in RHEL6. 
just hit it when trying to start 50 vms over single host. 
had 762 processes of dd running. 

Thread-2548::ERROR::2010-11-07 10:50:39,579::clientIF::48::vds::(wrapper) Traceback (most recent call last):
  File "/usr/share/vdsm/clientIF.py", line 44, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/clientIF.py", line 887, in getVdsCapabilities
    self.machineCapabilities = self._getCapabilities()
  File "/usr/share/vdsm/clientIF.py", line 875, in _getCapabilities
    caps['emulatedMachines'] = getEmulatedMachines()
  File "/usr/share/vdsm/clientIF.py", line 794, in getEmulatedMachines
    ' -M ?') ][1:]
  File "/usr/share/vdsm/utils.py", line 576, in execAndGetOutput
    out, err = p.communicate()
  File "/usr/lib64/python2.6/subprocess.py", line 701, in communicate
    return self._communicate(input)
  File "/usr/lib64/python2.6/subprocess.py", line 1199, in _communicate
    rlist, wlist, xlist = select.select(read_set, write_set, [])
ValueError: filedescriptor out of range in select()



Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 863, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/storage/hsm.py", line 1468, in public_getVolumeSize
    apparentsize = str(volume.Volume.getVSize(sdUUID, imgUUID, volUUID, bs=1))
  File "/usr/share/vdsm/storage/volume.py", line 244, in getVSize
    return mysd.getVolumeClass().getVSize(mysd, imgUUID, volUUID, bs)
  File "/usr/share/vdsm/storage/blockVolume.py", line 47, in getVSize
    return int(int(lvm.getLVInfo(sdobj.sdUUID, volUUID)) / bs)
  File "/usr/share/vdsm/storage/lvm.py", line 1122, in getLVInfo
    return getLV(vg, lv).size
  File "/usr/share/vdsm/storage/lvm.py", line 923, in getLV
    return _lvminfo.getLv(vg, lv)
  File "/usr/share/vdsm/storage/lvm.py", line 567, in getLv
    lvs = self._reloadlvs(vgName)
  File "/usr/share/vdsm/storage/lvm.py", line 429, in _reloadlvs
    rc, out, err = self.cmd(cmd)
  File "/usr/share/vdsm/storage/lvm.py", line 354, in cmd
    return misc.execCmd(cmd)
  File "/usr/share/vdsm/storage/misc.py", line 116, in execCmd
    env=env)
  File "/usr/lib64/python2.6/subprocess.py", line 633, in __init__
    errread, errwrite)
  File "/usr/lib64/python2.6/subprocess.py", line 1049, in _execute_child
    self.pid = os.fork()
OSError: [Errno 11] Resource temporarily unavailable



  File "/usr/share/vdsm/storage/storage_mailbox.py", line 667, in run
    self._checkForMail()
  File "/usr/share/vdsm/storage/storage_mailbox.py", line 626, in _checkForMail
    (rc, in_mail, err) = misc.execCmd(cmd, sudo=False, raw=True)
  File "/usr/share/vdsm/storage/misc.py", line 116, in execCmd
    env=env)
  File "/usr/lib64/python2.6/subprocess.py", line 626, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib64/python2.6/subprocess.py", line 968, in _get_handles
    p2cread, p2cwrite = os.pipe()
OSError: [Errno 24] Too many open files
Comment 7 Dan Kenigsberg 2011-01-02 05:40:27 EST
*** Bug 616055 has been marked as a duplicate of this bug. ***
Comment 13 Laura Bailey 2011-05-09 21:11:24 EDT
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, Python programs that used "ulimit -n" to enable communication with large numbers of subprocesses could still monitor only 1024 file descriptors at a time, due to the subprocess module using the "select" system call. This could cause an exception:

    ValueError: filedescriptor out of range in select()

The module now uses the "poll" system call, removing this limitation.
Comment 14 errata-xmlrpc 2011-05-19 07:36:13 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0554.html
Comment 15 errata-xmlrpc 2011-05-19 09:07:47 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-0554.html

Note You need to log in before you can comment on or make changes to this bug.