Bug 195312 - solaris2mpm *.pkg dies with too many open files
solaris2mpm *.pkg dies with too many open files
Status: CLOSED CURRENTRELEASE
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Solaris (Show other bugs)
500
All Linux
medium Severity medium
: ---
: ---
Assigned To: Pete Vetere
Brandon Perkins
:
Depends On:
Blocks: 196477
  Show dependency treegraph
 
Reported: 2006-06-14 14:40 EDT by Matthew Davis
Modified: 2007-06-26 11:12 EDT (History)
1 user (show)

See Also:
Fixed In Version: rhn410
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-07-19 18:21:12 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Matthew Davis 2006-06-14 14:40:27 EDT
1. Install solaris client
2. Create .pkg files from the install tree

mount bungee.rhnsun.rdu.redhat.com:/export/jump_5.10_sparc /mnt
mkdir /packages
for PKG in `ls /mnt/Solaris_10/Product/` ; do pkgtrans -s
/mnt/Solaris_10/Product/ /packages/$PKG.pkg $PKG ; done


3. Start converting the .pkg files to 

cd /packages
solaris2mpm *.pkg

4. Wait... and get this:

Opening archive, this may take a while
Traceback (most recent call last):
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 178, in _run
    archive_parser = get_archive_parser(archive, tempdir=options.TEMPDIR)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 390, in get_archive_parser
    return Class(archive, tempdir)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 52, in __init__
    self._explode(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 84, in _explode
    self._copy_archive(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 72, in _copy_archive
    shutil.copyfile(archive, a_path)
  File "/opt/redhat/rhn/solaris/lib/python2.4/shutil.py", line 48, in copyfile
    fdst = open(dst, 'wb')
IOError: [Errno 24] Too many open files: '/tmp/tmphDYn53/SUNWxildh.pkg'
Error creating mpm for /packages/SUNWxildh.pkg: [Errno 24] Too many open files:
'/tmp/tmphDYn53/SUNWxildh.pkg'

It created for me, roughly 1000 mpm's.
Comment 1 Matthew Davis 2006-06-14 15:32:12 EDT
Set ulimit -n 4096 & reran soliaris *.mpm with the same effect.
Comment 2 Pete Vetere 2006-06-15 16:19:25 EDT
Fixed.  The problem was that we weren't closing file descriptors in solaris2mpm.
 We do now.
Comment 4 Brandon Perkins 2006-06-20 16:05:04 EDT
Fails QA.  Using rhn-solaris-bootstrap-4.1.0-14-sparc-sol10:

  Opening archive, this may take a while
Traceback (most recent call last):
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.
py", line 178, in _run
    archive_parser = get_archive_parser(archive, tempdir=options.TEMPDIR)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 390, in get_archive_parser
    return Class(archive, tempdir)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 52, in __init__
    self._explode(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 84, in _explode
    self._copy_archive(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 72, in _copy_archive
    shutil.copyfile(archive, a_path)
  File "/opt/redhat/rhn/solaris/lib/python2.4/shutil.py", line 48, in copyfile
    fdst = open(dst, 'wb')
IOError: [Errno 24] Too many open files: '/tmp/tmpkZU0RY/TSIpgxw.pkg'
Error creating mpm for /packages/TSIpgxw.pkg: [Errno 24] Too many open files: '/
tmp/tmpkZU0RY/TSIpgxw.pkg'
Comment 6 Brandon Perkins 2006-06-21 15:39:30 EDT
Fails QA.  Same steps as above except, instead of doing 'solaris2mpm *.pkg' just
do it for 'SUNWsom' and/or 'SUNWsoam':

# solaris2mpm SUNWsom.pkg

And the result is:

Opening archive, this may take a while
Traceback (most recent call last):
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 204, in _run
    pkg_mpm = create_pkg_mpm(archive_parser, prefix=dir)
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 446, in create_pkg_mpm
    header['sigmd5'] = md5sum_for_stream(package.payload_stream)
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 568, in md5sum_for_stream
    md5obj = md5.new(data_stream.read())
MemoryError
Error creating mpm for /packages/SUNWsom.pkg: 

Both of these packages are over 250MB.  This is technically no longer the same
issue...so let me know if this should be split out into another...but the fact
of the matter is that I still am unable to run the test plan as written.
Comment 8 Brandon Perkins 2006-06-23 12:50:03 EDT
Verified.  The real problem is fixed.  And the secondary problem is a result of
system resources which cannot be fixed without hardware upgrade...and lets face
it, those packages are way too big to be trying to process on a slow machine
with little ram and storage capacity.

# solaris2mpm SUNWsom.pkg
Opening archive, this may take a while
Writing SUNWsom-7.0.4-52.2004.12.07.sparc-solaris.mpm
Comment 9 Beth Nackashi 2006-07-19 18:21:12 EDT
closing -- currentrelease

Note You need to log in before you can comment on or make changes to this bug.