Bug 195312 - solaris2mpm *.pkg dies with too many open files
Summary: solaris2mpm *.pkg dies with too many open files
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Solaris
Version: 500
Hardware: All
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Pete Vetere
QA Contact: Brandon Perkins
URL:
Whiteboard:
Depends On:
Blocks: 196477
TreeView+ depends on / blocked
 
Reported: 2006-06-14 18:40 UTC by Matthew Davis
Modified: 2007-06-26 15:12 UTC (History)
1 user (show)

Fixed In Version: rhn410
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2006-07-19 22:21:12 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Matthew Davis 2006-06-14 18:40:27 UTC
1. Install solaris client
2. Create .pkg files from the install tree

mount bungee.rhnsun.rdu.redhat.com:/export/jump_5.10_sparc /mnt
mkdir /packages
for PKG in `ls /mnt/Solaris_10/Product/` ; do pkgtrans -s
/mnt/Solaris_10/Product/ /packages/$PKG.pkg $PKG ; done


3. Start converting the .pkg files to 

cd /packages
solaris2mpm *.pkg

4. Wait... and get this:

Opening archive, this may take a while
Traceback (most recent call last):
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 178, in _run
    archive_parser = get_archive_parser(archive, tempdir=options.TEMPDIR)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 390, in get_archive_parser
    return Class(archive, tempdir)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 52, in __init__
    self._explode(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 84, in _explode
    self._copy_archive(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
line 72, in _copy_archive
    shutil.copyfile(archive, a_path)
  File "/opt/redhat/rhn/solaris/lib/python2.4/shutil.py", line 48, in copyfile
    fdst = open(dst, 'wb')
IOError: [Errno 24] Too many open files: '/tmp/tmphDYn53/SUNWxildh.pkg'
Error creating mpm for /packages/SUNWxildh.pkg: [Errno 24] Too many open files:
'/tmp/tmphDYn53/SUNWxildh.pkg'

It created for me, roughly 1000 mpm's.

Comment 1 Matthew Davis 2006-06-14 19:32:12 UTC
Set ulimit -n 4096 & reran soliaris *.mpm with the same effect.

Comment 2 Pete Vetere 2006-06-15 20:19:25 UTC
Fixed.  The problem was that we weren't closing file descriptors in solaris2mpm.
 We do now.


Comment 4 Brandon Perkins 2006-06-20 20:05:04 UTC
Fails QA.  Using rhn-solaris-bootstrap-4.1.0-14-sparc-sol10:

  Opening archive, this may take a while
Traceback (most recent call last):
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.
py", line 178, in _run
    archive_parser = get_archive_parser(archive, tempdir=options.TEMPDIR)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 390, in get_archive_parser
    return Class(archive, tempdir)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 52, in __init__
    self._explode(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 84, in _explode
    self._copy_archive(archive)
  File "/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/archive.py",
 line 72, in _copy_archive
    shutil.copyfile(archive, a_path)
  File "/opt/redhat/rhn/solaris/lib/python2.4/shutil.py", line 48, in copyfile
    fdst = open(dst, 'wb')
IOError: [Errno 24] Too many open files: '/tmp/tmpkZU0RY/TSIpgxw.pkg'
Error creating mpm for /packages/TSIpgxw.pkg: [Errno 24] Too many open files: '/
tmp/tmpkZU0RY/TSIpgxw.pkg'


Comment 6 Brandon Perkins 2006-06-21 19:39:30 UTC
Fails QA.  Same steps as above except, instead of doing 'solaris2mpm *.pkg' just
do it for 'SUNWsom' and/or 'SUNWsoam':

# solaris2mpm SUNWsom.pkg

And the result is:

Opening archive, this may take a while
Traceback (most recent call last):
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 204, in _run
    pkg_mpm = create_pkg_mpm(archive_parser, prefix=dir)
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 446, in create_pkg_mpm
    header['sigmd5'] = md5sum_for_stream(package.payload_stream)
  File
"/opt/redhat/rhn/solaris/lib/python2.4/site-packages/rhnpush/solaris2mpm.py",
line 568, in md5sum_for_stream
    md5obj = md5.new(data_stream.read())
MemoryError
Error creating mpm for /packages/SUNWsom.pkg: 

Both of these packages are over 250MB.  This is technically no longer the same
issue...so let me know if this should be split out into another...but the fact
of the matter is that I still am unable to run the test plan as written.

Comment 8 Brandon Perkins 2006-06-23 16:50:03 UTC
Verified.  The real problem is fixed.  And the secondary problem is a result of
system resources which cannot be fixed without hardware upgrade...and lets face
it, those packages are way too big to be trying to process on a slow machine
with little ram and storage capacity.

# solaris2mpm SUNWsom.pkg
Opening archive, this may take a while
Writing SUNWsom-7.0.4-52.2004.12.07.sparc-solaris.mpm


Comment 9 Beth Nackashi 2006-07-19 22:21:12 UTC
closing -- currentrelease


Note You need to log in before you can comment on or make changes to this bug.