Bug 468282 - reposync fails with too many open files
Summary: reposync fails with too many open files
Keywords:
Status: CLOSED DUPLICATE of bug 460694
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: yum-utils
Version: 5.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: James Antill
QA Contact: James Antill
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-10-23 21:03 UTC by James Cape
Modified: 2017-11-20 17:00 UTC (History)
0 users

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-10-23 21:22:24 UTC
Target Upstream Version:
Embargoed:
jamescape777: needinfo-


Attachments (Terms of Use)

Description James Cape 2008-10-23 21:03:32 UTC
Description of problem:

When using reposync to mirror an existing repository (e.g. EPEL), it will fail with "too many open files" after downloading ~1000 files (default 1024 open files limit). This only occurs with newly downloaded files, not existing files.

Version-Release number of selected component (if applicable):

yum-utils-1.1.10-9.el5

How reproducible:

Consistently


Steps to Reproduce:
1. $ cat /var/www/cobbler/repo_mirror/.origin/epel-x86_64-5.repo
[epel-x86_64-5]
name=epel-x86_64-5
baseurl=http://fedora.mirror.facebook.com/epel/5/x86_64
enabled=1
priority=99
gpgcheck=0

2. $ /usr/bin/reposync -l --config=/var/www/cobbler/repo_mirror/.origin/epel-x86_64-5.repo --repoid=epel-x86_64-5 --download_path=/var/www/cobbler/repo_mirror -a x86_64

  
Actual results:

Traceback (most recent call last):
  File "/usr/bin/reposync", line 264, in ?
  File "/usr/bin/reposync", line 245, in main
  File "/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 652, in getPackage
  File "/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 630, in _getFile
  File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 411, in urlgrab
  File "/usr/lib/python2.4/site-packages/urlgrabber/mirror.py", line 397, in _mirror_try
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 927, in urlgrab
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 845, in _retry
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 915, in retryfunc
  File "/usr/lib/python2.4/site-packages/urlgrabber/grabber.py", line 1200, in _do_grab
IOError: [Errno 24] Too many open files: '/var/www/cobbler/repo_mirror/epel-x86_64-5/ntfsprogs-1.13.1-5.el5.x86_64.rpm'
cobbler reposync failed


Expected results:

"James, you've now created a repository that you can use for rollout testing and caching, and no longer have to have the workstation(s) DoS the internet once a day. Here's a raise and a promotion to commend you for your fine work!"


Additional info:

Perhaps the expected results are a little much. It should at least close the file once it's downloaded it...

Comment 1 James Antill 2008-10-23 21:22:24 UTC
Can you access/test the 5.3 test package for python?

*** This bug has been marked as a duplicate of bug 460694 ***


Note You need to log in before you can comment on or make changes to this bug.