Bug 1012290 - dnf hits "Too many open files" when downloading RPMs
dnf hits "Too many open files" when downloading RPMs
Product: Fedora
Classification: Fedora
Component: librepo (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Tomas Mlcoch
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2013-09-26 04:39 EDT by Michal Schmidt
Modified: 2013-10-07 02:37 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-10-01 04:57:47 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Michal Schmidt 2013-09-26 04:39:47 EDT
Description of problem:
I tried upgrading from F19 to F20 Alpha using dnf. First I updated the "fedora-release" package to the fc20 version, then I did "dnf update rpm dnf". Then:

 # dnf distro-sync --best
 ... [ transaction was calculated and confirmed by me ] ...
 ... [ RPMs were being downloaded ] ...
 (993/2021): OpenEXR-libs-1.7.1-6.fc20.x86_64.rpm           | 217 kB  00:00:00
 (994/2021): PackageKit-0.8.11-2.fc20.x86_64.rpm            | 620 kB  00:00:00
 (995/2021): PackageKit-command-not-found-0.8.11-2.fc20.x86_64.rpm
                                                            |  26 kB  00:00:00
 (996/2021): PackageKit-device-rebind-0.8.11-2.fc20.x86_64.rpm
                                                            |  17 kB  00:00:00
 [Errno unknown] dup(1023) failed: Too many open files: 'Input/Output error'

Here dnf exited.
There may be an fd leak somewhere in dnf or in a library it uses.

Version-Release number of selected component (if applicable):

How reproducible:
I saw the same problem on two different systems where I attempted the same upgrade.

Steps to Reproduce:
1. "dnf distro-sync" where it results in many RPMs to download.
My guess is any dnf operation that would involve a download of ~1000 RPMs would
hit the same issue. Should be more easily reproducible by setting "ulimit -n" intentionally to a lower value. I didn't try that.

Actual results:
[Errno unknown] dup(1023) failed: Too many open files: 'Input/Output error'

Expected results:
Should be able to download any number of RPMs successfully.

Additional info:
Re-running the same dnf command is a workaround. It will simply recognize already downloaded RPMs and continue downloading where it hit the error before.
Comment 1 Zdeněk Pavlas 2013-09-26 04:50:48 EDT
Confirmed, librepo does not close files after download (both metadata and rpms). Reassigning.
Comment 3 Tomas Mlcoch 2013-10-01 04:57:47 EDT
Build for F20 is ready: http://koji.fedoraproject.org/koji/taskinfo?taskID=6009388, it will be pushed to updates soon.
Comment 4 Michal Schmidt 2013-10-07 02:37:22 EDT
I am now seeing a "Too many open files" error again, but in slightly different circumstances: bug 1015957.

Note You need to log in before you can comment on or make changes to this bug.