Bug 715149

Summary: Preupgrade F10 to F11 fails with IOError too many files open
Product: [Fedora] Fedora Reporter: nstickels
Component: preupgradeAssignee: Richard Hughes <richard>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 10CC: ofeeley, richard, wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 509360 Environment:
Last Closed: 2011-11-09 20:01:07 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description nstickels 2011-06-22 02:29:40 UTC 
+++ This bug was initially created as a clone of Bug #509360 +++

Description of problem:

Fedora 9 preupgrade to Fedora 10 fails with:

Downloading 570.4MB
Available disk space for /var/cache/yum/preupgrade: 1.8GB
Upgrade requires 500.0MB
Available disk space for /usr: 1.8GB
Traceback (most recent call last):
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 766, in <module>
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 374, in __init__
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 257, in _do_main
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 539, in main_preupgrade
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1396, in retrieveMD
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1400, in _retrieveMD
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1305, in <lambda>
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1297, in _getRepoXML
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1287, in _loadRepoXML
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1271, in _groupLoadRepoXML
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1117, in _commonLoadRepoXML
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 1087, in _latestRepoXML
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 948, in _saveOldRepoXML
  File "/usr/lib/python2.5/shutil.py", line 91, in copy2
  File "/usr/lib/python2.5/shutil.py", line 46, in copyfile
IOError: [Errno 24] Too many open files: '/var/cache/yum/preupgrade/repomd.xml'


Version-Release number of selected component (if applicable):

preupgrade-1.1.0-1.fc9.noarch

How reproducible:

Every time.  Also fails with same message if I try to do a Fedora9 -> Fedora 11.

Steps to Reproduce:
1. sudo preupgrade
2. preupgrade reports that /boot has too little space for the install.img but will download it later over the wired network.  Machine is correctly connected with wired network and downloads packages

3. let preupgrade proceed without intervention
  
Actual results:

Preupgrade steps through its GUI steps and I go to bed while it happily downloads packages. Upon return in the morning the GUI has disappeared and all that's left is the backtrace above and apparently a cache full of packages which should do the trick. I've tried it twice since then with the same error.

Expected results:

painless, magical upgrade to a later version of Fedora

Additional info:

/var/cache/yum is full of needed packages:
[ush@localhost ~]$ ls /var/cache/yum/preupgrade | wc -l
2225

[ush@localhost ~]$ ls /var/cache/yum/preupgrade | egrep -v "*.rpm"
4254aba126bd4df25bc40ebf1db3427b5726626f44286fdb0bced49dfdcec70b-Fedora-11-comps.xml
cachecookie
d8d34dabbba6003a7d8742aa95d5463f8c32cf744830641a5bb016a654b28cdc-primary.sqlite
mirrorlist.txt
packages
repodata
repomd.xml

--- Additional comment from ofeeley on 2009-07-02 19:09:17 EDT ---

I'm guessing this will be closed as a WONTFIX due to EOL of Fedora 9.  Fair enough. Thought it might be of interest anyway.

--- Additional comment from ofeeley on 2009-07-03 07:32:41 EDT ---

For anyone interested a brute-force solution of this was achieved by:
1. Disabling all but the official fedora repositories
2. yum removing several large package groups including openoffice and all of KDE

Unfortunately I did these as one single action and cannot say if only one of them would suffice or if both are needed.

It was then necessary to do a grub-install to which mapped hd(0,1) to /dev/sda

After that all was successful

--- Additional comment from wwoods on 2009-07-05 12:35:30 EDT ---

Sounds like a bug in F9 yum or createrepo.. or you have waaay more packages installed than anyone else has ever tested with. Heh.

It'd help if you could use lsof to figure out what files preupgrade has open, but since you've apparently finished your upgrade that may not be so easy for you to reproduce.

So: how many packages did you have installed? 

Seth, are you aware of any bugs in F9 yum/createrepo/python that would leave files open after they've been used?

--- Additional comment from svidal on 2009-07-06 10:56:14 EDT ---

A very old urlgrabber had some file handle leaks - but that's the last think I recall

--- Additional comment from triage.org on 2009-07-14 12:30:01 EDT ---


Fedora 9 changed to end-of-life (EOL) status on 2009-07-10. Fedora 9 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.

--------------------------------------------------------------------

I have reproduced this same problem trying to run preupgrade to upgrade from Fedora 10 to 11:


[root@neil upgrade]# /usr/bin/preupgrade
/usr/lib/python2.5/site-packages/yum/__init__.py:203: UserWarning: Use .preconf instead of passing args to _getConfig
  warnings.warn('Use .preconf instead of passing args to _getConfig')
Loaded plugins: blacklist, fastestmirror, refresh-packagekit, whiteout
Detected in-progress upgrade to Fedora 11 (Leonidas)
preupgrade-main (mirrorlist) 
  url: http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-11&arch=$basearch
  now: http://mirrors.fedoraproject.org/mirrorlist?repo=fedora-11&arch=x86_64
Loading mirror speeds from cached hostfile
 * fedora: ftp.heanet.ie
 * freshrpms: ayo.es6.freshrpms.net
 * rpmfusion-free: mirror.liberty.edu
 * rpmfusion-free-updates: mirror.liberty.edu
 * rpmfusion-nonfree: mirror.liberty.edu
 * rpmfusion-nonfree-updates: mirror.liberty.edu
preupgrade (mirrorlist) 
  url: http://mirrors.fedoraproject.org/mirrorlist?path=pub/fedora/linux/releases/11/Fedora/$basearch/os
  now: http://mirrors.fedoraproject.org/mirrorlist?path=pub/fedora/linux/releases/11/Fedora/x86_64/os
Checking for new repos for mirrors
 * preupgrade-main: ftp.cuhk.edu.hk
preupgrade-adobe-linux-i386 (baseurl) 
  url: http://linuxdownload.adobe.com/linux/i386/
  now: http://linuxdownload.adobe.com/linux/i386/
Checking for new repos for mirrors
preupgrade-fedora (baseurl) 
  url: http://download.fedora.redhat.com/pub/fedora/linux/releases/11/Everything/x86_64/os/
  now: http://download.fedora.redhat.com/pub/fedora/linux/releases/11/Everything/x86_64/os/
Checking for new repos for mirrors
preupgrade-freshrpms (mirrorlist) 
  url: http://ayo.freshrpms.net/fedora/linux/11/mirrors-freshrpms
  now: http://ayo.freshrpms.net/fedora/linux/11/mirrors-freshrpms
Checking for new repos for mirrors
Can't set up new repo preupgrade-freshrpms - removing
preupgrade-google-chrome (baseurl) 
  url: http://dl.google.com/linux/chrome/rpm/stable/x86_64
  now: http://dl.google.com/linux/chrome/rpm/stable/x86_64
Checking for new repos for mirrors
preupgrade-rpmfusion-free (mirrorlist) 
  url: http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-11&arch=x86_64
  now: http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-11&arch=x86_64
Checking for new repos for mirrors
Not using downloaded repomd.xml because it is older than what we have:
  Current   : Tue Nov  9 12:42:04 2010
  Downloaded: Sun Jun  7 05:25:41 2009
preupgrade-rpmfusion-free-updates (mirrorlist) 
  url: http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-11&arch=x86_64
  now: http://mirrors.rpmfusion.org/mirrorlist?repo=free-fedora-updates-released-11&arch=x86_64
Checking for new repos for mirrors
 * preupgrade-rpmfusion-free: mirror.liberty.edu
Not using downloaded repomd.xml because it is older than what we have:
  Current   : Thu Mar 17 01:53:05 2011
  Downloaded: Tue Jul 13 01:26:11 2010
preupgrade-rpmfusion-nonfree (mirrorlist) 
  url: http://mirrors.rpmfusion.org/mirrorlist?repo=nonfree-fedora-11&arch=x86_64
  now: http://mirrors.rpmfusion.org/mirrorlist?repo=nonfree-fedora-11&arch=x86_64
Checking for new repos for mirrors
 * preupgrade-rpmfusion-free-updates: mirror.liberty.edu
Not using downloaded repomd.xml because it is older than what we have:
  Current   : Tue Nov  9 12:45:27 2010
  Downloaded: Mon Jun  8 08:24:20 2009
preupgrade-rpmfusion-nonfree-updates (mirrorlist) 
  url: http://mirrors.rpmfusion.org/mirrorlist?repo=nonfree-fedora-updates-released-11&arch=x86_64
  now: http://mirrors.rpmfusion.org/mirrorlist?repo=nonfree-fedora-updates-released-11&arch=x86_64
Checking for new repos for mirrors
 * preupgrade-rpmfusion-nonfree: mirror.liberty.edu
Not using downloaded repomd.xml because it is older than what we have:
  Current   : Thu Mar 17 01:49:17 2011
  Downloaded: Sun Jul  4 15:26:02 2010
preupgrade-updates (baseurl) 
  url: http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/11/x86_64/
  now: http://archives.fedoraproject.org/pub/archive/fedora/linux/updates/11/x86_64/
Checking for new repos for mirrors
 * preupgrade-rpmfusion-nonfree-updates: mirror.liberty.edu
Fetched treeinfo from http://mirrordenver.fdcservers.net/fedora/releases/11/Fedora/x86_64/os//.treeinfo
treeinfo timestamp: Tue Jun  2 17:15:53 2009
/boot/upgrade/vmlinuz checksum OK
/boot/upgrade/initrd.img checksum OK
/boot/upgrade/install.img checksum OK
Checking for new repos for mirrors
xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libaudit.so.0()(64bit) is needed by package xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 (preupgrade-updates)
rhpl-0.218-1.x86_64 from installed has depsolving problems
  --> Missing Dependency: python(abi) = 2.5 is needed by package rhpl-0.218-1.x86_64 (installed)
boost-1.37.0-9.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libicudata.so.40 is needed by package boost-1.37.0-9.fc11.i586 (preupgrade-updates)
alchemist-1.0.37-4.fc9.i386 from installed has depsolving problems
  --> Missing Dependency: python(abi) = 2.5 is needed by package alchemist-1.0.37-4.fc9.i386 (installed)
rhpxl-1.9-3.fc10.x86_64 from installed has depsolving problems
  --> Missing Dependency: libpython2.5.so.1.0()(64bit) is needed by package rhpxl-1.9-3.fc10.x86_64 (installed)
xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 from preupgrade-updates has depsolving problems
  --> Missing Dependency: xkbdata is needed by package xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 (preupgrade-updates)
alchemist-1.0.37-4.fc9.x86_64 from installed has depsolving problems
  --> Missing Dependency: python-abi = 2.5 is needed by package alchemist-1.0.37-4.fc9.x86_64 (installed)
alchemist-1.0.37-4.fc9.x86_64 from installed has depsolving problems
  --> Missing Dependency: python(abi) = 2.5 is needed by package alchemist-1.0.37-4.fc9.x86_64 (installed)
xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libaudit.so.0()(64bit) is needed by package xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 (preupgrade-updates)
boost-1.37.0-9.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libicudata.so.40 is needed by package boost-1.37.0-9.fc11.i586 (preupgrade-updates)
glibc-2.12.90-17.x86_64 from preupgrade-fedora has depsolving problems
  --> glibc conflicts with kernel
kudzu-1.2.85-1.x86_64 from installed has depsolving problems
  --> Missing Dependency: python(abi) = 2.5 is needed by package kudzu-1.2.85-1.x86_64 (installed)
glibc-2.12.90-17.i686 from preupgrade-fedora has depsolving problems
  --> glibc conflicts with kernel
4:perl-5.10.0-82.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: perl-libs = 4:5.10.0-82.fc11 is needed by package 4:perl-5.10.0-82.fc11.i586 (preupgrade-updates)
xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 from preupgrade-updates has depsolving problems
  --> Missing Dependency: xkbdata is needed by package xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 (preupgrade-updates)
samba-winbind-3.4.7-0.50.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: samba-common = 3.4.7-0.50.fc11 is needed by package samba-winbind-3.4.7-0.50.fc11.i586 (preupgrade-updates)
boost-1.37.0-9.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libicui18n.so.40 is needed by package boost-1.37.0-9.fc11.i586 (preupgrade-updates)
alchemist-1.0.37-4.fc9.i386 from installed has depsolving problems
  --> Missing Dependency: python-abi = 2.5 is needed by package alchemist-1.0.37-4.fc9.i386 (installed)
rhpxl-1.9-3.fc10.x86_64 from installed has depsolving problems
  --> Missing Dependency: python(abi) = 2.5 is needed by package rhpxl-1.9-3.fc10.x86_64 (installed)
xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libcrypto.so.8()(64bit) is needed by package xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 (preupgrade-updates)
boost-1.37.0-9.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libicuuc.so.40 is needed by package boost-1.37.0-9.fc11.i586 (preupgrade-updates)
samba-winbind-3.4.7-0.50.fc11.i586 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libtalloc.so.1 is needed by package samba-winbind-3.4.7-0.50.fc11.i586 (preupgrade-updates)
rhpl-0.218-1.x86_64 from installed has depsolving problems
  --> Missing Dependency: libpython2.5.so.1.0()(64bit) is needed by package rhpl-0.218-1.x86_64 (installed)
xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 from preupgrade-updates has depsolving problems
  --> Missing Dependency: libssl.so.8()(64bit) is needed by package xorg-x11-server-Xorg-1.6.4-0.1.fc11.x86_64 (preupgrade-updates)
Downloading 1.8GB
Available disk space for /var/cache/yum/preupgrade: 9.3GB
Traceback (most recent call last):
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 773, in <module>
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 376, in __init__
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 259, in _do_main
  File "/usr/share/preupgrade/preupgrade-gtk.py", line 525, in main_preupgrade
  File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 1446, in downloadPkgs
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 830, in getPackage
  File "/usr/lib/python2.5/site-packages/yum/yumRepo.py", line 803, in _getFile
  File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 412, in urlgrab
  File "/usr/lib/python2.5/site-packages/urlgrabber/mirror.py", line 398, in _mirror_try
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 933, in urlgrab
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 851, in _retry
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 921, in retryfunc
  File "/usr/lib/python2.5/site-packages/urlgrabber/grabber.py", line 1227, in _do_grab
IOError: [Errno 24] Too many open files: u'/var/cache/yum/preupgrade-updates/packages/libvolume_id-141-8.fc11.i586.rpm'
Comment 1 John Poelstra 2011-11-09 20:01:07 UTC 
    Thank you for your bug report. 

    We are sorry, but the Fedora Project is no longer releasing bug fixes or any other updates for this version of Fedora. This bug will be set to CLOSED:WONTFIX to reflect this, but please reopen it if the problem persists after upgrading to the latest version of Fedora, which is available from: 

    http://fedoraproject.org/get-fedora