Bug 491577

Summary: preupgrade: Cannot upgrade release with 'exclude=' entries in /etc/yum.conf
Product: [Fedora] Fedora Reporter: Gerry Reno <greno>
Component: preupgradeAssignee: Seth Vidal <skvidal>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: low    
Version: 9CC: wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: 1.1.0-1.fc10 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-05-28 04:13:11 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description Gerry Reno 2009-03-23 01:16:31 EDT
Description of problem:
I've been trying to upgrade some of our systems using preupgrade, which uses a combination of yum + anaconda.  On our systems without any exclusions in /etc/yum.conf or /etc/yum/pluginconf.d/versionlock.list this all works fine, either by using 'preupgrade' or by doing a 'live upgrade' by installing the new release rpms and then doing a 'yum update --obsoletes'.  

However, when we try to preupgrade a machine that has exclude entries in the yum config files then the release upgrade fails and we get a message like:

Error Message:
TurboGears-1.0.4.4-2.fc9.noarch.rpm cannot be opened.  This is due to a missing file, a corrupt package or corrupt media.


It turns out that TurboGears is one of the packages in the 'exclude=' list in /etc/yum.conf.   So it looks like there is a disconnect between the 'yum' part of the preupgrade processing and the 'anaconda' part of the process.



Version-Release number of selected component (if applicable):
preupgrade-1.0.1-1.fc9

How reproducible:
always

Steps to Reproduce:
1. install and enable yum-versionlock
2. put exclude entries in /etc/yum.conf or /etc/pluginconf.d/versionlock.list
3. run preupgrade
  
Actual results:
preupgrade fails during the anaconda stage with error message above

Expected results:
anaconda respects yum exclude entries in yum config files.

Additional info:
Comment 1 Gerry Reno 2009-03-23 09:47:51 EDT
It appears that the yum portion of preupgrade is respecting the exclude entries in the yum config files but that the anaconda portion is not in its "Upgrade existing system" logic.  

We need for anaconda to respect the exclude entries in the yum config files on the existing system.
Comment 2 Will Woods 2009-03-23 13:46:31 EDT
Strongly disagree; making anaconda parse the system yum config  leads to wildly increased complexity and a whole range of new and exciting and mysterious and unsolvable possible problems.

Much simpler to make preupgrade just follow anaconda's logic and disable excludes, with the following one-line patch:

--- preupgrade/__init__.py	2009-03-16 16:05:09 +0000
+++ preupgrade/__init__.py	2009-03-23 17:44:42 +0000
@@ -68,6 +68,7 @@
         # TODO: substitute this for yum 3.2.21 and higher
         #self.preconf.enabled_plugins = ['blacklist','whiteout']
         self._getConfig(enabled_plugins=['blacklist','whiteout'])
+        self.conf.disable_excludes = ['all']
         self.orig_enabled_repos = self.repos.listEnabled()
         if release:
             self.setup(release)

committed to bzr; will be in next preupgrade release.

In the meantime, you can comment out your excludes= lines and re-run preupgrade and everything should work as expected.
Comment 3 Gerry Reno 2009-03-23 15:12:36 EDT
Just having anaconda disable all excludes is even worse.  Now preupgrade is going to run over peoples custom-compiled packages with all their special options and they won't even know this has happened until it is too late.

At the very least the user needs to be given a message telling them that if they proceed, anaconda is not going to respect their excluded packages and may overwrite any custom-built packages that they are trying to protect with yum excludes.
Comment 4 Will Woods 2009-03-23 15:22:17 EDT
This isn't a preupgrade problem. This is how all upgrades have always worked. Preupgrade follows anaconda policy here. 

If you have custom-built foo-1.1 and Fedora 11 has foo-2.0, and you install from the Fedora 11 DVD, your custom package is getting replaced.

If you're doing a lot of custom packaging work then you need to build new versions of your packages for the target system, or use epochs.
Comment 5 Gerry Reno 2009-03-23 16:05:58 EDT
Well, anaconda upgrades may have always worked that way.  But we haven't used anaconda for upgrades until trying preupgrade recently.  
We were doing 'live upgrades' with yum.  And yum respects the excluded pkgs.  But live upgrades are risky and we were hoping to lower that risk.

And yes, the users need to custom-compile a new version of their package.  But first they need to have their workstation upgraded so that they have the environment to do that.  So the idea was to leave their existing custom-package alone and then when the distro upgrade is finished they can set about compiling new versions of their packages and yum upgrading them.

Epochs... No thanks, then you're really committed.  No chance of ever pulling from updates even if your special compiled options show up in mainstream.
Comment 6 Will Woods 2009-03-23 17:30:32 EDT
preupgrade attempts to follow default anaconda policy as closely as possible. If you want anaconda to respect your yum config you'll need to tell anaconda about your repos and excludes and so on.

Luckily we have a very powerful language for controlling anaconda - Kickstart. The kickstart 'repo' command supports an '--exclude' argument. I can probably work something out so that there's a 'repo' command in /boot/upgrade/ks.cfg, and you can then add the '--exclude' argument to ks.cfg before rebooting to handle a special case like this. Would that work?

But one thing confuses me - why is this necessary at all? preupgrade-1.x should set up F10 anaconda to just continue quietly if a package (like TurboGears) wasn't downloaded during the preupgrade phase. Are you trying to upgrade to F10, or something older?
Comment 7 Gerry Reno 2009-03-23 18:25:28 EDT
QUOTE: Would that work?
Yes, it might if it isn't too awkward to use while upgrading the machines.  Each machine could have a different set of excludes so it would have to be configurable for each machine.


And I don't know why it's necessary.  All I know is that on any machine with a pkg entry in /etc/yum.conf exclude= line under the [main] area, that pkg is listed in the anaconda error message at "Starting installation process. ..."
So it looks like yum and anaconda end up working from two different lists.  Because the yum exclude does not allow an updating package to be downloaded.  Yet anaconda then tries to install this excluded package and therefore the "missing file" error.

If you want to try it, just put something that has an update like maybe exclude=bzip2 in /etc/yum.conf and try doing a preupgrade.  It will error.
Comment 8 Fedora Update System 2009-04-28 18:07:12 EDT
preupgrade-1.1.0-0.pre2.fc10 has been submitted as an update for Fedora 10.
http://admin.fedoraproject.org/updates/preupgrade-1.1.0-0.pre2.fc10
Comment 9 Fedora Update System 2009-05-02 12:41:27 EDT
preupgrade-1.1.0-0.pre2.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update preupgrade'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-4211
Comment 10 Fedora Update System 2009-05-13 18:46:35 EDT
preupgrade-1.1.0-1.fc9 has been submitted as an update for Fedora 9.
http://admin.fedoraproject.org/updates/preupgrade-1.1.0-1.fc9
Comment 11 Fedora Update System 2009-05-13 18:47:33 EDT
preupgrade-1.1.0-1.fc11 has been submitted as an update for Fedora 11.
http://admin.fedoraproject.org/updates/preupgrade-1.1.0-1.fc11
Comment 12 Fedora Update System 2009-05-15 19:33:34 EDT
preupgrade-1.1.0-1.fc10 has been pushed to the Fedora 10 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update preupgrade'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F10/FEDORA-2009-4211
Comment 13 Fedora Update System 2009-05-28 03:57:46 EDT
preupgrade-1.1.0-1.fc11 has been pushed to the Fedora 11 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 14 Fedora Update System 2009-05-28 04:10:34 EDT
preupgrade-1.1.0-1.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 15 Fedora Update System 2009-05-28 04:12:32 EDT
preupgrade-1.1.0-1.fc10 has been pushed to the Fedora 10 stable repository.  If problems still persist, please make note of it in this bug report.