Bug 548582

Summary: Doing lots of weird stuff with source repos. and excludes can crash yum
Product: Red Hat Enterprise Linux 5 Reporter: James Antill <james.antill>
Component: yumAssignee: James Antill <james.antill>
Status: CLOSED ERRATA QA Contact: Petr Sklenar <psklenar>
Severity: medium Docs Contact:
Priority: low    
Version: 5.2CC: bperkins, ddumas, psklenar, syeghiay
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-03-30 04:30:10 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Description James Antill 2009-12-17 15:45:42 EST
Description of problem:

 As I see it you need to do:

1. Have a repo. with N packages.

2. Have M packages which aren't "native binary arch". This could be a source repo. but it'd have to not end in the "-source" suffix (or have at least 1 none source package).

3. Have global excludes which remove P packages from the repo.

4. Have (N - M) < P. Again the easiest thing to do here is have a source repo. (so N == M), and not let yum optimize the repo. setup as a source repo.

5. Run some code which requires a full package list. Running base.pkgSack.returnPackages() will do this.

6. Run some code which requires the length of the package sack. Or add another repo. which will cause yum to look at the length. This must happen after #5.

...the fix that works for me is the trivial:

diff -ru yum-3.2.22-orig/yum/__init__.py yum-3.2.22/yum/__init__.py
--- yum-3.2.22-orig/yum/__init__.py	2009-12-17 12:00:55.000000000 -0500
+++ yum-3.2.22/yum/__init__.py	2009-12-17 15:27:04.000000000 -0500
@@ -503,8 +503,8 @@
             return self.repos.getPackageSack() # ha ha, see above
         self._pkgSack = self.repos.getPackageSack()
         
-        self.excludePackages()
         self._pkgSack.excludeArchs(archlist)
+        self.excludePackages()
         
         #FIXME - this could be faster, too.
         if repos == 'enabled':

...the big problem is that this is really hard to trigger. But aparently some people are hitting it. I don't know when it started, but it's in 5.4 and almost certaily wasn't in 5.0.

 The way I tested it was to configure a source repo (without the "-source" suffix) put in some global excludes and manually stick:

        print "JDBG: len 1:", len(self.pkgSack)
        print "JDBG: len 2:", len(self.pkgSack.returnPackages())
        print "JDBG: len 3:", len(self.pkgSack.returnPackages())
        print "JDBG: len 4:", len(self.pkgSack)

...into the code at a random point after everything would be setup.
Comment 4 Petr Sklenar 2010-02-22 10:05:26 EST
hello,
I reproduced this traceback with old version of yum and I couldn't see it with new. But I am not sure if its this bug:

1. set up src repo like you described in comment 0, with source of aspell
2. placed 'pkgSack' in file:
# grep -n4 "JDBG" /usr/lib/python2.4/site-packages/yum/__init__.py 
504-        self._pkgSack = self.repos.getPackageSack()
505-        
506-        self.excludePackages()
507-        self._pkgSack.excludeArchs(archlist)
508:        print "JDBG: len 1:", len(self.pkgSack)
509:        print "JDBG: len 2:", len(self.pkgSack.returnPackages())
510:        print "JDBG: len 3:", len(self.pkgSack.returnPackages())
511:        print "JDBG: len 4:", len(self.pkgSack)
512-
513-        #FIXME - this could be faster, too.
514-        if repos == 'enabled':
515-            repos = self.repos.listEnabled()


[root@ibm-crichton-02 repo-1]# yum list available --exclude=aspell\*Loaded plugins: rhnplugin, security
This system is not registered with RHN.
RHN support will be disabled.
Excluding Packages in global exclude list
Finished
JDBG: len 1: 1
JDBG: len 2: 1
JDBG: len 3: 1
JDBG: len 4:
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in ?
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 309, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 178, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 345, in doCommands
    self._getTs(needTsRemove)
  File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 101, in _getTs
    self._getTsInfo(remove_only)
  File "/usr/lib/python2.4/site-packages/yum/depsolve.py", line 112, in _getTsInfo
    pkgSack = self.pkgSack
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 665, in <lambda>
    pkgSack = property(fget=lambda self: self._getSacks(),
  File "/usr/lib/python2.4/site-packages/yum/__init__.py", line 511, in _getSacks
    print "JDBG: len 4:", len(self.pkgSack)
  File "/usr/lib/python2.4/site-packages/yum/packageSack.py", line 290, in __len__
    ret += len(sack)
ValueError: __len__() should return >= 0
--

Did I reproduce it in a right way?
Comment 5 James Antill 2010-02-22 11:27:54 EST
yeh, that's exactly right Petr, thanks.
Comment 6 Petr Sklenar 2010-02-23 01:55:17 EST
test procedure:

1. set up src repo like described in comment 0, with source of aspell
2. placed 'pkgSack' in file /usr/lib/python2.4/site-packages/yum/__init__.py , line 508
3. yum list available --exclude=aspell\*Loaded
OLD,yum-3.2.22-24.el5: Traceback
NEW, yum-3.2.22-25.el5: PASS
Comment 8 errata-xmlrpc 2010-03-30 04:30:10 EDT
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0254.html