Bug 591382 - Yum update fails with "list index out of range"
Summary: Yum update fails with "list index out of range"
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: yum
Version: 12
Hardware: All
OS: Linux
low
medium
Target Milestone: ---
Assignee: Seth Vidal
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-05-12 05:05 UTC by aktigger99645
Modified: 2014-01-21 23:14 UTC (History)
13 users (show)

Fixed In Version: yum-3.2.28-3.fc14
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2010-06-01 15:39:12 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
Old yumdb-package-checksums (175.07 KB, text/plain)
2010-05-12 23:46 UTC, aktigger99645
no flags Details
New yumdb-package-checksums (175.19 KB, text/plain)
2010-05-12 23:46 UTC, aktigger99645
no flags Details
yum history (1.72 KB, text/plain)
2010-05-13 11:58 UTC, aktigger99645
no flags Details
Computer 1 yum history info 82 83 (1.91 KB, text/plain)
2010-05-16 07:13 UTC, aktigger99645
no flags Details
new checksum (168.75 KB, text/plain)
2010-05-16 07:13 UTC, aktigger99645
no flags Details
old checksum (168.04 KB, application/octet-stream)
2010-05-16 07:14 UTC, aktigger99645
no flags Details
yum history (1.75 KB, text/plain)
2010-05-16 07:15 UTC, aktigger99645
no flags Details

Description aktigger99645 2010-05-12 05:05:40 UTC
Description of problem: Try to yum update or rpm update, get IndexError:  list index out of range.  Happened on 2 different machines, at 2 different times, no discernible link.  rebuilddb does not help.


Version-Release number of selected component (if applicable): yum-3.2.27-2.fc12.noarch


How reproducible:  Try to run yum or rpm update on both computers


Steps to Reproduce:
1.  Run yum update from command line or notification applet
2.
3.
  
Actual results:  Finds updates, downloads them, checks deps, fails on installation




Expected results:  Updates should be installed


Additional info:
Traceback (most recent call last):
  File "/usr/share/PackageKit/helpers/yum/yumBackend.py", line 2123, in _runYumTransaction
    rpmDisplay=rpmDisplay)
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 4170, in processTransaction
    self._doTransaction(callback,display=rpmDisplay)
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 4284, in _doTransaction
    self.runTransaction( cb=cb )
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 1182, in runTransaction
    self.rpmdb.transactionResultVersion(self.tsInfo.futureRpmDBVersion())
  File "/usr/lib/python2.6/site-packages/yum/transactioninfo.py", line 580, in futureRpmDBVersion
    self.rpmdb.preloadPackageChecksums()
  File "/usr/lib/python2.6/site-packages/yum/rpmsack.py", line 749, in preloadPackageChecksums
    pkg = self.searchNevra(n, e, v, r, a)[0]
IndexError: list index out of range

Comment 1 Richard Hughes 2010-05-12 06:37:53 UTC
Why on earth was this changed to a PackageKit bug? PackageKit is just calling runYumTransaction and yum is exploding internally with an obscure error?

Comment 2 James Antill 2010-05-12 14:30:26 UTC
Sorry, richard I misread the traceback.


sbills, have you done anything "weird" to these boxes?

Can you save out /var/cache/yum/*/12/installed ... and then run:

yum clean rpmdb

...does that fix it? If so can you upload the old and new installed/yumdb-package-checksums ?

Comment 3 aktigger99645 2010-05-12 23:46:32 UTC
Created attachment 413600 [details]
Old yumdb-package-checksums

Comment 4 aktigger99645 2010-05-12 23:46:53 UTC
Created attachment 413601 [details]
New yumdb-package-checksums

Comment 5 aktigger99645 2010-05-12 23:49:00 UTC
Nothing weird, just ordinary-use boxes.  Weird thing is, I've had this happen in the past and couldn't fix it, so just waited for the next Fedora release and did a clean install.  Strange that it would happen on two boxes this time.

Files attached.  Thanks for the help, I'll have to remember this for the future.  Sorry for filing a bug against a file corruption on my side.

Comment 6 James Antill 2010-05-13 05:10:36 UTC
 This specific problem is due to the cache being wrong, and yum not being able to spot it and thus. yumdb-package-checksums pointing to packages which don't exist. That couldn't happen before 3.2.27, because before then we didn't have yumdb-package-checksums. And we didn't cache anything before 3.2.26. F-12 came with 3.2.25-1, so it's just not possible for you to have seen this before.

 As to how this is happening to you now, I'm not sure.

 What does "yum history" show? Have any transactions failed/aborted?

 According to the checksum files your rpmdb is very different. AFAICS libvdpau has been added maybe mplayer-common too. A bunch of packages have been upgraded, neon looks like it was removed.
 It's hard to imagine how this could happen but the rpmdb cache still be "valid".
 Have you used anything other than yum to install/remove things?
 Are you using something other than ext3, like btrfs or NFS? Or copying /var/cache/yum between machines?

Comment 7 aktigger99645 2010-05-13 11:58:32 UTC
Created attachment 413732 [details]
yum history

Comment 8 aktigger99645 2010-05-13 12:02:42 UTC
Maybe it was a different problem, I don't know.  I do know that updates would stop working on random machines, but don't remember the exact problem.

Yum history is attached

I've only used the yum applet to update, and occasionally the command line 'sudo yum update -v' to see the behind-the-scenes stuff.

Filesystem is ext4.  And no, I haven't copied /var/cache/yum from another machine.

Comment 9 James Antill 2010-05-13 14:36:18 UTC
 Wow...

ID     | Login user             | Date and time    | Action(s)      | Altered
-------------------------------------------------------------------------------
    91 | *****                  | 2010-05-12 15:40 | I, U           |   44   
    90 | *****                  | 2010-05-10 19:28 | I, U           |   28 **
    89 | *****                  | 2010-05-09 02:31 | I, U           |   21 **
    88 | *****                  | 2010-05-06 23:05 | Update         |    1 **
    87 | *****                  | 2010-05-06 23:04 | Update         |    1 **
    86 | *****                  | 2010-05-06 23:04 | Update         |    1 **
    85 | *****                  | 2010-05-06 23:03 | Update         |    2 **
    84 | *****                  | 2010-05-06 22:56 | Update         |    5 **
    83 | System <unset>         | 2010-05-06 22:49 | Update         |    5 **
    82 | System <unset>         | 2010-05-05 22:02 | Update         |    3   


> I've only used the yum applet to update, and occasionally the command line
> 'sudo yum update -v' to see the behind-the-scenes stuff.

By yum applet, do you mean the thing in the gnome-panel?

Comment 10 James Antill 2010-05-13 14:53:48 UTC
Can you attach:

yum history info 82 83 86 87 88

Comment 11 James Antill 2010-05-13 23:45:35 UTC
I've been looking at this today, and fwiw 84-90 are probably just due to the "index out of range" ... as that's done after the history transaction starts, but just before the actual transaction.
Still no idea how it happened to you (twice!) But we'll probably have some workarounds in 3.2.28 (any help you can give us to find the bug would be a big help though :).

Comment 12 aktigger99645 2010-05-14 08:59:01 UTC
FWIW, the ***** obfuscate the logged-in username (they were the same non-root user in each case.)  And yes, yum applet = gnome-panel thingy. :-)

Do you still want yum history info 82 83 86 87 88?

I'm willing to do whatever I can to help track this down.  Would you like the information from both machines?  Keep in mind, I'm not a programmer, but am willing to learn.

Comment 13 James Antill 2010-05-14 14:01:40 UTC
Yes, at least "yum history info 82 83" will still be helpful. And a history list from the other machine (and history info from the last working, and first non-working transactions). I can't promise anything, but we'll try :).

Comment 14 aktigger99645 2010-05-16 07:13:05 UTC
Created attachment 414315 [details]
Computer 1 yum history info 82 83

yum history info 82 83 from first computer, as requested

Comment 15 aktigger99645 2010-05-16 07:13:56 UTC
Created attachment 414317 [details]
new checksum

computer 2 new yumdb-package-checksums as requested

Comment 16 aktigger99645 2010-05-16 07:14:44 UTC
Created attachment 414318 [details]
old checksum

Computer 2 old yumdb-package-checksums as requested

Comment 17 aktigger99645 2010-05-16 07:15:26 UTC
Created attachment 414319 [details]
yum history

Computer 2 yum history as requested

Comment 18 James Antill 2010-05-24 04:50:56 UTC
Can you run:

yum list '*yum*'

...also what plugins are you running with?

Comment 19 James Antill 2010-05-24 13:29:05 UTC
Actually ... if you can take a look at 595063 and see if there's anything there that seems familiar.

Comment 20 James Antill 2010-05-24 17:02:56 UTC
I don't suppose you've got akmod installed on these machines have you?

Comment 21 James Antill 2010-05-24 17:03:21 UTC
Or really, anything in:

/etc/kernel/postinst.d

?

Comment 22 Roland Pope 2010-05-24 21:03:04 UTC
I have an FC12 machine that has just started exhibiting this problem and I have not been able to figure how to get yum to install any pending updates.
I am able to supply any extra info if required.

Comment 23 Roland Pope 2010-05-24 23:08:03 UTC
Sorry, I should have said in my last comment, I have akmods installed (for my NVIDA graphics card) and I have one file in /etc/kernel/postinst.d and that is the akmods file.
There are no kernels in my pending updates however if that's relevent.


<snip>
 yum list *yum*
Loaded plugins: presto, refresh-packagekit
Installed Packages
PackageKit-yum.x86_64                                                              0.5.7-2.fc12                                           @updates                                      
PackageKit-yum-plugin.x86_64                                                       0.5.7-2.fc12                                           @updates                                      
anaconda-yum-plugins.noarch                                                        1:1.0-5.fc12                                           @anaconda-InstallationRepo-200911081904.x86_64
yum.noarch                                                                         3.2.27-3.fc12                                          @updates                                      
yum-metadata-parser.x86_64                                                         1.1.2-14.fc12                                          @anaconda-InstallationRepo-200911081904.x86_64
yum-presto.noarch                                                                  0.6.2-1.fc12                                           @updates                                      
yum-utils.noarch                                                                   1.1.26-1.fc12                                          @updates                                      
Available Packages
yum-NetworkManager-dispatcher.noarch                                               1.1.26-1.fc12                                          updates                                       
yum-arch.noarch                                                                    2.2.2-8.fc12                                           fedora                                        
yum-cron.noarch                                                                    0.9.2-1.fc12                                           updates                                       
yum-plugin-aliases.noarch                                                          1.1.26-1.fc12                                          updates                                       
yum-plugin-auto-update-debug-info.noarch                                           1.1.26-1.fc12                                          updates                                       
yum-plugin-basearchonly.noarch                                                     1.1.23-3.fc12                                          fedora                                        
yum-plugin-changelog.noarch                                                        1.1.26-1.fc12                                          updates                                       
yum-plugin-downloadonly.noarch                                                     1.1.26-1.fc12                                          updates                                       
yum-plugin-fastestmirror.noarch                                                    1.1.26-1.fc12                                          updates                                       
yum-plugin-filter-data.noarch                                                      1.1.26-1.fc12                                          updates                                       
yum-plugin-fs-snapshot.noarch                                                      1.1.26-1.fc12                                          updates                                       
yum-plugin-keys.noarch                                                             1.1.26-1.fc12                                          updates                                       
yum-plugin-list-data.noarch                                                        1.1.26-1.fc12                                          updates                                       
yum-plugin-local.noarch                                                            1.1.26-1.fc12                                          updates                                       
yum-plugin-merge-conf.noarch                                                       1.1.26-1.fc12                                          updates                                       
yum-plugin-post-transaction-actions.noarch                                         1.1.26-1.fc12                                          updates                                       
yum-plugin-priorities.noarch                                                       1.1.26-1.fc12                                          updates                                       
yum-plugin-protect-packages.noarch                                                 1.1.26-1.fc12                                          updates                                       
yum-plugin-protectbase.noarch                                                      1.1.26-1.fc12                                          updates                                       
yum-plugin-refresh-updatesd.noarch                                                 1.1.26-1.fc12                                          updates                                       
yum-plugin-remove-with-leaves.noarch                                               1.1.26-1.fc12                                          updates                                       
yum-plugin-rpm-warm-cache.noarch                                                   1.1.26-1.fc12                                          updates                                       
yum-plugin-security.noarch                                                         1.1.26-1.fc12                                          updates                                       
yum-plugin-show-leaves.noarch                                                      1.1.26-1.fc12                                          updates                                       
yum-plugin-tmprepo.noarch                                                          1.1.26-1.fc12                                          updates                                       
yum-plugin-tsflags.noarch                                                          1.1.26-1.fc12                                          updates                                       
yum-plugin-upgrade-helper.noarch                                                   1.1.26-1.fc12                                          updates                                       
yum-plugin-verify.noarch                                                           1.1.26-1.fc12                                          updates                                       
yum-plugin-versionlock.noarch                                                      1.1.26-1.fc12                                          updates                                       
yum-rhn-plugin.noarch                                                              0.7.6-1.fc12                                           fedora                                        
yum-updateonboot.noarch                                                            1.1.26-1.fc12                                          updates                                       
yum-updatesd.noarch                                                                1:0.9-3.fc12                                           fedora                                        
yumex.noarch                                                                       2.9.7-1.fc12                                           updates


rpm -qa |grep yum
anaconda-yum-plugins-1.0-5.fc12.noarch
yum-presto-0.6.2-1.fc12.noarch
yum-metadata-parser-1.1.2-14.fc12.x86_64
yum-3.2.27-3.fc12.noarch
PackageKit-yum-0.5.7-2.fc12.x86_64
yum-utils-1.1.26-1.fc12.noarch
PackageKit-yum-plugin-0.5.7-2.fc12.x86_64
</snip>

Comment 24 James Antill 2010-05-24 23:31:59 UTC
"yum clean rpmdb" will fix it.

Comment 25 Roland Pope 2010-05-25 00:03:03 UTC
I can confirm that a 'yum clean rpmdb' has enabled me to successfully to do a subsequent 'yum update.

Comment 26 Roderick Johnstone 2010-05-25 08:42:53 UTC
I've just had the same:
    pkg = self.searchNevra(n, e, v, r, a)[0]
IndexError: list index out of range

error, but on just one of my systems. Also have akmods for the nvidia driver.

Is there anything I can do to help diagnose this before I run the "yum clean rpmdb" ?

Comment 27 James Antill 2010-05-25 13:11:19 UTC
 It's fine Roderick we have a pretty good understanding of what the problem is with akmods:

1. akmods is installed, and has a kernel postinst scritplet.

2. kernel is updated within a yum transaction.

3. akmods spawns a build+install for the nvidia/etc. kernel module.

4. yum completes the transaction (thus. freeing the rpmdb lock).

5. yum drops all it's caches.

6. yum loads all the rpmdb info. again to write out the new yumdb stuff, and 
to create a new "rpmdb version" cache breaker.

7. akmods does the install part of #3, thus. changing the rpmdb behind yum.

8. yum writes out the new yumdb info. and generates the new "rpmdb version".

9. yum writes out the new "rpmdb version" cache breaker (and it thus. has a
newer mtime).

...at this point we have a valid cache of the rpmdb at time #6, but it's bad.
 I did two defensive upstream fixes last week, and both should help:

1. If the mtime of rpmdb changes between the start of #6 and the  end of #9, we
don't write the file.

2. We don't traceback if the packages in the cache aren't there.

...we might do something else upstream so that akmods can't run at all until
after #9, or something ... but both these should help, and are pretty harmless.  

...so unless someone is hitting this without akmods, just run "yum clean rpmdb" to fix it.

Comment 28 Roderick Johnstone 2010-05-25 13:39:13 UTC
Thanks for the update. Will use the  "yum clean rpmdb" until your upstream changes come on line.

Comment 29 James Antill 2010-06-01 15:39:12 UTC
Going to mark this as a dup. assuming that it is akmods.

*** This bug has been marked as a duplicate of bug 595063 ***

Comment 30 Fedora Update System 2010-08-02 14:12:17 UTC
yum-3.2.28-1.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/yum-3.2.28-1.fc13

Comment 31 Fedora Update System 2010-08-02 14:13:48 UTC
yum-3.2.28-1.fc14 has been submitted as an update for Fedora 14.
http://admin.fedoraproject.org/updates/yum-3.2.28-1.fc14

Comment 32 Fedora Update System 2010-08-02 14:15:18 UTC
yum-3.2.28-1.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/yum-3.2.28-1.fc12

Comment 33 Fedora Update System 2010-08-05 23:45:27 UTC
yum-3.2.28-1.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 34 Fedora Update System 2010-08-12 20:16:21 UTC
yum-3.2.28-3.fc12 has been submitted as an update for Fedora 12.
http://admin.fedoraproject.org/updates/yum-3.2.28-3.fc12

Comment 35 Fedora Update System 2010-08-12 20:18:45 UTC
yum-3.2.28-3.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/yum-3.2.28-3.fc13

Comment 36 Fedora Update System 2010-08-12 20:21:10 UTC
yum-3.2.28-3.fc14 has been submitted as an update for Fedora 14.
http://admin.fedoraproject.org/updates/yum-3.2.28-3.fc14

Comment 37 Fedora Update System 2010-08-17 05:46:31 UTC
yum-3.2.28-3.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 38 Fedora Update System 2010-08-20 02:20:06 UTC
yum-3.2.28-3.fc12 has been pushed to the Fedora 12 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 39 Fedora Update System 2010-08-24 01:37:46 UTC
yum-3.2.28-3.fc14 has been pushed to the Fedora 14 stable repository.  If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.