Bug 374921

Summary: metadata doesn't expire when doing upgrade
Product: [Fedora] Fedora Reporter: James Antill <james.antill>
Component: anacondaAssignee: Anaconda Maintenance Team <anaconda-maint-list>
Status: CLOSED CURRENTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 8CC: ffesti, james.antill, jonstanley, pmatilai, tla, wwoods
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: F9 GA Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-05-09 17:19:42 EDT Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 235706    
Attachments:
Description Flags
screenshot of PK wanting to install an f8 xterm update none

Description James Antill 2007-11-10 13:09:33 EST
Description of problem:
 If you move from F7 -> F8 "fast enough" and then use yum, yum hasn't expired
the metadata and so you get errors due to trying to update to F7 packages.

Version-Release number of selected component (if applicable):

yum(0:3.2.7-1.fc8)

...this might be worth just hacking a "yum clean all" fix into anaconda etc. for F9.
Comment 1 Seth Vidal 2007-11-12 10:10:12 EST
you mean if you move from f7->f8 by anaconda?

If so - why file it against yum?
Comment 2 Jeremy Katz 2007-11-12 10:20:31 EST
Yeah, probably worth doing.  Sticking on the F9 blocker list so we don't forget
about it.
Comment 3 James Antill 2007-11-12 10:25:30 EST
 Yeh, I did it via. anaconda but I thought that Live CD etc. had plans to do
upgrades eventually. But, yeh, I should probably have just filed against anaconda.
Comment 4 Jesse Keating 2008-03-31 14:04:56 EDT
Jeremy, are we going to do something like this post-feature freeze?
Comment 5 Jeremy Katz 2008-03-31 14:56:33 EDT
I thought we did it already -- is it still a problem?
Comment 6 Jesse Keating 2008-04-08 22:42:04 EDT
Probably just needs retesting.  Putting MODIFIED
Comment 7 Jon Stanley 2008-04-15 01:02:28 EDT
It is still a problem....~/md5sum.fedora was created prior to the upgrade.

[root@dhcp-137 fedora]# md5sum *
d41d8cd98f00b204e9800998ecf8427e  cachecookie
76f92af7e30d653a9fa043efa0d5bcd0  filelists.sqlite
5010e7caa981dfe76bdfcd75441599c6  mirrorlist.txt
md5sum: packages: Is a directory
e6b89f4c535a7a6bc1fd38899e112104  primary.sqlite
b97031bc1c23129c2ff771d9094f39cc  repomd.xml
[root@dhcp-137 fedora]# cat ~/md5sum.fedora 
d41d8cd98f00b204e9800998ecf8427e  cachecookie
76f92af7e30d653a9fa043efa0d5bcd0  filelists.sqlite
5010e7caa981dfe76bdfcd75441599c6  mirrorlist.txt
e6b89f4c535a7a6bc1fd38899e112104  primary.sqlite
b97031bc1c23129c2ff771d9094f39cc  repomd.xml
[root@dhcp-137 fedora]# 

Since this has really been this way forever, it is worth it being a blocker or not?
Comment 8 Jeremy Katz 2008-04-15 12:35:05 EDT
Fixed
Comment 9 Will Woods 2008-04-17 13:19:12 EDT
So how do you test this? something like: 

- Install F8, 
- let yum-updatesd fetch metadata (but do not install), 
- immediately upgrade to F9 from a fast mirror.

If the bug is present, you'll get errors from a 'yum upgrade' in F9 trying to
pull down the F8 packages. Right?
Comment 10 Jon Stanley 2008-04-17 15:25:27 EDT
Yeah, that is what I did - you can even install updates if you like (I think
that I installed all of them).  And I did the much lower tech solution of md5sum
in /var/cache/yum before and after.
Comment 11 Will Woods 2008-04-18 17:47:34 EDT
Tested a couple of different times - no complaints from yum. James, do you want
to retest and confirm the fix or should we just consider it closed?
Comment 12 Jon Stanley 2008-04-18 22:17:01 EDT
Interesting.  This really wasn't fixed til today:

commit b3d1e45042843693b2181714296756ce5c2ba7ae
Author: Jeremy Katz <katzj@redhat.com>
Date:   Fri Apr 18 16:06:16 2008 -0400

    Listing the directories before expiring yum caches helps

Again, this wasn't looking at complaints from yum - this was a simple "is the
metadata still there and usable by yum".  BTW, if you go the md5sum route
tomorrow, note that the data will still be there but the cachecookie will not be.
Comment 13 James Antill 2008-04-18 22:38:29 EDT
 As long as someone has tested it I'm fine with closing it (I believe y'all :).
 Note though that the best test is just to make sure all the MD files are gone
from /var/cache/yum/* (or at least all the repomd.xml files, but you might as
well not leave unused MD files due to the unique names thing) if just the
cachecookie is gone then yum can keep the cache.

 Anyway I'll stop rambling now ... Im' probably just spamming Jeremey's inbox :).
Comment 14 Will Woods 2008-04-18 23:44:07 EDT
I must have just been testing it wrong. Can someone who has actually seen this bug in action confirm the 
fix?
Comment 15 Jon Stanley 2008-04-19 00:44:19 EDT
Hmm, the code as it exists now removes cachecookie, and I just did a spin with
the new anaconda and verified that indeed the cachecookie files are gone now. 
Are you saying that's not sufficient, James?

BTW - the version wasn't bumped to 11.4.0.75 inside anaconda, still says 74.
Comment 16 James Antill 2008-04-19 00:56:15 EDT
 Yeh, in comment #0 I suggest just doing a "yum clean all". Not sure where the
cachecookie fix came from ... maybe because this is a suggested low BW way to
make sure you have the latest MD?

 Just removing cachecookie will _mostly_ work however if yum fails to download
new MD (fails to connect to network, mirror broken, etc.), yum will be clever
and revert to the old MD. In pretty much all cases this is better, a big
exception being if you've updated fedora-release and the old MD is from the
previous release.
Comment 17 James Antill 2008-04-19 01:09:02 EDT
 Oh, the other thing is that if you just remove the cachecookies then running
yum as a non-root user (or as root with -C) will still use the old data ... so
"yum list updates" as non-root will look like there's a major problem.
Comment 18 Jeremy Katz 2008-04-22 15:12:42 EDT
Changed to 'yum clean all'.  I was trying to go for keeping things simpler 
Comment 19 Will Woods 2008-04-28 18:59:58 EDT
Fix should be testable in tomorrow's rawhide (20080429). 
Comment 20 Will Woods 2008-05-01 16:53:24 EDT
Created attachment 304347 [details]
screenshot of PK wanting to install an f8 xterm update

Still seems to be happening with today's rawhide (20080501). Note that the
updates-testing repo was enabled before I upgraded.
Comment 21 Jeremy Katz 2008-05-01 18:58:48 EDT
updates-testing could well be in an odd state right now.  What's in
/var/log/anaconda.log?  Also (although it's probably too late now) what was left
in /var/cache/yum?
Comment 22 Jon Stanley 2008-05-01 21:39:46 EDT
Looking at anaconda in git, we still have:

        # expire yum caches on upgrade
        if anaconda.id.getUpgrade() and os.path.exists("%s/var/cache/yum"
%(anaconda.rootPath,)):
            log.info("Expiring yum caches")
            for d in os.listdir("%s/var/cache/yum" %(anaconda.rootPath,)):
                try:
                    os.unlink("%s/var/cache/yum/%s/cachecookie"
%(anaconda.rootPath, d))
                except:
                    pass

I thought you changed that awhile back?  You did it for preupgrade, just not the
normal upgrade case.
Comment 23 Jeremy Katz 2008-05-01 21:53:38 EDT
*sigh*  And that of course is the one that I tried out.  Fixed.  Again.
Comment 24 Will Woods 2008-05-02 17:55:15 EDT
I'm retesting with preupgrade, without updates-testing enabled, and still
hitting this. I think.

Retesting again with both upgrade paths, and also to be double-sure that I'm
using the right anaconda.
Comment 25 Will Woods 2008-05-02 19:23:59 EDT
Yes, still broken. Both normal upgrades and preupgrades have mirrorlist.txt for
'fedora' and 'updates', and they're full of F8 urls.

This is anaconda-11.4.0.81.

if anaconda.id.getUpgrade() and os.path.exists("%s/var/cache/yum"
%(anaconda.rootPath,)):
    log.info("Expiring yum caches")
    try:
        iutil.execWithRedirect("yum", ["clean", "all"],
                               stdout="/dev/tty5", stderr="/dev/tty5",
                               searchPath = 1)

Maybe you need to chroot there to make 'yum clean all' DTRT? Because running
'yum clean all' after the reboot definitely does the right thing.
Comment 26 Jeremy Katz 2008-05-04 13:53:41 EDT
Clearly I should not be allowed to touch this bug anymore, but yeah, a root=
would help.  Committed.  If you really want it for f9 Will, give me a holler. 
It is pushed to the branch, so if we do another build for something else, we'll
pick it up.  But I don't see making us go through testing again for it.
Comment 27 Will Woods 2008-05-09 17:19:42 EDT
These changes got pulled into anaconda-11.4.0.82. Tested a couple of upgrades
and it seems to be doing the right thing.