Bug 717483

Summary: yum cannot resume interrupted xml download for groups, in Fed-15 updates
Product: [Fedora] Fedora Reporter: Mohammed Arafa <bugzilla>
Component: createrepoAssignee: Luke Macken <lmacken>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 15CC: a.badger, dcantrell, dennis, ffesti, james.antill, lmacken, maxamillion, notting, pfrields, pmatilai, robatino, tla, zpavlas
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-06 20:03:06 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Mohammed Arafa 2011-06-28 22:37:03 UTC
Description of problem:
on a slow connection, i got a "connection reset by peer" error several time. each time yum would download the xml file from scratch.

Version-Release number of selected component (if applicable):


How reproducible:
everytime


Steps to Reproduce:
1.get on a slow net connection
2.do yum update
3. watch
  
Actual results:
yum downloads the xml file again and again as it gets interrupted at begins the download from scracth

Expected results:
resume the xml file download from where it stopped


Additional info:

Comment 1 Mohammed Arafa 2011-06-28 22:53:37 UTC
00 - OK : http://mirrors.med.harvard.edu/fedora/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml
Trying other mirror.
http://mirror.its.uidaho.edu/pub/fedora/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml: [Errno 14] HTTP Error 200 - OK : http://mirror.its.uidaho.edu/pub/fedora/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml
Trying other mirror.
http://mirror.hmc.edu/fedora/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml: [Errno 14] HTTP Error 200 - OK : http://mirror.hmc.edu/fedora/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml
Trying other mirror.
http://mirrors.usu.edu/fedora/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml: [Errno 14] HTTP Error 200 - OK : http://mirrors.usu.edu/fedora/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml
Trying other mirror.
updates/group                                                        100% [===========================================================] 3.3 kB/s | 110 kB     --:-- ETA 
updates/group                                                        100% [===========================================================] 3.0 kB/s | 122 kB     --:-- ETA 
updates/group                                                        100% [===========================================================]  505 B/s | 131 kB     --:-- ETA 
updates/group                                                        100% [===========================================================] 1.3 kB/s | 139 kB     --:-- ETA 
updates/group                                                        100% [===========================================================] 4.2 kB/s | 156 kB     --:-- ETA 
updates/group                                                        100% [===========================================================] 3.3 kB/s | 908 kB     --:-- ETA 
updates/group                                                        100% [===========================================================] 3.9 kB/s | 1.0 MB     --:-- ETA 
http://fedora.osuosl.org/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml: [Errno 14] HTTP Error 200 - OK : http://fedora.osuosl.org/linux/updates/15/x86_64/repodata/419d0c4c01a35c01e3ba97af91c3fd63bca1d65a1db6694be8a79b11787a2703-comps-f15.xml
Trying other mirror.
^Cupdates/group                                                        100% [===========================================================] 8.3 kB/s | 1.2 MB     --:-- ETA

Comment 2 James Antill 2011-06-29 14:28:13 UTC
 Yum uses this logic:

            if thisdata.size is None:
                reget = None
            else:
                reget = 'simple'
                if os.path.exists(local):
                    if os.stat(local).st_size >= int(thisdata.size):
                        misc.unlink_f(local)

...alas. the "groups" and "groups_gz" files don't have size metadata in the "updates" repo., so we are forced to turn download resuming off.

 Interestingly those files do have size data for fedora, so maybe someone can fix this? Sending to distribution, but I guess it's probably bodhi or something else.

Comment 3 Luke Macken 2012-01-04 23:41:18 UTC
Bodhi simply does a git pull of comps and hands it off to mash, which then calls createrepo. This feels like an issue lower in the stack.

Comment 4 Luke Macken 2012-01-04 23:51:18 UTC
Mash just seems to pass on the comps on via createrepo.MetaDataConfig().groupfile

Reassigning to createrepo...

Comment 5 Bill Nottingham 2012-01-05 15:59:40 UTC
mash uses the createrepo API. Is there something we should be setting here?

Comment 6 seth vidal 2012-01-05 18:19:55 UTC
Who is setting up the repository to download this info from? It's not createrepo, unless I've misunderstood the chain here.

Comment 7 Bill Nottingham 2012-01-05 18:30:52 UTC
mash is creating it, via the createrepo API.

See http://git.fedorahosted.org/git/?p=mash;a=blob_plain;f=mash/metadata.py;hb=HEAD, class MetadataNew.

To restate comment #2, the 'fedora' repo (created by pungi) has 'size' set for groups & groups_gz in repomd.xml, but the 'updates' & 'updates-testing' repos (created by mash) don't have 'size' set there.

Unless this is something where createrepo from EL6 (used for creating update repos) doesn't set this, and createrepo from Fedora (used for composing release trees) does?

Comment 8 Fedora End Of Life 2012-08-06 20:03:06 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 15. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained.  At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this 
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen 
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we were unable to fix it before Fedora 15 reached end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" (top right of this page) and open it against that 
version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 9 Fedora End Of Life 2012-08-06 20:03:06 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 15. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained.  At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this 
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen 
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we were unable to fix it before Fedora 15 reached end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" (top right of this page) and open it against that 
version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping