Bug 1019103

Summary: librepo ignores mm0:alternates
Product: [Fedora] Fedora Reporter: Ales Kozumplik <akozumpl>
Component: librepoAssignee: Tomas Mlcoch <tmlcoch>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 20CC: jzeleny, lnie, puiterwijk, tmlcoch
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: librepo-1.4.0-1.fc20 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-11-24 23:45:44 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
core hanging in select()
none
Traceback none

Description Ales Kozumplik 2013-10-15 07:38:53 UTC 
Created attachment 812376 [details]
core hanging in select()

I probably have a problem with network this morning and can not connect to the fedora updates mirror. I found out that librepo will never timeout in this particular case:

#! /usr/bin/python

import librepo

URL = 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-f19&arch=x86_64'

def main():
    h = librepo.Handle()
    h.setopt(librepo.LRO_MIRRORLIST, URL)
    h.repotype = librepo.LR_YUMREPO
    h.destdir = './'
    r = librepo.Result()
    h.interruptible = True
    h.connecttimeout = 1
    h.perform(r)

if __name__ == '__main__':
    main()

/////


Outputs when run and ^C-ed after cca 15s:

$ time ./main.py 
^CTraceback (most recent call last):
  File "./main.py", line 18, in <module>
    main()
  File "./main.py", line 15, in main
    h.perform(r)
  File "/home/akozumpl/repos/librepo/build/librepo/python/python2/librepo/__init__.py", line 983, in perform
    _librepo.Handle.perform(self, result)
KeyboardInterrupt

real	0m14.947s
user	0m0.137s
sys	0m0.175s

Also attaching core dump after sending another process the ABRT signal (also after some 10s).
Comment 1 Tomas Mlcoch 2013-10-15 09:01:56 UTC 
Hi,
I was able to reproduce this, but it's not a bug (or at least not on the librepo side), librepo doesn't hang, it is just trying other mirrors. For the metalink you've used it means 184 mirrors, and it takes some time.

It is all because the metalink and the mirrors are out of sync.

This is the current metalink: https://paste.fedoraproject.org/46787/
This is the current repomd.xml from the first mirror: https://paste.fedoraproject.org/46788/ (ftp://ftp.mirrorservice.org/sites/dl.fedoraproject.org/pub/fedora/linux/updates/19/x86_64/repodata/repomd.xml)

sha256sum of the repomd.xml is 90afabba594d0e1c9fa9656e6c174e8adebf099a73d39e82c9c54682f65ab6ac

But the expected one (loaded from metalink) is d4f9ad66f7c6e000d8ebf9ec92ad2c4636547853708554d93dab672bdfd98ca1

I suggest use of h.maxmirrortries option.
Comment 2 Tomas Mlcoch 2013-10-15 10:05:02 UTC 
Or you can turn off checksum check by the

h.checksum = False

From dnf point of view, librebo should provide a better mechanism to inform about what is wrong after each mirror failure during download of metadata.
Something like LrMirrorFailureCb which is used during download of packages.
I will talk with zpavlas about this.
Comment 3 Ales Kozumplik 2013-10-15 11:32:59 UTC 
(In reply to Tomas Mlcoch from comment #2)
> From dnf point of view, librebo should provide a better mechanism to inform
> about what is wrong after each mirror failure during download of metadata.
> Something like LrMirrorFailureCb which is used during download of packages.
> I will talk with zpavlas about this.

Thanks!
Comment 4 Ales Kozumplik 2013-10-15 11:35:17 UTC 
(In reply to Tomas Mlcoch from comment #1)
> Hi,
> I was able to reproduce this, but it's not a bug (or at least not on the
> librepo side), librepo doesn't hang, it is just trying other mirrors. For
> the metalink you've used it means 184 mirrors, and it takes some time.

Thanks for clarifying, what would be the recommended value to the h.maxmirrortries option? I want a safe value which however won't let the user wait 184*timeout seconds (if I count it right) for a failure.

Also, we should probably report this to the fedora infrastructure because it is clearly not dnf/librepo's fault.
Comment 5 Patrick Uiterwijk 2013-10-15 11:39:57 UTC 
If the metalink info and mirror content's checksum don't match, that's an error in the Fedora Infra.

I will look into it.
Comment 6 Patrick Uiterwijk 2013-10-15 11:48:27 UTC 
When I run sha256sum over both the link you give (ftp://ftp.mirrorservice.org/sites/dl.fedoraproject.org/pub/fedora/linux/updates/19/x86_64/repodata/repomd.xml) or another random mirror (ftp://mirror.1000mbps.com/fedora/linux/updates/19/x86_64/repodata/repomd.xml) I get the value:
099ff51a491eba3f16646efb5d16319249e0bb10f1dc1476d47e23b25e0e0f94

Which IS a valid checksum according to the metalink (check <mm0:alternatives>).

Could it be that you have a caching proxy that returns a stale repomd, or that the mirror you used is stale?
Comment 7 Patrick Uiterwijk 2013-10-15 12:31:28 UTC 
Okay, the summary is that librepo ignores mm0:alternates.

mm0:alternates was introduced because there's some delay between a new repomd file and the mirrors updating: we add mm:alternatives for some previous repomd revisions, so that one or two previous revisions can also be verified as correct until that mirror is synced.
Comment 8 Tomas Mlcoch 2013-10-16 11:40:00 UTC 
Patric,
Thank you for clarifying!

Fixed in head: https://github.com/Tojaj/librepo/commit/04ac52b072f3a23cf7b3177c96a23e6665e81f25
Comment 9 Fedora Update System 2013-11-19 08:52:12 UTC 
librepo-1.4.0-1.fc20 has been submitted as an update for Fedora 20.
https://admin.fedoraproject.org/updates/librepo-1.4.0-1.fc20
Comment 10 Fedora Update System 2013-11-19 21:50:33 UTC 
Package librepo-1.4.0-1.fc20:
* should fix your issue,
* was pushed to the Fedora 20 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing librepo-1.4.0-1.fc20'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2013-21715/librepo-1.4.0-1.fc20
then log in and leave karma (feedback).
Comment 11 lnie 2013-11-20 08:21:14 UTC 
Tested with librepo-1.4.0-1.fc20 （x86_64),seems fine,the output has been attached.
Comment 12 lnie 2013-11-20 08:22:21 UTC 
Created attachment 826466 [details]
Traceback
Comment 13 Tomas Mlcoch 2013-11-20 08:36:59 UTC 
Hi Inie,
as I could see in the traceback, it seems there was a connection problem
Are you sure, the link was working at the time when it was tested?
Comment 14 lnie 2013-11-20 08:43:58 UTC 
Hi Tomas,

 I turned off the network intentionally.From #comment0 :-),I think it's nessary to reproduce it
Comment 15 lnie 2013-11-20 08:46:42 UTC 
>I probably have a problem with network this morning and can not connect to the >fedora updates mirror. I found out that librepo will never timeout in this >particular case:
Comment 16 Fedora Update System 2013-11-24 23:45:44 UTC 
librepo-1.4.0-1.fc20 has been pushed to the Fedora 20 stable repository.  If problems still persist, please make note of it in this bug report.