Bug 1073459

Summary:	false error messages
Product:	[Fedora] Fedora	Reporter:	Ales Kozumplik <akozumpl>
Component:	librepo	Assignee:	Tomas Mlcoch <tmlcoch>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	20	CC:	jzeleny, tmlcoch
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2014-04-14 09:16:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ales Kozumplik 2014-03-06 13:26:16 UTC

For this one needs to create himself purposelly broken, remote repo, e.g. by removing the primary.xml. At the time of this writing the repo at http://akozumpl.fedorapeople.org/t/ can be used for this (i.e. no primary.xml).

Run the script below, erasing ./repodata between the runs. For me, this yields different error messages some times, including:

librepo.LibrepoException: (19, 'Yum repo downloading error: Downloading error(s): repodata/95942ac00a1b10366455e2eb8c1baed980ccea909c5256aca81afa89f7e340c0-prestodelta.xml.gz - Cannot download, all mirrors were already tried without success; repodata/88527bc0ed21d5a2acc2b9bd6242593af65102541551277bed147a7da5509f13-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/updateinfo.xml.gz - Cannot download, all mirrors were already tried without success', 'Usable URL not found')

but prestodelta *is* present on the server. It is OK if perform() ejects early on fatal error, but it should not report present files as impossible to download.


#! /usr/bin/python

import librepo

# URL = 'http://mirrors.fedoraproject.org/metalink?repo=updates-released-f19&arch=x86_64'
URL = 'http://akozumpl.fedorapeople.org/t/'

def progress(*args):
    print(args)
    raise RuntimeError('some')

def main():
    h = librepo.Handle()
    h.setopt(librepo.LRO_URLS, URL)
    h.repotype = librepo.LR_YUMREPO
    h.destdir = './'
    h.interruptible = True
    h.connecttimeout = 1
    h.perform()

if __name__ == '__main__':
    main()

Comment 1 Ales Kozumplik 2014-03-11 13:25:00 UTC

Hmm, I just found out the repo in comment 0 might be at fault: the files referenced from repomd.xml all have xml:base="blah" set (a testing artifact). So this could be the expected behavior but that makes me think whether it shouldn't be somehow reflected in the exception message?

Comment 2 Tomas Mlcoch 2014-03-11 15:56:11 UTC

Hi, thanks for the report and reproducer!

The "prestodelta" error is weird. I cannot reproduce it yet, but I'll look at it more closely.

Librepo currently ignores the "base" attribute. Librepo parses and stores it, but doesn't use it during download (will be fixed soon).

Comment 3 Ales Kozumplik 2014-03-12 08:46:34 UTC

Tomas I now made

http://akozumpl.fedorapeople.org/t/

a repository that contains all the files.

librepo.LibrepoException: (19, 'Yum repo downloading error: Downloading error(s): repodata/updateinfo.xml.gz - Cannot download, all mirrors were already tried without success', 'Usable URL not found')

I think it is really possible that the xml:base attributes are confusing librepo (despite comment 2). Let me know if I can help further.

Comment 4 Tomas Mlcoch 2014-03-17 16:04:15 UTC

The error is valid, the repodata/updateinfo.xml.gz has a bad checksum.

But the true is that the error message is too generic and unhelpful. And this issue need to be fixed.
But note that the message is generic intentionally - in the background of repodata downloading there can be a large amount of different failures - generate one big error message with all the failures is bad idea.

To solve the issue gracefully, I've added a LRO_HMFCB (Handle Mirror Failure Callback) option to the Handle() object. See: http://tojaj.github.io/librepo/lib.html#librepo.LRO_HMFCB

Via this option you can set a callback that will receive all failures during the metadata downloading with description what was bad. The callback signature: http://tojaj.github.io/librepo/lib.html#callback-handlemirrorfailurecb-label

Comment 5 Ales Kozumplik 2014-03-17 16:14:05 UTC

Thanks Tomas, this will help us track down some of the weirder errors.

Comment 6 Tomas Mlcoch 2014-03-18 12:52:42 UTC

Moreover, I 've just pushed an one-liner that adds support for xml:base tags in repomd.xml.
https://github.com/Tojaj/librepo/commit/d17354804bff668a13517c7739bf58113eafe790

Comment 7 Tomas Mlcoch 2014-04-14 09:16:13 UTC

The fixed librepo-1.7.1-1.fc20 is available in fedora-updates repo.