Bug 679783 - [abrt] abrt-1.1.17-2.fc15.1: metalink.py:184:__init__:MetaLinkRepoErrorParseFail: File /var/cache/yum/i386/15/updates-debuginfo/metalink.xml does not exist
Summary: [abrt] abrt-1.1.17-2.fc15.1: metalink.py:184:__init__:MetaLinkRepoErrorParseF...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: yum
Version: 15
Hardware: i686
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Fedora Packaging Toolset Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: abrt_hash:03969454 RejectedBlocker Re...
: 691468 699455 702696 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-02-23 14:05 UTC by John Watzke
Modified: 2014-01-21 23:17 UTC (History)
17 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-08-07 20:01:18 UTC


Attachments (Terms of Use)
File: backtrace (1.80 KB, text/plain)
2011-02-23 14:05 UTC, John Watzke
no flags Details
reproducer (282 bytes, text/x-python)
2011-03-25 13:45 UTC, Jiri Moskovcak
no flags Details
yum-test-2.py (383 bytes, text/plain)
2011-04-29 18:55 UTC, Steve Tyler
no flags Details

Description John Watzke 2011-02-23 14:05:08 UTC
abrt version: 1.1.17
architecture: i686
component: abrt
executable: /usr/bin/abrt-debuginfo-install
kernel: 2.6.38-0.rc5.git7.1.fc15.i686.PAE
package: abrt-1.1.17-2.fc15.1
reason: metalink.py:184:__init__:MetaLinkRepoErrorParseFail: File /var/cache/yum/i386/15/updates-debuginfo/metalink.xml does not exist
release: Fedora release 15 (Lovelock)
How to reproduce: 1. Run abrt-cli -bi @<bug_in_list>
time: 1298446930
uid: 0

backtrace
-----
metalink.py:184:__init__:MetaLinkRepoErrorParseFail: File /var/cache/yum/i386/15/updates-debuginfo/metalink.xml does not exist

Traceback (most recent call last):
  File "/usr/bin/abrt-debuginfo-install", line 454, in <module>
    result = downloader.download(missing)
  File "/usr/bin/abrt-debuginfo-install", line 189, in download
    self.repos.doSetup()
  File "/usr/lib/python2.7/site-packages/yum/repos.py", line 92, in doSetup
    self.ayum.plugins.run('postreposetup')
  File "/usr/lib/python2.7/site-packages/yum/plugins.py", line 184, in run
    func(conduitcls(self, self.base, conf, **kwargs))
  File "/usr/lib/yum-plugins/fastestmirror.py", line 197, in postreposetup_hook
    if downgrade_ftp and _len_non_ftp(repo.urls) == 1:
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 706, in <lambda>
    urls = property(fget=lambda self: self._geturls(),
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 703, in _geturls
    self._baseurlSetup()
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 649, in _baseurlSetup
    mirrorurls.extend(list(self.metalink_data.urls()))
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 746, in <lambda>
    metalink_data = property(fget=lambda self: self._getMetalink(),
  File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 742, in _getMetalink
    self._metalink = metalink.MetaLinkRepoMD(self.metalink_filename)
  File "/usr/lib/python2.7/site-packages/yum/metalink.py", line 184, in __init__
    raise MetaLinkRepoErrorParseFail, "File %s does not exist" %filename
MetaLinkRepoErrorParseFail: File /var/cache/yum/i386/15/updates-debuginfo/metalink.xml does not exist

Local variables in innermost frame:
self: <yum.metalink.MetaLinkRepoMD instance at 0x86dbeec>
filename: '/var/cache/yum/i386/15/updates-debuginfo/metalink.xml'

comment
-----
I was installing working with abrt-cli, which didn't appear to be installing debuginfos for me, so I 
manually installed debugs with debuginfo-install

Comment 1 John Watzke 2011-02-23 14:05:11 UTC
Created attachment 480459 [details]
File: backtrace

Comment 2 Jiri Moskovcak 2011-02-23 14:20:55 UTC
Seth, is this something we should catch in our code or is it a bug in yum?

Comment 3 seth vidal 2011-02-23 14:33:16 UTC
Is something else modifying the yum cache at the same time? B/c it really shouldn't be possible to get into the above situation unless SOMETHING is not honoring the yum lock.

Comment 4 John Watzke 2011-02-23 16:18:10 UTC
So, in the above situation, I was running with abrt-cli because I can't login to gnome-shell due to problem in mesa/mutter that causes crashes.  The should limit the things that could possibly be running at the same time, i.e. no yum-update-applet.  The only things I actually ran on this system after bootup would have been abrt-cli and debuginfo-install.  Both of these try to install debug infos.  Either one of them is not respecting the lock or perhaps one of them is removing the lock file but not actually releasing the yum resource?

Comment 5 Jiri Moskovcak 2011-03-24 21:53:00 UTC
I also manage to reproduce it when updated to F15, seems to work fine on F14. No other apps using yum were running when this happened (at least according to ps)

Comment 6 Jiri Moskovcak 2011-03-25 13:45:43 UTC
Created attachment 487565 [details]
reproducer

This is a simple reproducer, can someone who understands the YUM api please help us with it?

Comment 7 Jiri Moskovcak 2011-03-25 13:51:32 UTC
ABRT is useless on F15 without this fixed -> proposing to F15 beta blocker.

Comment 8 Jiri Moskovcak 2011-03-25 16:09:32 UTC
removing /var/cache/yum/i386/15/updates-debuginfo/ should help

Comment 9 Adam Williamson 2011-03-25 18:12:51 UTC
Discussed at 2011-03-25; due to inability of skvidal, geppetto and me to reproduce this, we think it's a transient cache issue and reject it as blocker or NTH for now. It can be reproposed if new info or a reliable reproducer show up.

Comment 10 Jiri Moskovcak 2011-03-25 20:47:12 UTC
I found the reproducer: in the latest abrt the script which installs the debuginfo is not run from the daemon and because it needs to write to /var/cache/abrt-di it needs SGID and the only way to run python script with SGID is to use a wrapper. So if I use the wrapper which is owned by abrt:abrt and has SGID I'm able to run our debuginfo install script only once and it fails when I try to run it again. The only workaround is to remove the /var/tmp/yum-.../ and then I can run the script again, but of course only once...

the yum cache is (obviously) created with "abrt" as a group:
[21:41:47 jmoskovc@dhcp-25-200 /]$ ls -l /var/tmp | grep yum
drwx------  3 jmoskovc abrt     4096 Mar 25 21:20 yum-jmoskovc-HzOhQ

Comment 11 John Watzke 2011-03-28 00:59:56 UTC
This is still happening in abrt 2.0.0 and the problem is a bit more severe as the new abrt doesn't seem to even generate backtraces when the script fails.  Removing /var/tmp/yum-* does fix this the first time around so it is somewhat of a workaround but this will severely impact crash reporting in F15 for those who don't know about this workaround.

Comment 12 Adam Williamson 2011-03-28 15:44:42 UTC
"but this will severely impact crash reporting in F15 for those who don't know about this workaround"

...and who are affected by the bug. I still haven't seen a reproducer and have not been able to reproduce this locally.

Comment 13 Jiri Moskovcak 2011-03-28 16:18:48 UTC
*** Bug 691468 has been marked as a duplicate of this bug. ***

Comment 14 DiEOrLivE 2011-04-14 03:04:23 UTC
Package: abrt-addon-ccpp-2.0.0-4.fc15
Architecture: x86_64
OS Release: Fedora release 15 (Lovelock)


Comment
-----
Report a bug

Comment 15 Jiri Moskovcak 2011-04-27 12:27:44 UTC
*** Bug 699455 has been marked as a duplicate of this bug. ***

Comment 16 Jiri Moskovcak 2011-04-27 12:28:52 UTC
Seems more like a yum problem -> reassigning.

Comment 17 Steve Tyler 2011-04-28 17:46:57 UTC
A workaround seems to be to disable the plugin:
/etc/yum/pluginconf.d/fastestmirror.conf

Tested with Jiri's reproducer (Comment 6):
$ sudo python yum-test-1.py

yum-3.2.29-4.fc15.noarch
yum-langpacks-0.2.2-1.fc15.noarch
yum-metadata-parser-1.1.4-4.fc15.x86_64
yum-plugin-auto-update-debug-info-1.1.30-2.fc15.noarch
yum-plugin-fastestmirror-1.1.30-2.fc15.noarch
yum-presto-0.6.2-3.fc15.noarch
yum-utils-1.1.30-2.fc15.noarch

Comment 18 Steve Tyler 2011-04-28 17:59:22 UTC
(In reply to comment #17)
> A workaround seems to be to disable the plugin:
> /etc/yum/pluginconf.d/fastestmirror.conf
> 
> Tested with Jiri's reproducer (Comment 6):
> $ sudo python yum-test-1.py

Except that no repo data is downloaded ...

As a normal user, repo data is downloaded even with the plugin enabled:
$ python yum-test-1.py

Comment 19 Steve Tyler 2011-04-28 19:57:00 UTC
I believe the problem is that F15 repos have a *prerelease* layout and the fastestmirror plugin is not taking that into account: 
/var/cache/yum/i386/15/updates-debuginfo/metalink.xml does not exist

F15 repo returns an error:
$ curl -i 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-debug-f15&arch=x86_64'
...
# repo = updates-released-debug-f15 arch = x86_64 error: invalid repo or arch
...

F14 repo returns metalink.xml:
$ curl -i 'https://mirrors.fedoraproject.org/metalink?repo=updates-released-debug-f14&arch=x86_64'
...
Content-Type: application/metalink+xml
...

Comment 20 Steve Tyler 2011-04-28 20:58:01 UTC
Methodology:
# strace -f -s 4096 python yum-test-1.py 2> x3.strace
# less x3.strace
Search for metalink.xml.

Comment 21 Steve Tyler 2011-04-29 01:14:20 UTC
Q: What is the difference between C programmers and Python programmers?
A: C programmers debug Python programs with strace.

Comment 22 Steve Tyler 2011-04-29 13:13:52 UTC
These are the only valid repos for F15, x86_64:

$ grep 15 metalink-3.xml | grep x86_64
# repo=fedora-15, arch=x86_64
# repo=fedora-debug-15, arch=x86_64
# repo=updates-released-f15, arch=x86_64
# repo=updates-testing-debug-f15, arch=x86_64
# repo=updates-testing-f15, arch=x86_64

Why would yum try to download from updates-released-debug-f15?

You can get the full list with:
$ curl -i 'https://mirrors.fedoraproject.org/metalink?repo=foo&arch=bar'

Comment 23 Steve Tyler 2011-04-29 18:55:40 UTC
Created attachment 495843 [details]
yum-test-2.py

(In reply to comment #22)
...
> Why would yum try to download from updates-released-debug-f15?

Because the test program iterates over all repos, regardless of whether they are enabled or not:
for repo in yumbase.repos.findRepos('*'):

The attached test program iterates only over each enabled repo and explicitly enables the corresponding "-debuginfo" repo. This eliminates the traceback in the fastestmirror plugin.

This is essentially a workaround, because enabling the "updates" repo in /etc/yum.repos.d/fedora-updates.repo will result in the fastestmirror traceback recurring.

The program should actually test that the enabled repo does not already match "-debuginfo", because the resulting name would then be "foo-repo-debuginfo-debuginfo", and that will give a trackback when it is enabled.

Comment 24 Steve Tyler 2011-04-29 19:54:22 UTC
Missing repos during development have been a problem before:

F14 updates-debuginfo repo is missing
Wed Sep 1 04:47:22 UTC 2010
http://lists.fedoraproject.org/pipermail/test/2010-September/093201.html

Comment 25 Jiri Moskovcak 2011-04-29 21:22:43 UTC
Actually it should be even more clever, it should enable only the -debuginfo repos for repos from which were installed the packages involved in the crash. ABRT should have all informations to determine this. But either way, yum shouldn't die on not-accessible repository so the fastestmirror plugin should be fixed.

Comment 26 Steve Tyler 2011-04-30 09:40:40 UTC
(In reply to comment #25)
> Actually it should be even more clever, it should enable only the -debuginfo
> repos for repos from which were installed the packages involved in the crash.
> ABRT should have all informations to determine this. But either way, yum
> shouldn't die on not-accessible repository so the fastestmirror plugin should
> be fixed.

Enabling specific repos based on the crash data is a good idea.

There is indeed a yum bug, reproducible by enabling all repos in
/etc/yum.repos.d/{fedora.repo, fedora-updates.repo, fedora-updates-testing.repo}.
This doesn't produce a traceback, but yum terminates with an error:

$ sudo yum repolist -q
Could not parse metalink https://mirrors.fedoraproject.org/metalink?repo=updates-released-debug-f15&arch=x86_64 error was 
No repomd file
Error: File /var/cache/yum/x86_64/15/updates-debuginfo/metalink.xml does not exist

This error occurs with certain prerelease repos:

$ curl 'http://mirrors.fedoraproject.org/metalink?repo=updates-released-debug-f15&arch=x86_64'
<?xml version="1.0" encoding="utf-8"?>
<metalink version="3.0" xmlns="http://www.metalinker.org/" type="dynamic" pubdate="Sat, 30 Apr 2011 09:33:39 GMT" generator="mirrormanager" xmlns:mm0="http://fedorahosted.org/mirrormanager">
<!--
# repo = updates-released-debug-f15 arch = x86_64 error: invalid repo or arch
# following repositories are available:
# repo=core-2, arch=i386
...
# repo=updates-testing-source-fc6, arch=source

-->
</metalink>

Comment 27 Steve Tyler 2011-04-30 09:58:17 UTC
(In reply to comment #26)
...
> There is indeed a yum bug, reproducible by enabling all repos in
...
Bug 700988 - Error: File /var/cache/yum/x86_64/15/updates-debuginfo/metalink.xml does not exist

Comment 28 Matt Domsch 2011-05-01 15:14:27 UTC
In the past we've asked releng to create empty repos for all the repos that will eventually exist, so mirrormanager can return valid results for the empty repos, rather than return an "empty" metalink file for the not-yet-created repos as it does today.

Comment 29 Steve Tyler 2011-05-01 15:22:07 UTC
(In reply to comment #28)
> In the past we've asked releng to create empty repos for all the repos that
> will eventually exist, so mirrormanager can return valid results for the empty
> repos, rather than return an "empty" metalink file for the not-yet-created
> repos as it does today.

They were going to make that "SOP": Bug 625922, Comment 6.

Comment 30 Matt Domsch 2011-05-01 19:20:42 UTC
right - apparently empty debuginfo repos weren't part of the SOP addition. :-(

Comment 31 Matt Domsch 2011-05-01 19:29:47 UTC
https://fedorahosted.org/rel-eng/ticket/4682
created to ask rel-eng to fix this.

Comment 32 Steve Tyler 2011-05-01 21:45:07 UTC
(In reply to comment #31)
> https://fedorahosted.org/rel-eng/ticket/4682
> created to ask rel-eng to fix this.

Thanks, Matt.
Blaming rel-eng is always best ... :-)

BTW, I looked at your code ...
Has anyone else ever reviewed it?

No shame
An old man
Polishes his honour
And dies
With a smile

Comment 33 Matt Domsch 2011-05-02 01:58:47 UTC
It's been quite a while since it's been reviewed, I'd welcome it.  This was one of my first biggest python projects, so I have no doubt there's room for significant improvement.

Comment 34 Jiri Moskovcak 2011-05-06 16:50:22 UTC
*** Bug 702696 has been marked as a duplicate of this bug. ***

Comment 35 Fedora Admin XMLRPC Client 2012-04-27 15:41:45 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 36 Fedora End Of Life 2012-08-07 20:01:22 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping


Note You need to log in before you can comment on or make changes to this bug.