Bug 1047709 - yum-config-manager corrupts repo file if repo url does not have .repo suffix
Summary: yum-config-manager corrupts repo file if repo url does not have .repo suffix
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: yum
Version: 20
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Packaging Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-01-01 22:12 UTC by Miro Hrončok
Modified: 2014-01-07 12:00 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-01-06 16:59:59 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Miro Hrončok 2014-01-01 22:12:22 UTC
Description of problem:
While the links on a Copr's Overview page looks like they lead to user-repo.repo files, they actually lead to an URL ending with slash.

This leads to several inconveniences:

1. If you use wget to get the .repo file, it is named index.html, unless you specify the output.

2. If you use yum-config-manager, the repo file is named very badly (and metadata are corrupted):

# yum-config-manager --add-repo=http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
Loaded plugins: auto-update-debuginfo, etckeeper, langpacks, refresh-packagekit
adding repo from: http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/

[copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_]
name=added from: http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
baseurl=http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
enabled=1


# cat /etc/yum.repos.d/copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_.repo 

[copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_]
name=added from: http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
baseurl=http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
enabled=1


While this is how it behaves with repo that has it's .repo file in URL:

# yum-config-manager --add-repo=http://repos.fedorapeople.org/repos/spot/chromium-stable/fedora-chromium-stable.repo
Loaded plugins: auto-update-debuginfo, etckeeper, langpacks, refresh-packagekit
adding repo from: http://repos.fedorapeople.org/repos/spot/chromium-stable/fedora-chromium-stable.repo
grabbing file http://repos.fedorapeople.org/repos/spot/chromium-stable/fedora-chromium-stable.repo to /etc/yum.repos.d/fedora-chromium-stable.repo
repo saved to /etc/yum.repos.d/fedora-chromium-stable.repo
# cat /etc/yum.repos.d/fedora-chromium-stable.repo
# Place this file in your /etc/yum.repos.d/ directory

[fedora-chromium-stable]
name=Builds of the "stable" tag of the Chromium Web Browser
baseurl=http://repos.fedorapeople.org/repos/spot/chromium-stable/fedora-$releasever/$basearch/
enabled=1
skip_if_unavailable=1
gpgcheck=0

[fedora-chromium-stable-source]
name=Builds of the "stable" tag of the Chromium Web Browser - Source
baseurl=http://repos.fedorapeople.org/repos/spot/chromium-stable/fedora-$releasever/SRPMS
enabled=0
skip_if_unavailable=1
gpgcheck=0

Comment 1 Miroslav Suchý 2014-01-04 09:22:03 UTC
ad 1. This is because wget by default ignore content-disposal header. Run wget with:
wget --content-disposition
or put in .wgetrc:
content_disposition = on/off

ad 2. I would dare to say that this is primary bug of yum-config-manager. Reassigning.

Comment 2 Zdeněk Pavlas 2014-01-06 09:11:08 UTC
> If you use yum-config-manager, the repo file is named very badly

I agree that [copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_] is not ideal, but can you come up with a better naming scheme?  Note that the generated name should contain the whole URL, not just the basename, which may be too generic.

Could you provide the expected bahavior?

> (and metadata are corrupted)

Really?

Comment 3 Miroslav Suchý 2014-01-06 09:43:23 UTC
Provided repo file already contain name. Can you use it?


>> (and metadata are corrupted)
>Really?

If you will try the reproducer in #0 you will get:

# yum-config-manager --add-repo=http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
# wget http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/

# diff -u /etc/yum.repos.d/copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_.repo churchyard-chromium-russianfedora.repo 
--- /etc/yum.repos.d/copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_.repo   2014-01-06 10:31:17.228191040 +0100
+++ churchyard-chromium-russianfedora.repo      2014-01-06 10:33:41.353943387 +0100
@@ -1,6 +1,9 @@
-
-[copr.fedoraproject.org_coprs_churchyard_chromium-russianfedora_repo_fedora-20-i386_]
-name=added from: http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
-baseurl=http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/
-enabled=1
-
+[churchyard-chromium-russianfedora]
+name=Copr repo for chromium-russianfedora owned by churchyard
+description=Chromium package from RFRemix.
+
+If you notice that version form [RFRemix repository](http://mirror.yandex.ru/fedora/russianfedora/russianfedora/free/fedora/development/rawhide/source/SRPMS/) is newer than this one, feel free to ping [Miro](https://fedoraproject.org/wiki/User:Churchyard).
+baseurl=http://copr-be.cloud.fedoraproject.org/results/churchyard/chromium-russianfedora/fedora-$releasever-$basearch/
+skip_if_unavailable=True
+gpgcheck=0
+enabled=1
\ No newline at end of file

I.e there is:
* missing skip_if_unavailable=True
* missing gpgcheck=0
* missing description (although description in original file is little borked, but that should not be problem, it should include at least first line).
* baseurl is url which was provided as option to --add-repo instead of url which was in that repo file.

So it seams to me that yum-config-manage is assuming that url provided to --add-repo is baseurl and not repo file.

Man page of yum-config-manager is not 100% clear what should be that url, but this documentation:
https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sec-Managing_Yum_Repositories.html
state it very clearly.

Comment 4 Zdeněk Pavlas 2014-01-06 10:00:19 UTC
> Provided repo file already contain name. Can you use it?

The use case you described did not provide a repo file, but a baseurl:

# yum-config-manager --add-repo=http://copr.fedoraproject.org/coprs/churchyard/chromium-russianfedora/repo/fedora-20-i386/

> So it seams to me that yum-config-manage is assuming that url provided to --add-repo is baseurl and not repo file.

Yes, if it ends with ".repo", it's assumed to be a repo file, otherwise it's assumed to be a baseurl, and the local .repo file is made up.  Yes, this is not documented.  I don't consider this a bug.

Comment 5 Miro Hrončok 2014-01-06 10:07:51 UTC
I guess that if you won't ignore content-disposal you would realize, this is actually repo file.

While I think it would be much easier, if Copr provider repo files with .repo in URL, content-disposal seems like a valid solution that yum-config-manager is silently ignoring.

If you don't consider this a bug, please consider this a feature request:

     Follow content-disposal when naming the file and realizing, if it's a repo file or baseurl.

Thanks

Comment 6 Zdeněk Pavlas 2014-01-06 16:59:59 UTC
> content-disposal seems like a valid solution that yum-config-manager is silently ignoring.

yum-config-manager ignores content-disposal header of the baseurl since it does not download from it at all. Baseurls are used only with some relativeurls.

And if it did, urlgrabber appends "index.html" to URLs ending with "/", so you don't get the correct header either.  And I'm not quite OK with breaking existing functionality to support a feature that's mostly ignored anyway (wget).

Also note that the parallel downloader API returns the target filename BEFORE starting the request, so we can't use server-side headers to alter this.. And there is no API to retrieve headers ex post.  It could be added, but I don't see the point.

Comment 7 Miroslav Suchý 2014-01-06 21:06:22 UTC
> yum-config-manager ignores content-disposal header of the baseurl
But it is not baseurl. It is url of repo file. And linked documentation clearly state, that it should be url of repo file.

So it is missing functionality in urlgrabber? Or more precise bug in urlgrabber? Can you instead of closing this one, open new one on urlgrabber and set it as blocking this one?

> And I'm not quite OK with breaking existing functionality to support a feature that's mostly ignored anyway (wget).
So because someone else ignore standard (Hypertext Transfer Protocol -- HTTP/1.1 RFC 2616, 19.5.1) it means we can ignore it as well?

Comment 8 Zdeněk Pavlas 2014-01-07 12:00:43 UTC
> But it is not baseurl. It is url of repo file. And linked documentation clearly state, that it should be url of repo file.

It's both, actually. But yum-config-manager has no means to guess that.

> So because someone else ignore standard (Hypertext Transfer Protocol -- HTTP/1.1 RFC 2616, 19.5.1) it means we can ignore it as well?

According to section 15.5, Content-Disposition is not part of the HTTP standard.
I understand that someone might consider saving http://coolsite.com/funny.gif to evil.exe a really cool feature.


Note You need to log in before you can comment on or make changes to this bug.