Bug 597212 - yumdownloader --source creates empty files
Summary: yumdownloader --source creates empty files
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: yum-utils (Show other bugs)
(Show other bugs)
Version: 6.0
Hardware: All Linux
low
medium
Target Milestone: rc
: ---
Assignee: James Antill
QA Contact: BaseOS QE Security Team
URL:
Whiteboard:
Keywords:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2010-05-28 11:52 UTC by Milos Malik
Modified: 2014-01-21 06:18 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-09-20 20:12:41 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

Description Milos Malik 2010-05-28 11:52:16 UTC
Description of problem:


Version-Release number of selected component (if applicable):
yum-3.2.27-9.el6.noarch
yum-metadata-parser-1.1.2-14.1.el6.i686
yum-rhn-plugin-0.9.1-5.el6.noarch
yum-utils-1.1.26-9.el6.noarch

How reproducible:
always

Steps to Reproduce:
# mkdir temp
# cd temp
# yumdownloader setroubleshoot
Loaded plugins: refresh-packagekit, rhnplugin
This system is not registered with RHN.
RHN support will be disabled.
setroubleshoot-2.2.86-1.el6.i686.rpm                     | 137 kB     00:00     
# ls -l
total 140
-rw-r--r--. 1 root root 139860 May 24 11:32 setroubleshoot-2.2.86-1.el6.i686.rpm
# yumdownloader --source setroubleshoot
Loaded plugins: refresh-packagekit, rhnplugin
This system is not registered with RHN.
RHN support will be disabled.
Enabling rhel-beta-optional-source repository
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/optional/source/repodata/repomd.xml: [Errno 14] HTTP Error 404 : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/optional/source/repodata/repomd.xml 
Trying other mirror.
Enabling rhel-beta-source repository
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/source/repodata/repomd.xml: [Errno 14] HTTP Error 404 : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/source/repodata/repomd.xml 
Trying other mirror.
No source RPM found for setroubleshoot-2.2.86-1.el6.i686
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/source/SRPMS/setroubleshoot-2.2.52-1.el6.src.rpm: [Errno 14] HTTP Error 404 : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/source/SRPMS/setroubleshoot-2.2.52-1.el6.src.rpm 
Trying other mirror.
Could not download/verify pkg setroubleshoot-2.2.52-1.el6.src: failure: SRPMS/setroubleshoot-2.2.52-1.el6.src.rpm from rhel-beta-source: [Errno 256] No more mirrors to try.
# ls -l
total 140
-rw-r--r--. 1 root root      0 May 28 07:47 setroubleshoot-2.2.52-1.el6.src.rpm
-rw-r--r--. 1 root root 139860 May 24 11:32 setroubleshoot-2.2.86-1.el6.i686.rpm
# 

  
Actual results:
src.rpm file is empty

Expected results:
either no src.rpm file exists or src.rpm file is complete

Comment 1 RHEL Product and Program Management 2010-05-28 11:55:41 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 2 Bill Nottingham 2010-05-28 17:15:50 UTC
Enabling rhel-beta-optional-source repository
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/optional/source/repodata/repomd.xml:
[Errno 14] HTTP Error 404 :

...

Are you sure your mirror is sane?

Comment 3 Milos Malik 2010-06-02 15:47:37 UTC
I'm not sure. I was looking for a repository with source RPMs and I could not find anything better than this. 
Following URL works:
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/optional/source/SRPMS/

Comment 4 Bill Nottingham 2010-06-02 20:49:24 UTC
Are you still able to reproduce this?

Comment 5 James Antill 2010-06-02 21:57:26 UTC
This BZ is still against yum-utils, but most of the conversation looks like "mirroring" or "rel-eng" or something ...

From the yum-utils side I can kind of see why you don't want us to create 0 length files, even if all the urls produce 404s ... the problem is I'm sure we will get 666 BZ reports if we delete non-zero files. And it's also probably more wrong to delete zero length files, if they already existed on disk.

So it's not _trivial_ to do this "well".

I don't mind leaving it as an RFE that we may be able to do at some point, but I'm also happy to not bother if the real problem here is that you don't want all your mirrors to give you 404s (in which case please close this BZ for me :).

Comment 6 Milos Malik 2010-06-03 07:16:39 UTC
With following records in /etc/yum.repos.d/rhel-beta.repo I'm still able to reproduce the problem (src.rpm files have zero length):

[rhel-beta-source]
name=Red Hat Enterprise Linux $releasever Beta - $basearch - Source
# baseurl=ftp://ftp.redhat.com/pub/redhat/rhel/beta/$releasever/source/
baseurl=http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100601.n.0/6.0/Server/source/SRPMS/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta-2

[rhel-beta-optional-source]
name=Red Hat Enterprise Linux $releasever Beta (Optional) - $basearch - Source
# baseurl=ftp://ftp.redhat.com/pub/redhat/rhel/beta/$releasever/optional/source/
baseurl=http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100601.n.0/6.0/Server/optional/source/SRPMS/
enabled=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-beta-2

Comment 7 Milos Malik 2010-06-03 07:35:06 UTC
In comment #0 you can see that yum wants to download src.rpm for setroubleshoot-2.2.86-1.el6.i686 which is a perfect idea, but it tries to download repodata from incorrect location. Then it prints:

    No source RPM found for setroubleshoot-2.2.86-1.el6.i686

After that it tries to download src.rpm from correct location ( http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100528.n.0/6.0/Server/source/SRPMS/ ), but somehow the NVR of the package changed ( setroubleshoot-2.2.52-1.el6 ). Why?

If yum tried to download src.rpm for setroubleshoot-2.2.86-1.el6 package from the second location, it would have succeeded.

Comment 8 James Antill 2010-06-08 04:01:23 UTC
I assume you have more repos. than what you are showing, that's the only sane reason the NVR could change ... what does the following print:

yum repolist -v disabled \*-source

Comment 9 Milos Malik 2010-06-09 05:48:17 UTC
# yum repolist -v disabled \*-source
Loading "rhnplugin" plugin
Loading "refresh-packagekit" plugin
Config time: 0.094
This system is not registered with RHN.
RHN support will be disabled.
Yum Version: 3.2.27
Setting up Package Sacks
pkgsack time: 0.029
Repo-id      : rhel-beta-optional-source
Repo-name    : Red Hat Enterprise Linux 6 Beta (Optional) - i386 - Source
Repo-status  : disabled
Repo-baseurl : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/optional/source/
Repo-expire  : 21,600 second(s) (last: Wed May 19 09:46:05 2010)

Repo-id      : rhel-beta-source
Repo-name    : Red Hat Enterprise Linux 6 Beta - i386 - Source
Repo-status  : disabled
Repo-baseurl : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source/
Repo-expire  : 21,600 second(s) (last: Wed May 19 09:46:11 2010)

repolist: 0
# echo $?
0

Comment 10 Milos Malik 2010-06-09 05:50:35 UTC
# yum clean all
Loaded plugins: refresh-packagekit, rhnplugin
Cleaning up Everything
# yumdownloader --source policycoreutils
Loaded plugins: refresh-packagekit, rhnplugin
This system is not registered with RHN.
RHN support will be disabled.
rhel-beta                                                | 3.8 kB     00:00     
rhel-beta/primary_db                                     | 2.4 MB     00:00     
rhel-beta-optional                                       | 3.0 kB     00:00     
rhel-beta-optional/primary_db                            | 878 kB     00:00     
rhts                                                     |  951 B     00:00     
rhts/primary                                             |  66 kB     00:00     
rhts                                                                    224/224
Enabling rhel-beta-optional-source repository
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/optional/source/repodata/repomd.xml: [Errno 14] HTTP Error 404 : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/optional/source/repodata/repomd.xml 
Trying other mirror.
Enabling rhel-beta-source repository
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source/repodata/repomd.xml: [Errno 14] HTTP Error 404 : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source/repodata/repomd.xml 
Trying other mirror.
No source RPM found for policycoreutils-2.0.82-28.el6.i686
http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source/SRPMS/policycoreutils-2.0.78-11.el6.src.rpm: [Errno 14] HTTP Error 404 : http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source/SRPMS/policycoreutils-2.0.78-11.el6.src.rpm 
Trying other mirror.
Could not download/verify pkg policycoreutils-2.0.78-11.el6.src: failure: SRPMS/policycoreutils-2.0.78-11.el6.src.rpm from rhel-beta-source: [Errno 256] No more mirrors to try.
# ls -l policycoreutils*
-rw-r--r--. 1 root root 0 Jun  9 01:47 policycoreutils-2.0.78-11.el6.src.rpm
#

Comment 11 James Antill 2010-06-09 14:04:03 UTC
Ok, in comment#10 yum is trying to download the repomd.xml file for the configured repos. with the baseurl of:

http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source

...which is what is shown in comment#9. This is apparently the wrong baseurl as it is failing (404). Yum will then "revert" back to any data it already had, which appears to be repomd.xml from when the baseurl was configured as:

http://download.englab.brq.redhat.com/nightly/RHEL6.0-20100608.n.2/6.0/Server/source/SRPMS

...this data includes old primary data etc. ... which is why it is looking for old packages which no longer exist.

Comment 12 RHEL Product and Program Management 2010-07-15 14:47:52 UTC
This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 13 James Antill 2010-09-20 20:12:41 UTC
Going to close this, as I said before if you really want us to try and delete zero-legnth files on 404s ... open an RFE against urlgrabber. But it's probably not going to happen for 6.1


Note You need to log in before you can comment on or make changes to this bug.