Bug 438031

Summary: yumdownloader doesn't do .src.rpms
Product: [Fedora] Fedora Reporter: James M. Leddy <james.leddy>
Component: yum-utilsAssignee: Seth Vidal <skvidal>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 8CC: gborsuk, gmetcalfe, james.antill, jhutar, opensource, pavel1r, pmatilai, tim.lauridsen
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-09-26 05:18:20 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
yum repos config dir
none
yum.conf none

Description James M. Leddy 2008-03-18 19:04:07 UTC
Description of problem:

yumdownloader doesn't work on source packages

Version-Release number of selected component (if applicable):
yum-utils-1.1.11-1.fc8


How reproducible:
Always


Steps to Reproduce:

$ yumdownloader --source kernel
updates-source            100% |=========================| 2.1 kB    00:00     
livna                     100% |=========================| 2.1 kB    00:00     
livna-debuginfo           100% |=========================| 1.9 kB    00:00     
fedora                    100% |=========================| 2.1 kB    00:00     
livna-source              100% |=========================| 1.9 kB    00:00     
updates-debuginfo         100% |=========================| 1.9 kB    00:00     
fedora-debuginfo          100% |=========================| 2.1 kB    00:00     
updates                   100% |=========================| 2.3 kB    00:00     
fedora-source             100% |=========================| 2.1 kB    00:00     
No source RPM found for kernel - 2.6.23.1-42.fc8.x86_64
No source RPM found for kernel - 2.6.24.3-34.fc8.x86_64
Nothing to download
$ yumdownloader kernel
kernel-2.6.24.3-34.fc8.x8 100% |=========================|  17 MB    01:47  
$ ls
BUILD                         kernel-2.6.24.3-34.fc8.x86_64.rpm  SOURCES  SRPMS
httpd-2.2.8-1.fc8.x86_64.rpm  RPMS                               SPECS

Expected results:
I get the .src.rpm

I've tried yum clean all, but that doesn't solve the problem either.

Comment 1 James M. Leddy 2008-03-18 19:04:07 UTC
Created attachment 298441 [details]
yum repos config dir

Comment 2 Seth Vidal 2008-03-18 19:19:36 UTC
Can you attach your yum.conf file and any excludes you might have in there?


Comment 3 James M. Leddy 2008-03-18 19:35:51 UTC
Created attachment 298443 [details]
yum.conf

in response to comment 2

Comment 4 seth vidal 2008-03-18 19:59:48 UTC
What version of yum is this? just 3.2.8 from f8?

I'm asking b/c I can't make this happen at all, locally using the same ver of
yum-utils.

Can you try out yum from rawhide?
yum install pygpgme
yum --enablerepo='development' update yum


Comment 5 James M. Leddy 2008-03-18 21:36:13 UTC
$ rpm -q yum
yum-3.2.8-2.fc8


Comment 6 James M. Leddy 2008-03-18 23:13:24 UTC
james@zeik ~
$ yumdownloader --source kernel

updates-source            100% |=========================| 2.1 kB    00:00     
livna                     100% |=========================| 2.1 kB    00:00     
livna-debuginfo           100% |=========================| 1.9 kB    00:00     
fedora                    100% |=========================| 2.1 kB    00:00     
livna-source              100% |=========================| 1.9 kB    00:00     
updates-debuginfo         100% |=========================| 1.9 kB    00:00     
fedora-debuginfo          100% |=========================| 2.1 kB    00:00     
updates                   100% |=========================| 2.3 kB    00:00     
fedora-source             100% |=========================| 2.1 kB    00:00     
No source RPM found for kernel-2.6.23.1-42.fc8.x86_64
No source RPM found for kernel-2.6.24.3-34.fc8.x86_64
Nothing to download

This after the steps in comment 4 were taken.  This issue isn't a big deal, I'll
take a look at it tomorrow

Comment 7 seth vidal 2008-03-19 02:21:04 UTC
okay  - just b/c this has me boggled.
run this:
yum --enablerepo='*-source' list kernel\*

and post the output, please.




Comment 8 James M. Leddy 2008-03-19 18:21:55 UTC
(un)fortunately it started working again.  I don't know why, I just sat down to
my computer and tried it this morning and it worked.  I think it might have been
an infrastructure problem, though that's a huge shot in the dark.

in response to comment 7:

$ yum --enablerepo='*-source' list kernel\*



Cannot access repository dir //var/cache/yum/updates-testing-source


Comment 9 James M. Leddy 2008-03-19 18:29:57 UTC
(In reply to comment #8)
> (un)fortunately it started working again.  I don't know why, I just sat down to
> my computer and tried it this morning and it worked.  I think it might have been
> an infrastructure problem, though that's a huge shot in the dark.

Sorry, I was incorrect when I said that, the problem still persists



Comment 10 seth vidal 2008-03-19 18:37:36 UTC
run the above yum command as root, please.


Comment 11 James M. Leddy 2008-03-19 20:09:06 UTC
reply to comment 10

# yum --enablerepo='*-source' list kernel\*

updates-source            100% |=========================| 2.1 kB    00:00     
livna                     100% |=========================| 2.1 kB    00:00     
updates-testing-source    100% |=========================| 2.1 kB    00:00     
primary.sqlite.bz2        100% |=========================|  97 kB    00:00     
livna-debuginfo           100% |=========================| 1.9 kB    00:00     
fedora                    100% |=========================| 2.1 kB    00:00     
livna-source              100% |=========================| 1.9 kB    00:00     
development-source        100% |=========================| 1.9 kB    00:00     
primary.sqlite.bz2        100% |=========================| 1.6 MB    00:10     
updates-debuginfo         100% |=========================| 1.9 kB    00:00     
fedora-debuginfo          100% |=========================| 2.1 kB    00:00     
livna-development-source  100% |=========================| 1.9 kB    00:00     
primary.sqlite.bz2        100% |=========================|  43 kB    00:00     
updates                   100% |=========================| 2.3 kB    00:00     
fedora-source             100% |=========================| 2.1 kB    00:00     
livna-testing-source      100% |=========================|  951 B    00:00     
primary.xml.gz            100% |=========================| 2.1 kB    00:00     
livna-test: ################################################## 8/8
Installed Packages
kernel.x86_64                            2.6.23.15-137.fc8      installed       
kernel.x86_64                            2.6.24.3-34.fc8        installed       
kernel-debuginfo.x86_64                  2.6.24.3-34.fc8        installed       
kernel-debuginfo-common.x86_64           2.6.24.3-34.fc8        installed       
kernel-headers.x86_64                    2.6.24.3-34.fc8        installed       
kernel-xen.x86_64                        2.6.21.7-2.fc8         installed       
kernel-xen.x86_64                        2.6.21-2952.fc8        installed       
Available Packages
kernel-debug.x86_64                      2.6.24.3-34.fc8        updates         
kernel-debug-debuginfo.x86_64            2.6.24.3-34.fc8        updates-debuginf
kernel-debug-devel.x86_64                2.6.24.3-34.fc8        updates         
kernel-devel.x86_64                      2.6.24.3-34.fc8        updates         
kernel-doc.noarch                        2.6.24.3-34.fc8        updates         
kernel-xen-2.6-debuginfo-common.x86_64   2.6.21.7-2.fc8         updates-debuginf
kernel-xen-2.6-doc.noarch                2.6.21.7-2.fc8         updates         
kernel-xen-debuginfo.x86_64              2.6.21.7-2.fc8         updates-debuginf
kernel-xen-devel.x86_64                  2.6.21.7-2.fc8         updates     

Comment 12 seth vidal 2008-03-19 20:30:30 UTC
one more thing just to make sure I'm not crazy:

repoquery --repoid=fedora-source --repoid=updates-source --archlist=src -q kernel

thanks


Comment 13 James M. Leddy 2008-03-19 20:37:23 UTC
(In reply to comment #12)
> one more thing just to make sure I'm not crazy:
> 
> repoquery --repoid=fedora-source --repoid=updates-source --archlist=src -q kernel


# repoquery --repoid=fedora-source --repoid=updates-source --archlist=src -q kernel

kernel-0:2.6.24.3-34.fc8.src

>
>thanks

no problem

Comment 14 Seth Vidal 2008-05-30 15:10:28 UTC
Can you still make this bug happen? B/c every test I've tried I cannot make it
happen, at all.



Comment 15 Pavel Rosenboim 2008-06-02 08:21:29 UTC
Happened to me just now.

Comment 16 Tim Lauridsen 2008-06-02 11:35:54 UTC
(In reply to comment #15)
> Happened to me just now.

Please submit some more information.
* is this on F8 or F9
* yum version ( yum --version )
* the output from 'yumdownloader kernel'
* the output from 'yum list kernel\*'
* the output from 'yum repolist'





Comment 17 Pavel Rosenboim 2008-06-02 18:43:55 UTC
(In reply to comment #16)
> (In reply to comment #15)

> * is this on F8 or F9
This is F8

> * yum version ( yum --version )
This is F8.
# yum --version
3.2.8

> * the output from 'yumdownloader kernel'
# yumdownloader kernel
Loading "protectbase" plugin
Loading "fedorakmod" plugin
Loading "kernel-module" plugin
Loading "upgrade-helper" plugin
Loading "priorities" plugin
Loading "skip-broken" plugin
livna                     100% |=========================| 2.1 kB    00:00     
fedora                    100% |=========================| 2.1 kB    00:00     
adobe-linux-i386          100% |=========================|  951 B    00:00     
updates                   100% |=========================| 2.3 kB    00:00     
freshrpms                 100% |=========================| 2.4 kB    00:00     
remi                      100% |=========================| 1.9 kB    00:00     
0 packages excluded due to repository protections
120 packages excluded due to repository priority protections
kernel-2.6.24.7-92.fc8.i6 100% |=========================| 4.4 MB    00:14 


> * the output from 'yum list kernel\*'
# yum list kernel\*
Loading "protectbase" plugin
Loading "downloadonly" plugin
Loading "tmprepo" plugin
Loading "fedorakmod" plugin
Loading "kernel-module" plugin
Loading "protect-packages" plugin
Loading "upgrade-helper" plugin
Loading "priorities" plugin
Loading "changelog" plugin
Loading "allowdowngrade" plugin
Loading "skip-broken" plugin
0 packages excluded due to repository protections
120 packages excluded due to repository priority protections
Installed Packages
kernel.i686                              2.6.23.1-42.fc8        installed       
kernel.i686                              2.6.24.7-92.fc8        installed       
kernel.i686                              2.6.24.5-85.fc8        installed       
kernel-devel.i686                        2.6.24.5-85.fc8        installed       
kernel-devel.i686                        2.6.24.7-92.fc8        installed       
kernel-devel.i686                        2.6.23.1-42.fc8        installed       
kernel-doc.noarch                        2.6.24.7-92.fc8        installed       
kernel-headers.i386                      2.6.24.7-92.fc8        installed       
kernel-xen.i686                          2.6.21.7-3.fc8         installed       
kernel-xen-2.6-doc.noarch                2.6.21.7-3.fc8         installed       
Available Packages
kernel.i586                              2.6.24.7-92.fc8        updates         
kernel-PAE.i686                          2.6.24.7-92.fc8        updates         
kernel-PAE-debug.i686                    2.6.24.7-92.fc8        updates         
kernel-PAE-debug-devel.i686              2.6.24.7-92.fc8        updates         
kernel-PAE-devel.i686                    2.6.24.7-92.fc8        updates         
kernel-debug.i686                        2.6.24.7-92.fc8        updates         
kernel-debug-devel.i686                  2.6.24.7-92.fc8        updates         
kernel-devel.i586                        2.6.24.7-92.fc8        updates         
kernel-xen-devel.i686                    2.6.21.7-3.fc8         updates 

> * the output from 'yum repolist'
# yum repolist
Loading "protectbase" plugin
Loading "downloadonly" plugin
Loading "tmprepo" plugin
Loading "fedorakmod" plugin
Loading "kernel-module" plugin
Loading "protect-packages" plugin
Loading "upgrade-helper" plugin
Loading "priorities" plugin
Loading "changelog" plugin
Loading "allowdowngrade" plugin
Loading "skip-broken" plugin
repo id              repo name                                 status
adobe-linux-i386     Adobe Systems Incorporated                enabled
fedora               Fedora 8 - i386                           enabled
freshrpms            Fedora Core 8 - i386 - Freshrpms          enabled
livna                Livna for Fedora Core 8 - i386 - Base     enabled
remi                 Les RPM de remi pour Fedora 8 - i386      enabled
updates              Fedora 8 - i386 - Updates                 enabled

Also:
# yumdownloader --source kernel
Loading "protectbase" plugin
Loading "fedorakmod" plugin
Loading "kernel-module" plugin
Loading "upgrade-helper" plugin
Loading "priorities" plugin
Loading "skip-broken" plugin
0 packages excluded due to repository protections
120 packages excluded due to repository priority protections
Enabling updates-source repository
updates-source            100% |=========================| 2.1 kB    00:00     
0 packages excluded due to repository protections
2044 packages excluded due to repository priority protections
Enabling livna-source repository
livna-source              100% |=========================| 1.9 kB    00:00     
0 packages excluded due to repository protections
174 packages excluded due to repository priority protections
Enabling fedora-source repository
fedora-source             100% |=========================| 2.1 kB    00:00     
0 packages excluded due to repository protections
4778 packages excluded due to repository priority protections
No source RPM found for kernel - 2.6.23.1-42.fc8.i586
No source RPM found for kernel - 2.6.23.1-42.fc8.i686
No source RPM found for kernel - 2.6.24.7-92.fc8.i586
No source RPM found for kernel - 2.6.24.7-92.fc8.i686
Nothing to download



Comment 18 James M. Leddy 2008-06-02 21:17:58 UTC
Started working for me after I _disabled_ the source repo.  Suspected that it
might work after reading this code block from line 79 of /usr/bin/yumdownloader:

    def setupSourceRepos(self):
        # enable the -source repos for enabled primary repos
        archlist = rpmUtils.arch.getArchList() + ['src']    
        # Ok, we have src and bin repos. What we want to do here is:
        #
        # 1. _enable_ source repos for which the bin repos are enabled.
        # 2. _disable_ the _other_ src repos.
        #
        # ...also we don't want to disable the src repos. for #1 and then
        # re-enable them as then we get annoying messages and call .close() on
        # them losing the primarydb data etc.

        # Get all src repos.
        src_repos = {}
        for repo in self.repos.findRepos('*-source'):
            src_repos[repo.id] = False

        #  Find the enabled bin repos, and mark their respective *-source repo.
        # as good.
        for repo in self.repos.listEnabled():
            if repo.id not in src_repos:
                srcrepo = '%s-source' % repo.id
                if srcrepo in src_repos:
                    src_repos[srcrepo] = True

        # Toggle src repos that are set the wrong way
        for repo in self.repos.findRepos('*-source'):
            if     repo.isEnabled() and not src_repos[repo.id]:
                repo.close()
                self.repos.disableRepo(repo.id)
            if not repo.isEnabled() and     src_repos[repo.id]:
                self.logger.info('Enabling %s repository' % repo.id)
                repo.enable()
                # Setup the repo, without a cache
                repo.setup(0)
                # Setup pkgSack with 'src' in the archlist
                self._getSacks(archlist=archlist, thisrepo=repo.id)


Seth, 

Based on this, it's hard to know why source repos should have an enabled flag to
begin with.  Is it possible to install .src.rpm from yum?

Comment 19 Seth Vidal 2008-06-03 04:30:04 UTC
no, it is not possible to install .src.rpms from yum.

source repos are not enabled at all. If they are for you then something changed
that option in your config file.


Comment 20 James Antill 2008-06-03 14:27:34 UTC
 So I specifically enable the source repos. to make sure this gets tested, and I
can't reproduce your problem (although I have the latest Fed-9 yum installed).

% rpm -q yum yum-utils
yum-0:3.2.14-7.fc9.noarch
yum-utils-0:1.1.14-3.fc8.noarch
% yumdownloader --source --urls ustr
[...]
No source RPM found for ustr-1.0.4-1.fc8.x86_64
No source RPM found for ustr-1.0.4-1.fc8.i386
http://mirrors.usc.edu/pub/linux/distributions/fedora/linux/updates/8/SRPMS/ustr-1.0.4-6.fc8.src.rpm
% yumdownloader -x ustr-1.0.4\* --source --urls ustr
[...]
http://fedora.mirrors.tds.net/pub/fedora/releases/8/Everything/source/SRPMS/ustr-1.0.1-6.fc8.src.rpm


...which is with fedora-source and updates-source pre-enabled (along with the
binary repos.)
 The only thing I can think is that changing the config. tells yum not to trust
the timeout on the metadata, but that seems unlikely.

 I guess it's possible there is a problem with 3.2.8, if you turn the optuion
back and still have problems ... could you try this patch to yumdownloader:

diff --git a/yumdownloader.py b/yumdownloader.py
index 8141b48..cc331c5 100755
--- a/yumdownloader.py
+++ b/yumdownloader.py
@@ -112,6 +112,7 @@ class YumDownloader(YumUtilBase):
             if not repo.isEnabled() and     src_repos[repo.id]:
                 self.logger.info('Enabling %s repository' % repo.id)
                 repo.enable()
+            if repo.isEnabled():
                 # Setup the repo, without a cache
                 repo.setup(0)
                 # Setup pkgSack with 'src' in the archlist


...although, as I said, yumdownloader works fine for me without it.


Comment 21 Till Maas 2008-07-07 18:53:51 UTC
yumdownloader --source kernel does not download anything on my F8 machine, too. 

I do not see which information is missing here, i.e. why this bug is in
NEEDINFO, therefore I remove the NEEDINFO.


Comment 22 Seth Vidal 2008-07-07 19:06:18 UTC
Till,
 If you could, upgrade your yum to the version in F9, temporarily, and tell me
if the problem vanishes. If it does, then that tells us what we need to do. If
it does not then we do other things.

thanks,


Comment 23 James M. Leddy 2008-07-07 20:28:16 UTC
(In reply to comment #19)
> no, it is not possible to install .src.rpms from yum.
> 
> source repos are not enabled at all. If they are for you then something changed
> that option in your config file.
> 

I know they weren't enabled by default.  However, I went through myself an
enabled them, thinking that it was required in order to use yumdownloader. 
After I disabled them again, it started working properly, though this doesn't
seem to fix it for others that still see the issue.

Comment 24 Till Maas 2008-07-07 20:58:59 UTC
I found out why it did not work for me. I use protect=yes and the protectbase
plugin, but I did not use "protect=yes" for the -source repositories in the
yum.repos.d dir. Quoting from comment #17:

| updates-source            100% |=========================| 2.1 kB    00:00     
| 0 packages excluded due to repository protections
| 2044 packages excluded due to repository priority protections

It seems that the packages in updates mask all packages in updates-source then
and therefore no src.rpms are found. But maybe it is some related cause, because
the updates repository itself is not mentioned here. Using yumdownload
--noplugins --source kernel also works.


Comment 25 Till Maas 2008-07-07 21:01:49 UTC
Btw. with the yum (yum-3.2.16-2.fc9) and yum-* from F9 on F8 I still have to add
protect=yes to the -source repositories or use --noplugins.

Comment 26 Jan Hutař 2008-07-22 12:08:36 UTC
FYI: I have tried to manually allow updates-source repo and then call 
`yumdownloader --source kernel` and kernel is correctly downloaded (F9, 
yum-3.2.16-4.fc9.noarch)

Comment 27 Tim Lauridsen 2008-07-25 08:33:47 UTC
Look like the issue here is that the proctectbase plugins don't work very well
with the yumdownloader when downloading source rpms.

can everybody confirm that

if 
yumdownloader --source kernel
fails

then 
yumdownloader --disableplugin=protectbase --source kernel 
will work.


Comment 28 Jan Hutař 2008-07-25 11:22:42 UTC
Still works for me with both yum-protectbase (is there any config I should 
add?) not installed/installed.

# yum repolist
Loaded plugins: allowdowngrade, changelog, downloadonly, merge-conf, presto,
              : protectbase, refresh-packagekit, refresh-updatesd, tmprepo,
              : upgrade-helper, verify
Setting up and reading Presto delta metadata
...
fedora               Fedora 9 - x86_64                        enabled :  12,444
fedora-debuginfo     Fedora 9 - x86_64 - Debug                enabled :   3,597
updates              Fedora 9 - x86_64 - Updates              enabled :   3,670
updates-debuginfo    Fedora 9 - x86_64 - Updates - Debug      enabled :     873
updates-presto       Fedora 9 - x86_64 - Updates Presto       enabled :   4,827

# rpm -q yum
yum-3.2.17-2.fc9.noarch

Comment 29 Greg Metcalfe 2008-08-21 19:57:29 UTC
I was able to grab a .src.rpm (kernel-doc), in a fully-updated F8 system, but it didn't install--just left it in PWD. I did that via rpm. WRT #23, I'd made no changes to any repos. I don't know if that's intended behavior.

Comment 30 Tim Lauridsen 2008-08-25 09:50:15 UTC
(In reply to comment #29)
> I was able to grab a .src.rpm (kernel-doc), in a fully-updated F8 system, but
> it didn't install--just left it in PWD. I did that via rpm. WRT #23, I'd made
> no changes to any repos. I don't know if that's intended behavior.

yumdownloader just download packages to PWD, it don't install anything, sp this is expected behavior.

Comment 31 James Antill 2008-09-26 05:18:20 UTC
 I'm going to close this NAB, in theory the plugin should "be more clever" but it's doing exactly what you told it to ... feel free to open an RFE against yum-utils, but making protectbase easier to use isn't high on anyone's TODO list.