Bug 710469

Summary: RFE: adjust yum-downloader logic for source repos in CDN
Product: Red Hat Enterprise Linux 6 Reporter: Dennis Gregorovic <dgregor>
Component: yum-utilsAssignee: James Antill <james.antill>
Status: CLOSED ERRATA QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.0CC: dmach, josh.yggdrasil, ksrot, mmalik, mvadkert, pknirsch, toracat, zpavlas
Target Milestone: rcKeywords: FutureFeature
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of:
: 1082050 (view as bug list) Environment:
Last Closed: 2011-12-06 17:35:24 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 743047    

Description Dennis Gregorovic 2011-06-03 13:40:39 UTC
It came to my attention that the logic for --source in yumdownloader is not going to work with the repos that we've created in the CDN.  yumdownloader only looks at repos with a label of <base repo>-source.  For example, if you have a repo of binary rpms named rhel-6-workstation-rpms, it expects the SRPMs to be in rhel-6-workstation-rpms-source.  However, in the CDN the repo is named rhel-6-workstation-source-rpms.

One low-tech solution would be to have yum downloader check for this pattern as well.  Something like:

srcrepo = repo.id.replace('-rpms', '-source-rpms')
if srcrepo in src_repos:
    src_repos[srcrepo] = True

Alternatively, we could use yum repo tagging to explicitly list the label of the source repo in the repodata of the binary repo.

Comment 1 James Antill 2011-06-15 14:38:48 UTC
Can you add a 'source' tag to the repos. ... I'd much prefer that over having two naming conventions :).

Fedora has:

 <tags>
   <content>source</content>
[...]
 </tags>

...so we could trigger off that.

Comment 2 Dennis Gregorovic 2011-06-21 14:07:34 UTC
James, if we add the 'source' tag to the repos containing SRPMs, do you still need some way of mapping those source repos back to the corresponding binary repos?

Todd, does Pulp supporting adding these kind of tags to repo data?

Comment 3 James Antill 2011-06-21 21:18:26 UTC
 n/m ... I've thought about it more, and one problem is finding the tag from a repo. before we've enabled it. We'll basically have to downloaded just the repomd.xml files for all disabled repos. ... which sucks at first glance, and I don't want to sign off for 6.2.

 So we'll just do the low tech. thing as a RHEL-6.2 hack ... we can maybe fix it better later on (ha :).

 Also note that currently there's no generic "enable source" code/API/etc. ... so if you want repoquery changed as well, then speak now (or open another BZ) ... as I'll hopefully assign this to someone else :).

Comment 4 James Antill 2011-07-07 16:16:38 UTC
*** Bug 719654 has been marked as a duplicate of this bug. ***

Comment 5 Dennis Gregorovic 2011-07-07 16:42:33 UTC
Low tech for 6.2 sounds good.  I don't think we need to worry about repoquery or any other tools.

Comment 6 Dennis Gregorovic 2011-07-07 17:22:52 UTC
(In reply to comment #4)
> *** Bug 719654 has been marked as a duplicate of this bug. ***

Bug 719654 is actually a slightly different issue.  There, the problem is that the repo we provide has a label of "rhel-source", but there is no "rhel" repo, so the source repo doesn't get enabled.  The low-tech hack for this would be:

if "rhel-source" in src_repos and re.match(r'^rhel-.+-6(?:$|-)', repo.id):
    src_repos["rhel-source"] = True

Alternatively, we could change the label of the source repo to "rhel-x86_64-server-6-source".

Comment 7 Karel Srot 2011-07-25 11:04:44 UTC
Hi,
I just want to make sure I understand this feature so it will be properly tested. yumdownloader --source is enabling disabled repos if the repo id matches the xxx-source pattern (where xxx is ordinary repo). The request here is to update yumdownloader so it enables also repos with 'xxx-source-rpms' (related to xxx-rpms repo).
Is that right? Thank you in advance.

Comment 9 James Antill 2011-08-09 20:24:00 UTC
Did this simple hack for yumdownloader and yum-builddep.

Comment 11 Karel Srot 2011-08-26 10:57:22 UTC
Hi James,

yum-builddep patch had broken the old functionality.
That's because 

  if rhn_source_repos and (not repo.id.endswith('-source-rpms') and
          repo.id.endswith('-rpms')):
      srcrepo = repo.id.replace('-rpms', '-source-rpms')

ignores xxxx-rpms-source repos made from 'xxx-rpms' repo.

Can be fixed with something like

--- /usr/bin/yum-builddep	2011-08-26 06:50:50.917012374 -0400
+++ /usr/bin/yum-builddep.new	2011-08-26 06:50:31.179981191 -0400
@@ -122,7 +122,7 @@
                 issource_repo = True
             if rhn_source_repos and (not repo.id.endswith('-source-rpms') and
                                      repo.id.endswith('-rpms')):
-                srcrepo = repo.id.replace('-rpms', '-source-rpms')
+                srcrepo = '%s,%s' % (repo.id+'-source', repo.id.replace('-rpms', '-source-rpms'))
             elif not issource_repo:
                 srcrepo = '%s-source' % repo.id
             else:


Switching bug back to assigned.

Comment 12 Akemi Yagi 2011-08-31 20:22:43 UTC
Bug #688321 (closed as INSUFFICIENT_DATA) may be related to this bug?

Comment 13 Karel Srot 2011-09-01 05:12:37 UTC
(In reply to comment #12)
> Bug #688321 (closed as INSUFFICIENT_DATA) may be related to this bug?

I don't think so. James have added the patch (RFE implementation) to yum-utils-1.1.30-7. Bug 688321 was for yum-utils-1.1.26-11.el6.

Comment 14 Zdeněk Pavlas 2011-09-13 10:38:09 UTC
*** Bug 732649 has been marked as a duplicate of this bug. ***

Comment 15 Zdeněk Pavlas 2011-09-13 10:41:30 UTC
BZ-710469-rhn-source-repos.patch updated, merged in Karel's change, also fixes BZ 732649.

Comment 16 Karel Srot 2011-09-14 11:12:10 UTC
with yum-utils-1.1.30-8 the new functionality doesn't work (both with yumdownloader and yum-builddep):

# yumdownloader --source test-pkgB
Loaded plugins: product-id
Enabling bz710469-repoA-rpms-source repository
No source RPM found for test-pkgB-1.0-1.noarch
Nothing to download

but yum should enable bz710469-repoA-source-rpms repository too (or instead):

# cat /etc/yum.repos.d/bz710469.repo 
[bz710469-repoA-rpms]
name=bz710469-repoA-rpms
baseurl=file:///tmp/tmp.vUj599YG5v/repoA-rpms
enabled=1
gpgcheck=0

[bz710469-repoA-rpms-source]
name=bz710469-repoA-rpms-source
baseurl=file:///tmp/tmp.vUj599YG5v/repoA-rpms-source
enabled=0
gpgcheck=0

[bz710469-repoB-rpms]
name=bz710469-repoB-rpms
baseurl=file:///tmp/tmp.vUj599YG5v/repoB-rpms
enabled=1
gpgcheck=0

[bz710469-repoB-source-rpms]
name=bz710469-repoB-source-rpms
baseurl=file:///tmp/tmp.vUj599YG5v/repoB-source-rpms
enabled=0
gpgcheck=0

Comment 17 Karel Srot 2011-09-14 11:16:05 UTC
Btw, yumdownloader traceback fix + my patch fixes this bug just fine. Something extra you have done breaks the functionality.

Comment 18 Zdeněk Pavlas 2011-09-14 12:45:33 UTC
Sorry, missed that the previous patch was different from what's upstream and also changed the default value of rhn_source_repos.

Comment 21 errata-xmlrpc 2011-12-06 17:35:24 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1703.html