Bug 697087

Summary: yum repolist is not producing a list when one of the repo baseurl causes a forbidden 403
Product: Red Hat Enterprise Linux 5 Reporter: John Sefler <jsefler>
Component: yumAssignee: James Antill <james.antill>
Status: CLOSED ERRATA QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: high Docs Contact:
Priority: high    
Version: 5.7CC: bkearney, cduryee, ebenes, ksrot, pknirsch, spandey
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: yum-3.2.22-34.el5 Doc Type: Bug Fix
Doc Text:
Previously, when one of the repository baseurl addresses caused an HTTP error code to be issued, the "yum repolist" command failed to produce the list of available repositories. This bug has been fixed and the repository list is now properly returned even if an error occurs.
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-07-21 11:24:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description John Sefler 2011-04-15 19:16:50 UTC
Description of problem:
As a follow-on to the fix for bug 696786, I'm finding that in rhel5 "yum repolist" appears to block on the first repo encountered that has a bad baseurl where as in rhel6, it did not block.

For example, using an onpremises candlpin server with the TESTDATA imported and all the generated product certs copied to the client, I autosubscribe and run yum repolist as follows:

[root@jsefler-onprem-server ~]# subscription-manager register --username=testuser1 --password=password --autosubscribe
814e2f9f-2c01-4fcd-ad90-4a925eb481e6 jsefler-onprem-server.usersys.redhat.com
Installed Products:
   Awesome OS Modifier Bits - Subscribed
   Awesome OS Scalable Filesystem Bits - Subscribed
   Awesome OS Workstation Bits - Subscribed
   Awesome OS Server Bits - Subscribed
   Shared Storage Bits - Subscribed
   Management Bits - Subscribed
   Large File Support Bits - Subscribed
   Load Balancing Bits - Subscribed
   Clustering Bits - Subscribed
   Red Hat Enterprise Linux Scalable File System (for RHEL 6 Server) - Not Subscribed
   Red Hat Enterprise Linux Load Balancer (for RHEL 6 Server) - Not Subscribed
   Awesome OS Developer Bits - Not Subscribed
   Awesome OS for S390X Bits - Not Subscribed
   Red Hat Enterprise Linux 6 Server - Not Subscribed
   Red Hat Enterprise Linux High Availability (for RHEL 6 Server) - Not Subscribed
   Awesome OS Developer Basic - Not Subscribed
   Red Hat Enterprise Linux Resilient Storage (for RHEL 6 Server) - Not Subscribed
   Multiplier Product Bits - Not Subscribed
   Awesome OS Premium Architecture Bits - Not Subscribed
[root@jsefler-onprem-server ~]# yum repolist enabled --disableplugin=rhnplugin
Loaded plugins: product-id, refresh-packagekit, subscription-manager
No plugin match for: rhnplugin
Updating Red Hat repositories.
https://cdn.redhat.com/foo/path/always/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403"
Trying other mirror.
repo id                              repo name                            status
always-enabled-content               always-enabled-content               0
awesomeos-modifier                   awesomeos-modifier                   0
awesomeos-server-scalable-fs         awesomeos-server-scalable-fs         0
awesomeos-workstation-scalable-fs    awesomeos-workstation-scalable-fs    0
content-label                        content                              0
repolist: 0

^^^ NOTICE THAT DESPITE THE 403 ERROR, I STILL SEE THE awesomeos* REPOS IN THE LIST.  I have grown to expect this.


NOW LET'S TRY ON RHEL 5...

[root@jsefler-onprem-5server tmp]# subscription-manager register --username=testuser1 --password=password --autosubscribe 
facbcf18-d2e6-4a5e-a78b-81713fa421a6 jsefler-onprem-5server.usersys.redhat.com
Installed Products:
   Awesome OS Workstation Bits - Subscribed
   Awesome OS Modifier Bits - Subscribed
   Shared Storage Bits - Subscribed
   Management Bits - Subscribed
   Load Balancing Bits - Subscribed
   Awesome OS Server Bits - Subscribed
   Large File Support Bits - Subscribed
   Clustering Bits - Subscribed
   Awesome OS Scalable Filesystem Bits - Subscribed
   Awesome OS Developer Bits - Not Subscribed
   Awesome OS Developer Basic - Not Subscribed
   Multiplier Product Bits - Not Subscribed
   Awesome OS for S390X Bits - Not Subscribed
   Awesome OS Premium Architecture Bits - Not Subscribed
[root@jsefler-onprem-5server tmp]# yum repolist enabled --disableplugin=rhnplugin
Loaded plugins: product-id, security, subscription-manager
Updating Red Hat repositories.
https://cdn.redhat.com/foo/path/always/repodata/repomd.xml: [Errno 14] HTTP Error 403: Forbidden
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: always-enabled-content. Please verify its path and try again


^^^ BANG! WHERE ARE THE awesomeos* REPOS THAT WERE DISPLAYED ON RHEL6?  IT APPEARS THAT THE FIRST BAD baseurl (for always-enabled-content) IS BLOCKING THE REPOLIST (good or bad) FROM BEING LISTED.



Version-Release number of selected component (if applicable):
[root@jsefler-onprem-5server tmp]# rpm -q subscription-manager
subscription-manager-0.95.5.7-1.git.12.6e2fc59.el5



Additional info:
This breaks many of our automated tests.

Comment 1 Chris Duryee 2011-04-15 19:38:35 UTC
This is a difference in the way yum works between 5 and 6, our code isn't being touched during this. We need to look into it more to see if we can avoid it via our repo config.

Comment 2 John Sefler 2011-04-15 21:08:04 UTC
I agreed that this may be a difference in the way yum works between rhel5 (yum-3.2.22-33.el5) and rhel6 (yum-3.2.29-7.el6.noarch).  Unfortunately it impacts our subscription-manager tests that were originally authored using rhel6.  Is it possible that:

1. We could open a defect against RHEL5.7 to pull in a newer version of yum that does not exhibit this failure.  (Maybe this has already has been done?)  
2. We could run yum on rhel5 (as well as rhel6) with an option or configuration setting that ignores the 403 when running yum repolist.  (Does this option/config exist?)

Comment 3 James Bowes 2011-04-27 17:54:12 UTC
Moving over to yum since we can't do anything about this (or shouldn't) from our plugin

Comment 4 John Sefler 2011-04-27 18:24:37 UTC
Additional Info....
Here are the contents of my repo file....

[root@jsefler-onprem-5server ~]# cat /etc/yum.repos.d/redhat.repo 
#
# Red Hat Repositories
# Managed by (rhsm) subscription-manager
#

[awesomeos-workstation-scalable-fs]
name = awesomeos-workstation-scalable-fs
baseurl = https://cdn.redhat.com/path/to/awesomeos-workstation-scalable-fs
enabled = 1
gpgcheck = 1
gpgkey = https://cdn.redhat.com/path/to/awesomeos/gpg/
sslverify = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslclientkey = /etc/pki/entitlement/key.pem
sslclientcert = /etc/pki/entitlement/8367634176913608279.pem
metadata_expire = 3600

[awesomeos-modifier]
name = awesomeos-modifier
baseurl = http://example.com/awesomeos-modifier
enabled = 1
gpgcheck = 1
gpgkey = http://example.com/awesomeos-modifier/gpg
sslverify = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslclientkey = /etc/pki/entitlement/key.pem
sslclientcert = /etc/pki/entitlement/834137930840854788.pem

[awesomeos-server-scalable-fs]
name = awesomeos-server-scalable-fs
baseurl = https://cdn.redhat.com/path/to/awesomeos-server-scalable-fs
enabled = 1
gpgcheck = 1
gpgkey = https://cdn.redhat.com/path/to/awesomeos/gpg/
sslverify = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslclientkey = /etc/pki/entitlement/key.pem
sslclientcert = /etc/pki/entitlement/8367634176913608279.pem
metadata_expire = 3600

[always-enabled-content]
name = always-enabled-content
baseurl = https://cdn.redhat.com/foo/path/always
enabled = 1
gpgcheck = 1
gpgkey = https://cdn.redhat.com/foo/path/always/gpg
sslverify = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslclientkey = /etc/pki/entitlement/key.pem
sslclientcert = /etc/pki/entitlement/7590966368525320704.pem
metadata_expire = 200

[content-label]
name = content
baseurl = https://cdn.redhat.com/foo/path
enabled = 1
gpgcheck = 1
gpgkey = https://cdn.redhat.com/foo/path/gpg/
sslverify = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslclientkey = /etc/pki/entitlement/key.pem
sslclientcert = /etc/pki/entitlement/7590966368525320704.pem
metadata_expire = 0

[never-enabled-content]
name = never-enabled-content
baseurl = https://cdn.redhat.com/foo/path/never
enabled = 0
gpgcheck = 1
gpgkey = https://cdn.redhat.com/foo/path/never/gpg
sslverify = 1
sslcacert = /etc/rhsm/ca/redhat-uep.pem
sslclientkey = /etc/pki/entitlement/key.pem
sslclientcert = /etc/pki/entitlement/7590966368525320704.pem
metadata_expire = 600




Here is what I get when running "yum repolist enabled" ...

[root@jsefler-onprem-5server ~]# yum repolist enabled
Loaded plugins: product-id, rhnplugin, security, subscription-manager
Updating Red Hat repositories.
This system is not registered with RHN.
RHN Satellite or RHN Classic support will be disabled.
https://cdn.redhat.com/foo/path/always/repodata/repomd.xml: [Errno 14] HTTP Error 403: Forbidden
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: always-enabled-content. Please verify its path and try again
[root@jsefler-onprem-5server ~]# 

^^^
Note the output from yum repolist seems to get blocked on the first 403 and the output does not produce a any list of enabled repos.  On the otherhand, rhel 6 does produce a list of enabled repos depite the 403 error.  I'd like to see the same behavior in rhel5.7 as I see in rhel6.

Comment 5 James Antill 2011-04-28 20:04:52 UTC
 Ok, so if I apply the following, then it should do what you want ... and the only conflicts I get are in docs/yum.8


 It's still quite a bit for RHEL-5, but I'm ok with it (it should all be confined to repolist command):

commit fd1632c733c47719fb7e69d7e2a9d18f7c05b0ad
Author: James Antill <james>
Date:   Tue Aug 11 13:33:27 2009 -0400

     Add expire data to repolist -v
    

commit 7eaae631092cdd2d09ece8b87b648ca316c909fe
Author: James Antill <james>
Date:   Tue Aug 11 17:01:07 2009 -0400

    Cleanup whitespace for repolist -v

commit 2ffe59eb3e441f3fce3038d1c19079cc5c59eb74
Author: James Antill <james>
Date:   Mon Sep 21 02:05:29 2009 -0400

    Catch and ignore any repo. errors, and allow cache only (BZ 524454)

commit 1934f8832dc22802913ce79eb45724f10d44edc8
Author: Ville-Pekka Vainio <vpivaini.fi>
Date:   Mon Nov 16 20:38:50 2009 +0200

    Fix UnicodeDecodeErrors in yumcommands.py
    
    Fix two UnicodeDecodeErrors which were raised when running 'yum -v
    repolist' in fi_FI.utf8 with the yet uncommitted Finnish translation.

commit 4aa1a6639a1544bc714b75b61d5cbf9d5ce34fc3
Author: James Antill <james>
Date:   Tue Dec 1 10:06:09 2009 -0500

    Make status output nicer for repolist disabled/all/enabled, BZ 540489

commit ac8eed8587a347dddf827efd1c0d7ab8674b2a79
Author: James Antill <james>
Date:   Sat Jan 16 12:42:51 2010 -0500

     A few minor tweaks to repolist output:
[...]

commit 7f9d8c86ed44f3826288c86bf74e9366b22ac532
Author: James Antill <james>
Date:   Mon Jan 18 08:42:14 2010 -0500

    Fix the repolist cacheonly feature, document it

Comment 14 Tomas Capek 2011-07-15 16:13:26 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
Previously, when one of the repository baseurl addresses caused an HTTP error code to be issued, the "yum repolist" command failed to produce the list of available repositories. This bug has been fixed and the repository list is now properly returned even if an error occurs.

Comment 15 errata-xmlrpc 2011-07-21 11:24:05 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2011-1060.html