Bug 817124

Summary:

unable to sync unprotected custom repo's in 2.0.1 -> 2.0.3 migration

Product:

Red Hat Update Infrastructure for Cloud Providers

Reporter:

wes hayutin <whayutin>

Component:

RHUA

Assignee:

James Slagle <jslagle>

Status:

CLOSED NOTABUG

QA Contact:

wes hayutin <whayutin>

Severity:

unspecified

Docs Contact:

Priority:

unspecified

Version:

2.0.3

CC:

kbidarka, sghai, tsanders

Target Milestone:

---

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2012-04-30 20:05:27 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
sos report	none

Description wes hayutin 2012-04-27 18:40:14 UTC

Description of problem:
.com//pulp/repos/protected_arch/x86_64]
2012-04-27 14:28:00,445 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [protected_arch] from [https://dhcp231-185.rdu.redhat.com//pulp/repos/protected_arch/x86_64] to [/var/lib/pulp-cds/repos/protected_arch/x86_64]
2012-04-27 14:28:00,446 [DEBUG][worker-0] _sync_repo() @ cdslib.py:331 - Configuring repository for authentication
2012-04-27 14:28:02,410 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://dhcp231-185.rdu.redhat.com//pulp/repos/protected_arch/x86_64]
2012-04-27 14:28:02,411 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://dhcp231-185.rdu.redhat.com//pulp/repos/unprotected/x86_64]
2012-04-27 14:28:02,411 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [unprotected] from [https://dhcp231-185.rdu.redhat.com//pulp/repos/unprotected/x86_64] to [/var/lib/pulp-cds/repos/unprotected/x86_64]
2012-04-27 14:28:03,306 [ERROR][worker-0] sync() @ cdslib.py:181 - Error performing repo sync
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/cds/cdslib.py", line 178, in sync
    self._sync_repo(base_url, repo)
  File "/usr/lib/python2.6/site-packages/pulp/cds/cdslib.py", line 359, in _sync_repo
    fetch.fetchYumRepo(repo_path, verify_options=verify_options)
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 144, in fetchYumRepo
    self.setupYumInfo()
  File "/usr/lib/python2.6/site-packages/grinder/RepoFetch.py", line 108, in setupYumInfo
    info.setUp()
  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 343, in setUp
    skip=self.skip)
  File "/usr/lib/python2.6/site-packages/grinder/activeobject.py", line 82, in __call__
    return self.object(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/grinder/activeobject.py", line 267, in __call__
    return self.__call(method, args, kwargs)
  File "/usr/lib/python2.6/site-packages/grinder/activeobject.py", line 243, in __call
    return self.__rmi(method.name, args, kwargs)
  File "/usr/lib/python2.6/site-packages/grinder/activeobject.py", line 136, in __rmi
    raise Exception(ex)
Exception: Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/grinder/activeobject.py", line 429, in process
    retval = method(*args, **kwargs)

  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 96, in getDownloadItems
    self.__getRepoData()

  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 169, in __getRepoData
    for ftype in self.__getRepoXmlFileTypes():

  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 156, in __getRepoXmlFileTypes
    return self.repo.repoXML.fileTypes()

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1462, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1458, in _getRepoXML
    raise Errors.RepoError, msg

RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: . Please verify its path and try again

2012-04-27 14:28:03,308 [INFO][worker-0] update_cluster_membership() @ cdslib.py:236 - Received cluster membership update; Cluster [rhui-cds-cluster], Members [dhcp-230-102.rdu.redhat.com]
2012-04-27 14:28:03,308 [INFO][worker-0] update_cluster_membership() @ cdslib.py:247 - No changes needed to be made to cluster memberships

Comment 1 wes hayutin 2012-04-27 18:43:16 UTC

Created attachment 580839 [details]
sos report

Comment 2 wes hayutin 2012-04-27 18:46:10 UTC

[root@dhcp231-185 ~]# ll /var/lib/pulp/repos/unprotected/x86_64/repodata/
total 28
-rw-r--r--. 1 apache apache  940 Apr 27 14:27 filelists.sqlite.bz2
-rw-r--r--. 1 apache apache  444 Apr 27 14:27 filelists.xml.gz
-rw-r--r--. 1 apache apache 1739 Apr 27 14:27 other.sqlite.bz2
-rw-r--r--. 1 apache apache 1066 Apr 27 14:27 other.xml.gz
-rw-r--r--. 1 apache apache 2647 Apr 27 14:27 primary.sqlite.bz2
-rw-r--r--. 1 apache apache 1071 Apr 27 14:27 primary.xml.gz
-rw-r--r--. 1 apache apache 2689 Apr 27 14:27 repomd.xml

Comment 3 wes hayutin 2012-04-27 19:22:54 UTC

recreate:
in 2.0.1 or a 2.0.3 migrated from 2.0.1

1. setup rhui
2. create a custom unprotected repo
3. run a cds sync

RHUA

[root@dhcp231-185 ~]# rpm -qa | grep pulp
m2crypto-0.21.1.pulp-7.el6.x86_64
pulp-0.0.263-18.el6.noarch
pulp-common-0.0.263-18.el6.noarch
pulp-client-lib-0.0.263-18.el6.noarch
pulp-admin-0.0.263-18.el6.noarch
pulp-consumer-0.0.263-18.el6.noarch
pulp-selinux-server-0.0.263-18.el6.noarch
mod_wsgi-3.3-2.pulp.el6.x86_64
python-isodate-0.4.4-4.pulp.el6.noarch
python-oauth2-1.5.170-2.pulp.el6.noarch
[root@dhcp231-185 ~]# 

CDS

[root@dhcp-230-102 ~]# rpm -qa | grep pulp
mod_wsgi-3.3-2.pulp.el6.x86_64
pulp-common-0.0.263-18.el6.noarch
pulp-selinux-server-0.0.263-18.el6.noarch
m2crypto-0.21.1.pulp-7.el6.x86_64
pulp-cds-0.0.263-18.el6.noarch
[root@dhcp-230-102 ~]#

Comment 4 wes hayutin 2012-04-29 13:53:32 UTC

unable to recreate in a fresh install of 2.0.3


2012-04-29 09:52:36,113 [DEBUG][worker-0] _sync_repo() @ cdslib.py:331 - Configuring repository for authentication
2012-04-29 09:52:38,693 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://ip-10-12-19-13.ec2.internal//pulp/repos/content/dist/rhel/rhui/server/6/6Server/x86_64/rhui/2.0/os]
2012-04-29 09:52:38,694 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://ip-10-12-19-13.ec2.internal//pulp/repos/unprotected]
2012-04-29 09:52:38,695 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [unprotected] from [https://ip-10-12-19-13.ec2.internal//pulp/repos/unprotected] to [/var/lib/pulp-cds/repos/unprotected]
2012-04-29 09:52:40,762 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://ip-10-12-19-13.ec2.internal//pulp/repos/unprotected]
2012-04-29 09:52:40,764 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://ip-10-12-19-13.ec2.internal//pulp/repos/wes/x86_64]
2012-04-29 09:52:40,765 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [wes] from [https://ip-10-12-19-13.ec2.internal//pulp/repos/wes/x86_64] to [/var/lib/pulp-cds/repos/wes/x86_64]
2012-04-29 09:52:42,800 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://ip-10-12-19-13.ec2.internal//pulp/repos/wes/x86_64]


Seems to be only a migration issue

Comment 5 wes hayutin 2012-04-30 20:05:27 UTC

After migrating to the very latest RHUI code..
2.0.3 seems to resolve the above issue w/ unprotected repos..

Red Hat content was available
custom protected was available
custom unprotected was available


[root@qeblade6 ~]# yum repolist
Loaded plugins: product-id, rhui-lb, security, subscription-manager
Updating certificate-based repositories.
repo id                         repo name                                 status
rhui-custom-10000               Custom Repositories - 10000                1
rhui-rhel-x86_64-6-rhui-2-rpms  Red Hat Update Infrastructure 2.0 (RPMs)  39
rhui-unprotected_wes            unprotected_wes                            2
repolist: 42
[root@qeblade6 ~]# ls
all2-1.0-1.el6.noarch.rpm  install.log.syslog
anaconda-ks.cfg            KORN-1.0-1.el6.noarch.rpm
cobbler.ks                 RECIPE.TXT
install.log                RHUI20-1.0-1.el6.noarch.rpm
[root@qeblade6 ~]# yumdownloader aspell
Loaded plugins: product-id, rhui-lb
aspell-0.60.6-12.el6.x86_64.rpm                          | 648 kB     00:00     
aspell-0.60.6-12.el6.i686.rpm                            | 629 kB     00:00     
[root@qeblade6 ~]# yumdownloader zsh
Loaded plugins: product-id, rhui-lb
zsh-4.3.10-4.1.el6.x86_64.rpm                            | 2.1 MB     00:00     
[root@qeblade6 ~]# yumdownloader js
Loaded plugins: product-id, rhui-lb
js-1.70-12.el6_0.x86_64.rpm                              | 365 kB     00:00

Comment 6 wes hayutin 2012-04-30 20:06:08 UTC

[root@dhcp-230-102 ~]# pulp-migrate 
removing persisted tasks
database migration to version 40 complete
[root@dhcp-230-102 ~]# cat /dev/null > /var/log/audit/audit.log 
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# service pulp-server restart
Stopping httpd:                                            [  OK  ]
Stopping Qpid AMQP daemon:                                 [  OK  ]
Stopping mongod:                                           [  OK  ]
Starting mongod:                                           [  OK  ]
Starting Qpid AMQP daemon:                                 [  OK  ]
Starting httpd:                                            [  OK  ]
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# cat /var/log/audit/audit.log | grep -i denied
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# setenforce 1
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# 
[root@dhcp-230-102 ~]# rpm -qa | grep pulp
m2crypto-0.21.1.pulp-7.el6.x86_64
pulp-consumer-0.0.263-19.el6.noarch
pulp-selinux-server-0.0.263-19.el6.noarch
pulp-common-0.0.263-19.el6.noarch
pulp-client-lib-0.0.263-19.el6.noarch
pulp-0.0.263-19.el6.noarch
python-oauth2-1.5.170-2.pulp.el6.noarch
pulp-admin-0.0.263-19.el6.noarch
python-isodate-0.4.4-4.pulp.el6.noarch
mod_wsgi-3.3-2.pulp.el6.x86_64