Bug 1021672

Summary:	RHEL 5 repository metadata incorrect for anaconda
Product:	[Retired] Pulp	Reporter:	Justin Sherrill <jsherril>
Component:	rpm-support	Assignee:	Barnaby Court <bcourt>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	high	Docs Contact:
Priority:	urgent
Version:	Master	CC:	mhrivnak, mmccune, pthomas
Target Milestone:	---	Keywords:	Triaged
Target Release:	2.3.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:
Clones:	1021674 (view as bug list)		Environment:
Last Closed:	2013-12-09 14:30:11 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	950743, 1021674

Description Justin Sherrill 2013-10-21 18:51:32 UTC

Description of problem:

Metadata for published RHEL 5 repositories are not structured in a way that is usable by anaconda.

Similar to this bugzilla (but slightly different): https://bugzilla.redhat.com/show_bug.cgi?id=805925

Pulp generates:

./repodata/
and
./Server/repodata/

and puts the rpms in ./

The metadata in ./repodata is correct, but the metadta in ./Server/repodata is not, it makes it appear as if the rpms are actually stored in ./Server:


eg:

  <location href="gnome-python2-gtkmozembed-2.14.2-7.el5.x86_64.rpm"/>


Version-Release number of selected component (if applicable):

pulp-server-2.3.0-0.22.beta.el6sat.noarch


How reproducible:
Always

Steps to Reproduce:  
1. Sync a RHEL 5 kickstart tree from the cdn
2.  Try to install/kickstart a machine against it


Actual results:
Failure, system can't find packages

Expected results:
RHEL 5 system can be provisioned.

Comment 1 Michael Hrivnak 2013-10-21 19:13:17 UTC

This looks similar to an issue we've seen before, which resulted in us adding a "Packages" symlink in our repo. Jconnor did that work and can hopefully shed some light on this issue as well.

Comment 2 Barnaby Court 2013-10-22 19:16:58 UTC

Justin, Can you provide a URL for the repo on the CDN that you are running into trouble syncing?

Comment 4 Barnaby Court 2013-10-23 17:42:36 UTC

PR: https://github.com/pulp/pulp_rpm/pull/361

This fix requires a sync to be performed on the affected repo after it has been applied.  After a sync the repo will publish a link to all of the files to the directory specified in [general]->packagedir subdirectory specified in the treeinfo file.

Comment 5 Mike McCune 2013-10-23 20:22:33 UTC

pulp bug is MODIFIED, reflecting accordingly

Comment 6 Mike McCune 2013-10-23 20:23:19 UTC

ignore previous comment, wrong BZ

Comment 7 Jeff Ortel 2013-10-23 23:27:07 UTC

build: 2.3.0-0.24.beta

Comment 8 Preethi Thomas 2013-10-24 19:30:30 UTC

fails-qa
Looks like repo publish is failing



[root@ibm-x3550m3-11 ~]#  pulp-admin rpm repo create --repo-id rhel59-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/5/5.9/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key 
Successfully created repository [rhel59-os]


On the first attempt


[root@ibm-x3550m3-11 ~]#  pulp-admin rpm repo sync run --repo-id rhel59-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel59-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       15362/15362 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 19/19 items
... completed

Importing errata...
[|]
... completed

Importing package groups/categories...
[\]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 19/19 items
... completed

Publishing packages...
[==================================================] 100%
Packages: 15362/15362 items
Individual errors encountered during publishing:

An unexpected error has occurred. More information can be found in the client
log file ~/.pulp/admin.log.


On the 2nd attempt


[root@ibm-x3550m3-11 ~]#  pulp-admin rpm repo sync run --repo-id rhel59-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel59-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 19/19 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[\]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
[root@ibm-x3550m3-11 ~]#  pulp-admin rpm repo publish run --repo-id rhel59-os
+----------------------------------------------------------------------+
                   Publishing Repository [rhel59-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
[root@ibm-x3550m3-11 ~]# 


From pulp.log on publish

2013-10-24 14:58:58,440 pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [rhel59-os]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 121, in _do_publish
    publish_report = distributor_instance.publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/pulp/plugins/distributors/yum_distributor/distributor.py", line 328, in publish_repo
    distro_status, distro_errors = self.symlink_distribution_unit_files(distro_units, repo.working_dir, publish_conduit, progress_callback)
  File "/usr/lib/pulp/plugins/distributors/yum_distributor/distributor.py", line 609, in symlink_distribution_unit_files
    os.makedirs(package_path)
  File "/usr/lib64/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/var/lib/pulp/working/repos/rhel59-os/distributors/yum_distributor/Server'
2013-10-24 14:58:58,440 pulp.server.dispatch.task:ERROR: Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 97, in publish
    result = self._do_publish(repo, distributor_id, distributor_instance, transfer_repo, conduit, call_config)
  File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 121, in _do_publish
    publish_report = distributor_instance.publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/pulp/plugins/distributors/yum_distributor/distributor.py", line 328, in publish_repo
    distro_status, distro_errors = self.symlink_distribution_unit_files(distro_units, repo.working_dir, publish_conduit, progress_callback)
  File "/usr/lib/pulp/plugins/distributors/yum_distributor/distributor.py", line 609, in symlink_distribution_unit_files
    os.makedirs(package_path)
  File "/usr/lib64/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
PulpExecutionException: Pulp exception occurred: PulpExecutionException
2013-10-24 14:58:58,441 pulp.server.dispatch.task:INFO: FAILURE: Task 843233ad-5ed1-4f16-9dd2-c9e63cc5f55e: CallRequest: RepoPublishManager.publish(u'rhel59-os', 'yum_distributor', publish_config_override=None)

Comment 9 Barnaby Court 2013-10-24 20:40:00 UTC

PR: https://github.com/pulp/pulp_rpm/pull/362

Comment 10 Jeff Ortel 2013-10-29 15:29:40 UTC

build: 2.3.0-0.26.beta

Comment 11 Preethi Thomas 2013-10-29 17:27:35 UTC

Resynced a rhel5 repo

kickstated a vm with that

 virt-install --vnc --name=rhel5-ks --ram=1024 --vcpu=4 --file=/var/lib/libvirt/images/rhel5-ks --file-size=14 --vnc --location=http://ibm-x3550m3-11.lab.eng.brq.redhat.com/pulp/repos/content/dist/rhel/rhui/server/5/5.9/x86_64/os/ --prompt

Comment 12 Preethi Thomas 2013-10-30 12:18:56 UTC

[root@dell-pe2900-01 ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.26.beta.el6.noarch
[root@dell-pe2900-01 ~]# 


root@dell-pe2900-01 ~]# pulp-admin rpm repo create --repo-id rhel59-os --feed https://cdn.redhat.com/content/dist/rhel/rhui/server/5/5.9/x86_64/os/ --feed-ca-cert CDN/cdn.redhat.com-chain.crt --feed-cert CDN/1359391926_4512.crt  --feed-key CDN/1359391926_4512.key  --proxy-host http://pulp-proxy.usersys.redhat.com --proxy-port 8888 --proxy-user admin --proxy-pass admin 
Successfully created repository [rhel59-os]
[root@dell-pe2900-01 ~]#  pulp-admin rpm repo sync run --repo-id rhel59-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel59-os]
+----------------------------------------------------------------------+

A sync task is already in progress for this repository. Its progress will be
tracked below.

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       15362/15362 items
Delta RPMs: 0/0 items

... completed

Individual package errors encountered during sync:

An unexpected error has occurred. More information can be found in the client
log file ~/.pulp/admin.log.

[root@dell-pe2900-01 ~]#  pulp-admin rpm repo sync run --repo-id rhel59-os
+----------------------------------------------------------------------+
                  Synchronizing Repository [rhel59-os]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       8/8 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 19/19 items
... completed

Importing errata...
[/]
... completed

Importing package groups/categories...
[-]
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 19/19 items
... completed

Publishing packages...
[==================================================] 100%
Packages: 15362/15362 items
... completed

Generating metadata
[|]
... completed

Publishing repository over HTTPS
[-]
... completed

Publishing repository over HTTP
[-]
... completed


Kickstarted a vm

[root@qe-blade-15 ~]#  virt-install --vnc --name=rhel5-ks-new --ram=1024 --vcpu=4 --file=/var/lib/libvirt/images/rhel5-ks-new --file-size=14 --vnc --location=http://dell-pe2900-01.rhts.eng.bos.redhat.com/pulp/repos/content/dist/rhel/rhui/server/5/5.9/x86_64/os/ --prompt
Would you like to use KVM acceleration? (yes or no) yes

Comment 13 Preethi Thomas 2013-12-09 14:30:11 UTC

Pulp 2.3 released.