Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1095829

Summary: Fedora repo publish failing in Publishing Distribution files
Product: [Retired] Pulp Reporter: Preethi Thomas <pthomas>
Component: rpm-supportAssignee: Chris Duryee <cduryee>
Status: CLOSED CURRENTRELEASE QA Contact: Ina Panova <ipanova>
Severity: unspecified Docs Contact:
Priority: high    
Version: 2.4 BetaCC: cduryee, igulina, ipanova, skarmark
Target Milestone: ---Keywords: Triaged
Target Release: 2.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-09 06:56:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Preethi Thomas 2014-05-08 15:29:54 UTC
Description of problem:

Fedora repo publish failing in Publishing Distribution files

Version-Release number of selected component (if applicable):
[root@pulp-24-server ~]# rpm -qa pulp-server
pulp-server-2.4.0-0.13.beta.el6.noarch
[root@pulp-24-server ~]# 

How reproducible:


Steps to Reproduce:
1.create a fedora repo
2. sync the repo
3.

Actual results:
[root@pulp-24-server ~]# pulp-admin rpm repo create --repo-id fedora20 --feed http://download.eng.bos.redhat.com/released/F-20/GOLD/Fedora/x86_64/os/
Successfully created repository [fedora20]

[root@pulp-24-server ~]# 
[root@pulp-24-server ~]# 
[root@pulp-24-server ~]# pulp-admin rpm repo sync run --repo-id fedora20
+----------------------------------------------------------------------+
                  Synchronizing Repository [fedora20]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       4071/4071 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 8/8 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[|]
... completed


Task Succeeded



Publishing Distribution files
[-]

Task Failed

Pulp exception occurred: PulpExecutionException

Expected results:


Additional info:


May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [fedora20]
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last):
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     publish_report = publish_repo(transfer_repo, conduit, call_config)
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 543, in wrap_f
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     return f(*args, **kwargs)
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 130, in publish_repo
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     return self._publisher.publish()
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 169, in publish
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     step.process()
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 368, in process
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     self._process_block()
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 603, in _process_block
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     self.process_unit(package_unit)
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 375, in process_unit
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     self._publish_distribution_files(unit)
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 426, in _publish_distribution_files
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     self._create_symlink(source_path, symlink_path)
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 447, in _create_symlink
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR:     raise RuntimeError(msg % {'l': link_path})
May  8 12:15:46 pulp-24-server pulp: pulp.server.managers.repo.publish:ERROR: RuntimeError: Link path [/var/lib/pulp/working/repos/fedora20/distributors/yum_distributor/repodata/repomd.xml] exists, but is not a symbolic link
May  8 12:15:46 pulp-24-server pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[8c4c9cc9-4f16-4c94-923d-66f852a52ac7]

Comment 1 Chris Duryee 2014-05-12 19:56:02 UTC
the /os/ dir has a .treeinfo file in it which should not exist (per bcourt). We are investigating further.

Comment 2 Preethi Thomas 2014-05-13 16:08:19 UTC
just fyi,
[root@ibm-x3250m4-04 ~]# pulp-admin rpm repo create --repo-id f20 --feed http://ftp.swin.edu.au/fedora/linux/releases/20/Fedora/x86_64/os/
Successfully created repository [f20]


I tried an external fedora repo and that has failed in repo publishing of Distribution files

[root@ibm-x3250m4-04 ~]# pulp-admin rpm repo sync run --repo-id f20+----------------------------------------------------------------------+
                     Synchronizing Repository [f20]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       4071/4071 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 8/8 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[\]
... completed


Task Succeeded



[|]
Waiting to begin...
Publishing Distribution files
[-]

Task Failed

Pulp exception occurred: PulpExecutionException

[root@ibm-x3250m4-04 ~]# 

From the log

pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [f20]
pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last):
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
pulp.server.managers.repo.publish:ERROR:     publish_report = publish_repo(transfer_repo, conduit, call_config)
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 543, in wrap_f
pulp.server.managers.repo.publish:ERROR:     return f(*args, **kwargs)
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 130, in publish_repo
pulp.server.managers.repo.publish:ERROR:     return self._publisher.publish()
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 169, in publish
pulp.server.managers.repo.publish:ERROR:     step.process()
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 368, in process
pulp.server.managers.repo.publish:ERROR:     self._process_block()
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 603, in _process_block
pulp.server.managers.repo.publish:ERROR:     self.process_unit(package_unit)
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 375, in process_unit
pulp.server.managers.repo.publish:ERROR:     self._publish_distribution_files(unit)
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 426, in _publish_distribution_files
pulp.server.managers.repo.publish:ERROR:     self._create_symlink(source_path, symlink_path)
pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 447, in _create_symlink
pulp.server.managers.repo.publish:ERROR:     raise RuntimeError(msg % {'l': link_path})
pulp.server.managers.repo.publish:ERROR: RuntimeError: Link path [/var/lib/pulp/working/repos/f20/distributors/yum_distributor/repodata/repomd.xml] exists, but is not a symbolic link

Comment 3 Chris Duryee 2014-05-14 18:31:08 UTC
https://github.com/pulp/pulp_rpm/pull/485

Comment 4 Randy Barlow 2014-05-15 17:00:14 UTC
The fix for this bug is included in the pulp-2.4.0-0.14.beta builds.

Comment 5 Ina Panova 2014-05-15 18:42:57 UTC
# pulp-admin -u admin -p admin rpm repo create --repo-id fedora --feed http://ftp.linux.cz/pub/linux/fedora/linux/releases/20/Fedora/x86_64/os/
Successfully created repository [fedora]


[root@ec2-54-216-83-245 ~]# pulp-admin -u admin -p admin rpm repo sync run --repo-id fedora
+----------------------------------------------------------------------+
                   Synchronizing Repository [fedora]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       4071/4071 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 7/7 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[/]
... completed


Task Succeeded



[|]
Waiting to begin...
Publishing Distribution files
[\]
... completed
Publishing RPMs
[==================================================] 100%
4071 of 4071 items
... completed

Publishing Delta RPMs
[-]
... completed

Publishing Errata
[-]
... completed

Publishing Comps file
[==================================================] 100%
124 of 124 items
... completed

Publishing Metadata.
[-]
... completed

Copying files to master directory
[|]
... completed

Publishing via http
... skipped

Publishing via https
[-]
... completed


Task Succeeded

root@ec2-54-216-83-245 ~]# pulp-admin -u admin -p admin rpm repo publish run --repo-id fedora
+----------------------------------------------------------------------+
                     Publishing Repository [fedora]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
4071 of 4071 items
... completed

Publishing Delta RPMs
[-]
... completed

Publishing Errata
[-]
... completed

Publishing Comps file
[==================================================] 100%
124 of 124 items
... completed

Publishing Metadata.
[-]
... completed

Copying files to master directory
[/]
... completed

Publishing via http
... skipped

Publishing via https
[-]
... completed


Task Succeeded

Tested in  pulp-2.4.0-0.14.beta 
Moving to Verified.

Comment 6 Ina Panova 2014-05-15 18:56:53 UTC
*** Bug 1098192 has been marked as a duplicate of this bug. ***

Comment 7 Sayli Karmarkar 2014-05-16 16:09:19 UTC
*** Bug 1098195 has been marked as a duplicate of this bug. ***

Comment 8 Randy Barlow 2014-08-09 06:56:27 UTC
This has been fixed in Pulp 2.4.0-1.