Bug 1102745

Summary: publish error when the synced rpm exists in an uploaded repo
Product: [Retired] Pulp Reporter: Preethi Thomas <pthomas>
Component: rpm-supportAssignee: Chris Duryee <cduryee>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 2.4 BetaCC: cduryee, skarmark
Target Milestone: ---Keywords: Triaged
Target Release: 2.4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-09 06:56:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Preethi Thomas 2014-05-29 13:23:58 UTC
Description of problem:
When an rpm exist in the uploaded repo and if you try to sync repo which has the same rpm, the publish fails with the following error

Will not create a symlink to a non-existent source

Version-Release number of selected component (if applicable):
[root@ibm-x3250m4-04 ~]# rpm -qa pulp-server
pulp-server-2.4.0-0.18.beta.el6.noarch
[root@ibm-x3250m4-04 ~]# 

How reproducible:


Steps to Reproduce:
1. upload an rpm to a repo
2. create another repo with the feed pointing to repo that has the same rpm you have uploaded
3.Run repo sync

Actual results:
root@ibm-x3250m4-04 ~]# pulp-admin rpm repo sync run --repo-id pulp
+----------------------------------------------------------------------+
                    Synchronizing Repository [pulp]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[================================                  ] 65%
RPMs:       2/3 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 3/3 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed


Task Succeeded



Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[================                                  ] 33%
1 of 3 items
... failed
Will not create a symlink to a non-existent source
[/var/lib/pulp/content/rpm/pulp-dot-2.0-test/0.1.2/1.fc11/x86_64/435d92e6c09248b
501b8d2ae786f92ccfad69fab8b1bc774e2b66ff6c0d83979/pulp-dot-2.0-test-0.1.2-1.fc11
.x86_64.rpm]


Task Failed

Will not create a symlink to a non-existent source
[/var/lib/pulp/content/rpm/pulp-dot-2.0-test/0.1.2/1.fc11/x86_64/435d92e6c09248b
501b8d2ae786f92ccfad69fab8b1bc774e2b66ff6c0d83979/pulp-dot-2.0-test-0.1.2-1.fc11
.x86_64.rpm]




Expected results:


Additional info:


May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR: Exception caught from plugin during publish for repo [pulp]
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR: Traceback (most recent call last):
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 126, in _do_publish
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     publish_report = publish_repo(transfer_repo, conduit, call_config)
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 473, in wrap_f
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     return f(*args, **kwargs)
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 130, in publish_repo
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     return self._publisher.publish()
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 323, in publish
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     self.process_lifecycle()
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 92, in process_lifecycle
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     step.process()
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 150, in process
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     self._process_block()
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 562, in _process_block
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     self.process_unit(package_unit)
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 373, in process_unit
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     self._create_symlink(source_path, destination_path)
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:   File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 401, in _create_symlink
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR:     raise RuntimeError(msg % {'s': source_path})
May 29 09:22:28 ibm-x3250m4-04 pulp: pulp.server.managers.repo.publish:ERROR: RuntimeError: Will not create a symlink to a non-existent source [/var/lib/pulp/content/rpm/pulp-dot-2.0-test/0.1.2/1.fc11/x86_64/435d92e6c09248b501b8d2ae786f92ccfad69fab8b1bc774e2b66ff6c0d83979/pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm]

Comment 1 Chris Duryee 2014-06-04 19:50:41 UTC
This works for both Preethi and I with beta 19. Marking as ON_QA.

Comment 2 Preethi Thomas 2014-06-04 19:54:16 UTC
verified
[root@hp-dl380pgen8-01 ~]# rpm -qa pulp-server
pulp-server-2.4.0-0.19.beta.fc20.noarch
[root@hp-dl380pgen8-01 ~]# 

root@hp-dl380pgen8-01 ~]# pulp-admin rpm repo create --repo-id upload
Successfully created repository [upload]

[root@hp-dl380pgen8-01 ~]# pulp-admin rpm repo uploads rpm --repo-id upload -f pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm  -v
+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm
... completed

Files to be uploaded:
  pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm

Creating upload requests on the server...
[==================================================] 100%
Initializing: pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Uploading: pulp-dot-2.0-test-0.1.2-1.fc11.x86_64.rpm
[==================================================] 100%
2359/2359 bytes
... completed

Importing into the repository...
This command may be exited via ctrl+c without affecting the request.


[\]
Running...

Task Succeeded


Deleting the upload request...
... completed

[root@hp-dl380pgen8-01 ~]# 
[root@hp-dl380pgen8-01 ~]# 
[root@hp-dl380pgen8-01 ~]# 
[root@hp-dl380pgen8-01 ~]# pulp-admin rpm repo create --repo-id pulp --feed http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/pulp_unittest/
Successfully created repository [pulp]

[root@hp-dl380pgen8-01 ~]# pulp-admin rpm repo sync run --repo-id pulp+----------------------------------------------------------------------+
                    Synchronizing Repository [pulp]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       2/2 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 3/3 items
... completed

Importing errata...
[\]
... completed

Importing package groups/categories...
[-]
... completed


Task Succeeded



Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[==================================================] 100%
3 of 3 items
... completed

Publishing Delta RPMs
[-]
... completed

Publishing Errata
[==================================================] 100%
52 of 52 items
... completed

Publishing Comps file
[==================================================] 100%
5 of 5 items
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed


Task Succeeded

Comment 3 Randy Barlow 2014-08-09 06:56:16 UTC
This has been fixed in Pulp 2.4.0-1.