Bug 672329

Summary:	Incomplete local re-syncs when someone manually deletes a package from pulp's file system
Product:	[Retired] Pulp	Reporter:	John Matthews <jmatthew>
Component:	z_other	Assignee:	John Matthews <jmatthew>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Preethi Thomas <pthomas>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	unspecified	CC:	skarmark
Target Milestone:	---	Keywords:	Triaged
Target Release:	Sprint 20
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2011-08-16 12:09:13 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	647488

Description John Matthews 2011-01-24 20:37:08 UTC

I came across this issue while trying to force some errors.  It requires manual intervention of someone deleting packages from the filesystem, not using any API/CLI calls.

If someone deletes a package from /var/lib/pulp/packages/....../blah.rpm and then attempts to resync a local feed repository there is a problem.  The local repository says the sync completed but the file "blah.rpm" is not pulled down.

Assume that repo had 185 packages.  Sync will report 185 packages syncing, then logs will display 184 packages in repo.  No errors are reported, but the repo never pulls down the missing file.

Fix is currently to go into the repo's directory and delete the bad symbolic link for "blah.rpm" and re-sync.

We should be able to handle this from inside of pulp.

Comment 1 John Matthews 2011-02-02 20:09:25 UTC

I'm not able to reproduce this problem.

I believe my system was in an odd state.  I notice that the particular repo which had this problem had packages stored in 2 locations for each package.  Each package was stored with the path of it's prefix for a sha256sum on the package, as well as a sha1sum.  I am thinking this might be related to running with sha256 in the past and never deleting the older data.


QE:  Verify that you can manually delete a package synced from a local sync, then re-sync and verify that the package is downloaded.

Delete the package under /var/lib/pulp/packages, not the symbolic link under /var/lib/pulp/repos

I originally saw this when the link under "repos" was dead.

Comment 2 Preethi Thomas 2011-02-07 21:06:41 UTC

verified
Looks like this is working


[root@preethi ~]# rm -rf /var/lib/pulp/repos/test/pulp-cds-0.0.135-1.fc14.noarch.rpm 
[root@preethi ~]# 
[root@preethi ~]# 
[root@preethi ~]# 
[root@preethi ~]# pulp-admin repo sync --id=test -FSync for repository test started
Sync: Finished
0/11 new items downloaded
11/11 existing items processed

Item Details: 
Rpms: 11/11

[root@preethi ~]# ls  /var/lib/pulp/repos/test/
drpms                                  python-gofer-0.17-1.fc14.noarch.rpm
gofer-0.17-1.fc14.noarch.rpm           python-pymongo-1.6-5.fc14.x86_64.rpm
grinder-0.0.77-1.fc14.noarch.rpm       python-qpid-0.7.946106-1.fc14.noarch.rpm
pulp-0.0.135-1.fc14.noarch.rpm         python-webpy-0.32-6.fc14.noarch.rpm
pulp-cds-0.0.135-1.fc14.noarch.rpm     repodata
pulp-client-0.0.135-1.fc14.noarch.rpm  ruby-gofer-0.17-1.fc14.noarch.rpm
pulp-common-0.0.135-1.fc14.noarch.rpm

Comment 3 John Matthews 2011-02-08 14:20:23 UTC

Preethi,

Another test for the issue I saw was to delete the actual rpm under /var/lib/pulp/packages/....../blah.rpm

It looks like your test only deleted the symlink under the repo directory.  That wouldn't trigger the problem I saw, yet it's still a good thing to test and verify is working.

The issue this bug intended is that the actual package contents have been manually deleted, yet the repo still has a symlink to it under the repo dir.  It should be a dead symlink, when we resync we should download the new package.

Would you re-test that this scenario I described is Ok?

Comment 4 Preethi Thomas 2011-02-08 15:08:44 UTC

verifed

Thanks for explaining the test.

Here is the new result

[root@preethi ~]# rm -rf /var/lib/pulp/packages/ea5/pulp-cds/0.0.135/1.fc14/noarch/pulp-cds-0.0.135-1.fc14.noarch.rpm 

[root@preethi ~]# pulp-admin repo sync --id=test -FSync for repository test started
Sync: Finished
1/11 new items downloaded
10/11 existing items processed

Item Details: 
Rpms: 11/11

[root@preethi ~]# pulp-admin repo sync --id=mypulp -FSync for repository mypulp started
Sync: Finished
0/11 new items downloaded
11/11 existing items processed

Item Details: 
Rpms: 11/11

Comment 5 Preethi Thomas 2011-08-16 12:09:13 UTC

Closing with Community Release 15

pulp-0.0.223-4.

Comment 6 Preethi Thomas 2011-08-16 12:21:51 UTC

Closing with Community Release 15

pulp-0.0.223-4.