Bug 916308

Summary: The CDS deletes all repositories during its sync
Product: Red Hat Update Infrastructure for Cloud Providers Reporter: James Slagle <jslagle>
Component: CDSAssignee: James Slagle <jslagle>
Status: CLOSED ERRATA QA Contact: mkovacik
Severity: urgent Docs Contact:
Priority: high    
Version: 2.1CC: juwu, sclewis, vkuznets, whayutin
Target Milestone: ---   
Target Release: 2.1.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
During a synchronization, Red Hat Update Appliance returned different relative URLs than the ones in the CDS repository list and caused CDS to delete those repositories. This bug fix updates 'relative URL checking' in src/pulp/cds/cdslib.py so the correct relative URLs are returned.
 Story Points: ---
Clone Of: 836640 Environment:
Last Closed: 2013-04-23 11:41:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 836640    
Bug Blocks:    
Attachments:
Description Flags
Verification log none

Description James Slagle 2013-02-27 19:35:35 UTC 
+++ This bug was initially created as a clone of Bug #836640 +++

The bug was introduced in the fix for 800525.

Because of that change, the local file store for what repos exist on the CDS contains:

$ cat .cds_repo_list 
repos/repos/pulp/pulp/v1/stable/fedora-16/x86_64

The relative URL coming back from the server:

repos/pulp/pulp/v1/stable/fedora-16/x86_64

Since those are not the same, the CDS logic thinks the existing repo on disk was removed and deletes it.

You can see that it's happening in the log on the CDS:

2012-06-29 13:30:28,503 [INFO][worker-0] _delete_removed_repos() @ cdslib.py:400 - Removing old repo [/var/lib/pulp-cds/repos/repos/pulp/pulp/v1/stable/fedora-16/x86_64]

That was seen by synchronizing the CDS twice with no changes to its repositories in the interim.

--- Additional comment from Jay Dobies on 2012-06-29 13:54:33 EDT ---

commit 0ad6606b1d61fd41c8186bc4cd263efdeac7d6da
Author: Jay Dobies <jason.dobies>
Date:   Fri Jun 29 13:51:37 2012 -0400

    836640 - The storage of the relative URL changed so we need to update
    the relative URL checking to match that

src/pulp/cds/cdslib.py

--- Additional comment from Jay Dobies on 2012-06-29 14:24:37 EDT ---

In 1.1.11

--- Additional comment from Preethi Thomas on 2012-07-02 08:50:59 EDT ---

[root@pulp-f16 ~]# pulp-admin -u admin -p admin cds sync --hostname=f16-cds.usersys.redhat.com 
Sync for CDS [f16-cds.usersys.redhat.com] started
Use "cds status" to check on the progress


2012-07-02 08:46:52,013 [INFO][worker-0] sync() @ cdslib.py:120 - Received sync call
2012-07-02 08:46:52,014 [INFO][worker-0] sync() @ cdslib.py:121 - {'repos': [{'name': 'zoo', 'publish': True, 'relative_path': 'fakerepos/zoo', 'source': {'url': 'http://inecas.fedorapeople.org/fakerepos/zoo/', 'type': 'remote'}, '_id': 'zoo', 'arch': 'noarch', 'id': 'zoo'}, {'name': 'pulp-f17', 'publish': True, 'relative_path': 'repos/pulp/pulp/v1/stable/fedora-17/x86_64', 'source': {'url': 'http://repos.fedorapeople.org/repos/pulp/pulp/v1/stable/fedora-17/x86_64/', 'type': 'remote'}, '_id': 'pulp-f17', 'arch': 'noarch', 'id': 'pulp-f17'}, {'name': 'pulp-f16-32', 'publish': True, 'relative_path': 'repos/pulp/pulp/v1/stable/fedora-16/i386', 'source': {'url': 'http://repos.fedorapeople.org/repos/pulp/pulp/v1/stable/fedora-16/i386/', 'type': 'remote'}, '_id': 'pulp-f16-32', 'arch': 'noarch', 'id': 'pulp-f16-32'}, {'name': 'pulp', 'publish': True, 'relative_path': 'repos/pulp/pulp/v1/stable/fedora-16/x86_64', 'source': {'url': 'http://repos.fedorapeople.org/repos/pulp/pulp/v1/stable/fedora-16/x86_64/', 'type': 'remote'}, '_id': 'pulp', 'arch': 'noarch', 'id': 'pulp'}], 'repo_base_url': 'https://pulp-f16.usersys.redhat.com//pulp/repos', 'repo_cert_bundles': {'pulp-f17': None, 'pulp-f16-32': None, 'pulp': None, 'zoo': None}, 'cluster_id': None, 'cluster_members': None, 'server_ca_cert': None, 'global_cert_bundle': None}
2012-07-02 08:46:52,017 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://pulp-f16.usersys.redhat.com//pulp/repos/fakerepos/zoo]
2012-07-02 08:46:52,018 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [zoo] from [https://pulp-f16.usersys.redhat.com//pulp/repos/fakerepos/zoo] to [/var/lib/pulp-cds/repos/fakerepos/zoo]
2012-07-02 08:46:53,644 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://pulp-f16.usersys.redhat.com//pulp/repos/fakerepos/zoo]
2012-07-02 08:46:53,646 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-17/x86_64]
2012-07-02 08:46:53,646 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [pulp-f17] from [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-17/x86_64] to [/var/lib/pulp-cds/repos/repos/pulp/pulp/v1/stable/fedora-17/x86_64]
2012-07-02 08:46:55,141 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-17/x86_64]
2012-07-02 08:46:55,143 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-16/i386]
2012-07-02 08:46:55,143 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [pulp-f16-32] from [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-16/i386] to [/var/lib/pulp-cds/repos/repos/pulp/pulp/v1/stable/fedora-16/i386]
2012-07-02 08:46:56,619 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-16/i386]
2012-07-02 08:46:56,620 [INFO][worker-0] _sync_repo() @ cdslib.py:318 - Synchronizing repo at [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-16/x86_64]
2012-07-02 08:46:56,620 [INFO][worker-0] _sync_repo() @ cdslib.py:324 - Synchronizing repo [pulp] from [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-16/x86_64] to [/var/lib/pulp-cds/repos/repos/pulp/pulp/v1/stable/fedora-16/x86_64]
2012-07-02 08:46:58,204 [INFO][worker-0] _sync_repo() @ cdslib.py:361 - Successfully finished synccing [https://pulp-f16.usersys.redhat.com//pulp/repos/repos/pulp/pulp/v1/stable/fedora-16/x86_64]
2012-07-02 08:46:58,204 [INFO][worker-0] update_cluster_membership() @ cdslib.py:236 - Received cluster membership update; Cluster [None], Members []
2012-07-02 08:46:58,205 [INFO][worker-0] update_cluster_membership() @ cdslib.py:247 - No changes needed to be made to cluster memberships

--- Additional comment from Preethi Thomas on 2012-07-02 09:35:39 EDT ---

verified 

[root@pulp-f16 ~]# rpm -q pulp
pulp-1.1.11-1.fc16.noarch

--- Additional comment from Preethi Thomas on 2013-01-07 09:09:38 EST ---

Pulp 2.0 released.
Comment 1 James Slagle 2013-03-21 19:49:35 UTC 
commit 1ca7d5deff08a527c04b6dcb8f89715c992ea57e
Comment 3 Vitaly Kuznetsov 2013-04-09 11:24:14 UTC 
Created attachment 733148 [details]
Verification log

Verified with pulp-cds-0.0.263-32.el6_4.noarch
Comment 5 errata-xmlrpc 2013-04-23 11:41:40 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0767.html