Bug 730752

Summary: Cannot change feed URL of repo in most circumstances
Product: [Retired] Pulp Reporter: Chris St. Pierre <cstpierr>
Component: user-experienceAssignee: Sayli Karmarkar <skarmark>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: cperry, jason.dobies, jokajak, jperrin, pkilambi, ptr
Target Milestone: ---Keywords: Reopened, Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-09 17:09:42 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Chris St. Pierre 2011-08-15 16:13:40 UTC
Description of problem:

When updating the feed URL of a repository, it is required that the edited repo's relative path remain the same.  The problem is that there is no --relativepath argument to "repo update", so there's no way to work around this if the feed path actually changed, or if you set a relative path explicitly on repo creation.

Version-Release number of selected component (if applicable):

Pulp 0.0.223

How reproducible:

Always.

Steps to Reproduce:

1.  Create a repository with --relativepath:

% pulp-admin repo create --id=rhel-6-x86_64-pulp --name="Pulp - RHEL 6 x86_64" --arch=x86_64 --feed=http://repos.fedorapeople.org/repos/pulp/pulp/testing/6Server/x86_64/ --relativepath=rhel-6-x86_64-pulp

2.  Attempt to change the feed URL:

% pulp-admin repo update --id=generic-6-x86_64-pulp --feed=http://repos.fedorapeople.org/repos/pulp/pulp/6Server/x86_64/
error: operation failed: PulpException: u'Relativepath of the new feed [repos/pulp/pulp/5Server/i386] does not match existing feed [generic-5-i386-pulp]; cannot perform update'

3.  Realize that you apparently have to use the --relativepath argument:

% pulp-admin repo update --id=generic-6-x86_64-pulp --feed=http://repos.fedorapeople.org/repos/pulp/pulp/6Server/x86_64/ --relativepath=rhel-6-x86_64-pulp
Usage: pulp-admin <options> repo update <options>

pulp-admin: error: no such option: --relativepath

Actual results:

Errors.

Expected results:

One way or another, the feed URL should be changeable without deleting the repo and re-adding it.

Additional info:

Comment 1 Sayli Karmarkar 2011-08-17 15:47:53 UTC
We purposely remove relativepath update from repo update. So this is intentional. Prad will know a bit more about that.

Comment 2 Pradeep Kilambi 2011-08-17 17:09:56 UTC
This is behaving as designed. You can change the feed url only if the relative path of the new url matches the existing feed's relative path.  If we allow users to update the relative path, then the content itself needs to be moved and in your case, you'll be left with 32 and 64 bit packages with 32 bit packages orphaned. This is not supported. If you really want to chance the feed in your situation, you'll have to create a new repo. Closing this as not an issue.

Comment 3 Chris St. Pierre 2011-08-17 17:41:41 UTC
I must not have explained myself well.

If you create a repo with an explicit relative path, you cannot change the feed URL ever, even though the relative path _should not_ change.  Consider the circumstances in my first post -- the repo was created with --relativepath=rhel-6-x86_64-pulp.  But since I cannot give --relativepath to update, I cannot change the feed URL -- it wants to update the relative path, too.

What's more, if you create a repo without a relative path, you cannot subsequently change the URL, even if you want to keep the relative path the same.  What's the harm in doing something like this:

pulp-admin repo create --id=rhel-6-x86_64-pulp --name="Pulp - RHEL 6 x86_64"
--arch=x86_64
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/testing/6Server/x86_64/

At this point, the relative path is repos/pulp/pulp/testing/6Server/x86_64.  But we can't change the feed URL, even if we're fine with the relative path staying the same.  I.e., this is not legal:

pulp-admin repo update --id=generic-6-x86_64-pulp
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/6Server/x86_64/
--relativepath=repos/pulp/pulp/testing/6Server/x86_64

I do not want to change the relative path of a repo.  I want the relative path to stay the same, but repo update _insists_ on changing it, even if I gave the repo an explicit relative path on creation.

Comment 4 Pradeep Kilambi 2011-08-17 18:14:23 UTC
Ah there are two issues here. Your first case makes sense. I think that is indeed a bug. If you have a --relativepath set manually, we still should use the relativepath of the feed to compare. If not, you cant update the hostname even if the relativepath is same.

The second case is what I was explaining, pulp only allows update if the relativepath is same as you used when the repo was created. The relativepath option was primarily removed to not allow the case you just mentioned. The only thing we allow an update for is the hostname of the feed itself. The relativepath restriction is not from pulp's pov, its the source's relative path.

Supported:
old feed: https://foo.example.com/a/b/c
new feed: https://bar.example.com/a/b/c

Not Supported:
old feed: https://foo.example.com/a/b/c
new feed: https://bar.example.com/a/b/d
or
new feed: https://foo.example.com/a/b/d

in your example:

you can update the created repo with:

pulp-admin repo update --id=generic-6-x86_64-pulp
--feed=http://repos.pulp.org/repos/pulp/pulp/6Server/x86_64/

where the hostname itself changed but not the relative path.

We'll fix your first issue, to not rely on what you pass in for relativepath but compare the feed's relativepath to check for update on feed. The second case is behaving as intended if you dont use --relativepath.

Comment 5 Pradeep Kilambi 2011-08-17 18:29:35 UTC
fixed!091175561bf0eb641aabdb102f1cd845ca5bce6c


$ sudo pulp-admin repo create --id=foo2 --feed=http://foo.example.com/a/b/c/d --relativepath=/mypath/
Successfully created repository [ foo2 ]

$ sudo pulp-admin repo update --id=foo2 --feed=http://bar.example.com/a/b/c/d
Successfully updated repository [ foo2 ]


$ sudo pulp-admin repo update --id=foo2 --feed=http://bar.example.com/a/b/c/d/eerror: operation failed: PulpException: u'Relativepath of the new feed [a/b/c/d/e] does not match existing feed [a/b/c/d]; cannot perform update'

Comment 6 Jeff Ortel 2011-08-17 20:45:13 UTC
build: 0.225

Comment 7 Preethi Thomas 2011-08-18 18:16:32 UTC
verified
[root@preethi ~]# rpm -q pulp
pulp-0.0.225-1.fc14.noarch


[root@preethi ~]# pulp-admin repo create --id=foo2 --feed=http://foo.example.com/a/b/c/d --relativepath=/mypath/
Successfully created repository [ foo2 ]

[root@preethi ~]# pulp-admin repo update --id=foo2 --feed=http://bar.example.com/a/b/c/d --relativepath=/mypath/
Usage: pulp-admin <options> repo update <options>

pulp-admin: error: no such option: --relativepath
[root@preethi ~]# pulp-admin repo update --id=foo2 --feed=http://bar.example.com/a/b/c/d 
Successfully updated repository [ foo2 ]

[root@preethi ~]# 
[root@preethi ~]# 
[root@preethi ~]# 
[root@preethi ~]# pulp-admin repo update --id=foo2 --feed=http://bar.example.com/a/b/c/d/error
error: operation failed: PulpException: u'Relativepath of the new feed [a/b/c/d/error] does not match existing feed [a/b/c/d]; cannot perform update'
[root@preethi ~]#

Comment 8 Chris St. Pierre 2012-02-16 21:11:48 UTC
This only fixed a subset of problem sets, which I should have noticed from the testing Preethi did.  For instance, I created a local Pulp repo:

% pulp-admin repo create --id=rhel-6-x86_64-pulp --name="Pulp - RHEL 6 x86_64"
--arch=x86_64
--feed=http://repos.fedorapeople.org/repos/pulp/pulp/testing/6Server/x86_64/
--relativepath=rhel-6-x86_64-pulp

But now the repo layout has changed, so I want to change the feed.  Note that I do _not_ want the path to change; I most assuredly want the relativepath to stay exactly the same.  But alas, I cannot change the feed URL:

% pulp-admin repo update --id=generic-6-x86_64-pulp --feed=http://repos.fedorapeople.org/repos/pulp/pulp/dev/stable/6Server/x86_64/ 
error:  error: operation failed: PulpException: Relativepath of the new feed [repos/pulp/pulp/dev/stable/6Server/x86_64] does not match existing feed [repos/pulp/pulp/6Server/x86_64]; cannot perform update

It would be really really really nice to be able to change the feed URL *without changing the relative path*.  It would be particularly nice because the Pulp repo itself is the use case for this.  I don't want to have to delete all of my mirrors of the Pulp repos and re-create them when I should be able to just update the feed URL.

Comment 9 Jim Perrin 2012-05-02 17:45:48 UTC
I'm starting to run into this issue as well. It would be great to see this resolved. 

+1 if that matters.

Comment 10 Josh 2012-05-11 12:54:48 UTC
I just ran in to this myself and had to delete and re-add.  My exact steps were:

$ pulp-admin repo create --id epel-6-x86_64 --feed https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=
$ echo # forgot to quote the url, woops
$ pulp-admin repo update --id=epel-6-x86_64 --feed="https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=x86_64"
$ pulp-admin repo sync --id=epel-6-x86_64 -F
Sync for repository epel-6-x86_64 started
Sync: Error

Item Details: 
error:  Exception: Traceback (most recent call last):

  File "/usr/lib/python2.6/site-packages/grinder/activeobject.py", line 429, in process
    retval = method(*args, **kwargs)

  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 100, in getDownloadItems
    rpms = self.__getRPMs(newest, remove_old, numOldPackages)

  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 197, in __getRPMs
    pkglist = self.__getPackageList(newest)

  File "/usr/lib/python2.6/site-packages/grinder/YumInfo.py", line 141, in __getPackageList
    sack.populate(self.repo, 'metadata', None, 0)

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 183, in populate
    xml = repo_get_function()

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1634, in getPrimaryXML
    return self.retrieveMD('primary')

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1559, in retrieveMD
    return self._retrieveMD(mdtype)

  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1567, in _retrieveMD
    thisdata = self.repoXML.getData(mdtype)

  File "/usr/lib/python2.6/site-packages/yum/repoMDObject.py", line 180, in getData
    raise RepoMDError, "requested datatype %s not available" % type

RepoMDError: requested datatype primary not available



$ pulp-admin repo update --id=epel-6-x86_64 --feed="http://www.gtlib.gatech.edu/pub/fedora-epel/6/x86_64/"
error:  operation failed: PulpException: Relativepath of the new feed [pub/fedora-epel/6/x86_64] does not match existing feed [metalink]; cannot perform update


+1

Comment 11 Patrick Taylor Ramsey 2012-06-07 06:49:09 UTC
Running into this as well.  +1

Comment 12 Jay Dobies 2012-07-11 20:32:27 UTC
This should be fixed in v2. There are no restrictions on changing the feed URL of an existing repository.

Comment 13 Preethi Thomas 2012-07-12 14:16:10 UTC
 feed url can be updated in v2 moving to verified

[root@preethi-el6-pulp ~]# rpm -qa |grep pulp-rpm
pulp-rpm-admin-client-0.0.312-4.el6.noarch
pulp-rpm-server-0.0.312-4.el6.noarch

[root@preethi-el6-pulp ~]# pulp-admin repo create --repo relpath-test --feed=http://repos.fedorapeople.org/repos/pulp/pulp/v1/testing/6Server/x86_64/ --relative-path=/tmp/relpath-test1


root@preethi-el6-pulp ~]# pulp-admin repo update --repo relpath-test --feed=http://repos.fedorapeople.org/repos/pulp/pulp/v1/testing/6Server/i386/ --relative-url=/tmp/relpath-test1
Repository [relpath-test] successfully updated

[root@preethi-el6-pulp ~]# pulp-admin repo update --repo relpath-test --feed=http://repos.fedorapeople.org/repos/pulp/pulp/v1/testing/fedora-17/i386/ --relative-url=/tmp/relpath-test1
Repository [relpath-test] successfully updated

[root@preethi-el6-pulp ~]# pulp-admin repo sync run --repo-id relpath-test+----------------------------------------------------------------------+
                Synchronizing Repository [relpath-test]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       20/20 items
Delta RPMs: 0/0 items
Tree Files: 0/0 items
Files:      0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 0/0 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[-]
... completed

Publishing repository over HTTP
[-]
... skipped

Publishing repository over HTTPS
[-]
... completed

Successfully synchronized repository



[root@preethi-el6-pulp ~]# pulp-admin repo create  --repo-id=generic-6-x86_64-pulp --feed http://repos.fedorapeople.org/repos/pulp/pulp/v1/testing/fedora-17/x86_64/
Successfully created repository [generic-6-x86_64-pulp]

[root@preethi-el6-pulp ~]# pulp-admin repo update  --repo-id=generic-6-x86_64-pulp --feed http://repos.fedorapeople.org/repos/pulp/pulp/v1/stable/fedora-17/x86_64/
Repository [generic-6-x86_64-pulp] successfully updated

[root@preethi-el6-pulp ~]# pulp-admin repo create --repo-id epel-6-x86_64 --feed https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=


[root@preethi-el6-pulp ~]# Successfully created repository [epel-6-x86_64]


[1]+  Done                    pulp-admin repo create --repo-id epel-6-x86_64 --feed https://mirrors.fedoraproject.org/metalink?repo=epel-6
[root@preethi-el6-pulp ~]# 
[root@preethi-el6-pulp ~]# 
[root@preethi-el6-pulp ~]# pulp-admin repo update --repo-id epel-6-x86_64 --feed="https://mirrors.fedoraproject.org/metalink?repo=epel-6&arch=x86_64"
Repository [epel-6-x86_64] successfully updated

[root@preethi-el6-pulp ~]# pulp-admin repo sync run --repo-id epel-6-x86_64
+----------------------------------------------------------------------+
                Synchronizing Repository [epel-6-x86_64]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... failed

Error during repository synchronization

[u'PulpExecutionException: Importer indicated a failed response\n']



[root@preethi-el6-pulp ~]# pulp-admin repo update --repo-id epel-6-x86_64 --feed="http://www.gtlib.gatech.edu/pub/fedora-epel/6/x86_64/"
Repository [epel-6-x86_64] successfully updated

[root@preethi-el6-pulp ~]# pulp-admin repo sync run --repo-id epel-6-x86_64+----------------------------------------------------------------------+
                Synchronizing Repository [epel-6-x86_64]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... completed

Downloading repository content...
[====                                              ] 8%
RPMs:       874/7630 items
Delta RPMs: 0/0 items
Tree Files: 0/0 items
Files:      0/0 items
^C[root@preethi-el6-pulp ~]# pulp-admin repo sync
Usage: pulp-admin sync [SUB_SECTION, ..] COMMAND
Description: run, schedule, or view the status of sync tasks

Available Sections:
  schedules - manage sync schedules for a repository

Available Commands:
  run    - triggers an immediate sync of a repository
  status - displays the status of a repository's sync tasks
[root@preethi-el6-pulp ~]# pulp-admin repo tasks
Usage: pulp-admin tasks [SUB_SECTION, ..] COMMAND
Description: list and cancel tasks related to a specific repository

Available Commands:
  cancel  - cancel one or more tasks
  details - displays more detailed information about a specific task
  list    - lists tasks queued or running in the server
[root@preethi-el6-pulp ~]# pulp-admin repo tasks list
Command: list
Description: lists tasks queued or running in the server

Available Arguments:

  --repo-id - (required) identifies the repository to display

Comment 14 Preethi Thomas 2013-01-09 17:09:42 UTC
Pulp v2.0 released