Bug 974235 - repo copy with --recursive fails for group copy if you run rpm copy --recursive first
repo copy with --recursive fails for group copy if you run rpm copy --recursi...
Status: CLOSED CURRENTRELEASE
Product: Pulp
Classification: Community
Component: user-experience (Show other bugs)
2.2 Beta
Unspecified Unspecified
medium Severity medium
: ---
: 2.3.0
Assigned To: Barnaby Court
Preethi Thomas
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-06-13 14:34 EDT by Preethi Thomas
Modified: 2013-12-09 09:31 EST (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-12-09 09:31:36 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Preethi Thomas 2013-06-13 14:34:46 EDT
Description of problem:

If you run rpm copy rpm --recursive first on an rpm and then run rpm copy group --recursive on a group that contains the rpm it blows up

Version-Release number of selected component (if applicable):

[root@hp-dl360g6-01 ~]# rpm -q pulp-server
pulp-server-2.2.0-0.3.beta.fc18.noarch
[root@hp-dl360g6-01 ~]# 

How reproducible:


Steps to Reproduce:
[root@hp-dl360g6-01 ~]# pulp-admin rpm repo create --repo-id pulp-copySuccessfully created repository [pulp-copy]

[root@hp-dl360g6-01 ~]# pulp-admin rpm repo copy rpm  --from-repo-id pulp --to-repo-id pulp-copy --str-eq "name=pulp-server" --recursive
This command may be exited via ctrl+c without affecting the request.

[-]
Waiting to begin...

[-]
Running...

Units:
  pulp-server-2.2.0-0.3.beta.fc18-noarch

[root@hp-dl360g6-01 ~]# pulp-admin rpm repo copy group --from-repo-id pulp --to-repo-id pulp-copy --str-eq "id=pulp-server" --recursive 
This command may be exited via ctrl+c without affecting the request.

[-]
Waiting to begin...

[-]
Running...

Task Failed

[u'PulpExecutionException: Pulp exception occurred: PulpExecutionException\n']


Actual results:


Expected results:


Additional info:


2013-06-13 14:33:30,313 pulp.server.webservices.controllers.repositories:INFO: Itinerary: ['c6351969-8da4-49e2-aec1-4b2c5abb1aab']
2013-06-13 14:33:30,389 pulp.server.dispatch.task:INFO: SUCCESS: Task c6351969-8da4-49e2-aec1-4b2c5abb1aab: CallRequest: RepoManager.delete_repo(u'pulp-copy')
2013-06-13 14:33:37,898 pulp.server.dispatch.task:INFO: SUCCESS: Task 57738b5a-5140-49b3-8ab9-88e25c1a8ff3: CallRequest: RepoManager.create_and_configure_repo('pulp-copy', None, None, {'_repo-type': 'rpm-repo'}, importer_repo_plugin_config=****, distributor_list=****, importer_type_id='yum_importer')
2013-06-13 14:33:53,326 pulp.server.dispatch.task:INFO: SUCCESS: Task 27447159-bbcd-4bc8-baf4-9059469cef1d: CallRequest: RepoUnitAssociationManager.associate_from_repo('pulp', u'pulp-copy', import_config_override=****, criteria=****)
2013-06-13 14:34:02,598 pulp.plugins.conduits.mixins:ERROR: Exception from server requesting all content units for repository [pulp-copy]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/mixins.py", line 617, in do_get_repo_units
    u = common_utils.to_plugin_associated_unit(unit, type_defs[type_id])
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/_common.py", line 92, in to_plugin_associated_unit
    unit_key[k] = pulp_unit['metadata'].pop(k)
KeyError: u'epoch'
2013-06-13 14:34:02,599 pulp.server.managers.repo.unit_association:ERROR: Exception from importer [yum_importer] while importing units into repository [pulp-copy]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 276, in associate_from_repo
    call_config, units=transfer_units)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 68, in import_units
    return associate.associate(source_repo, dest_repo, import_conduit, config, units)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 84, in associate
    names_to_copy = get_rpms_to_copy_by_name(rpm_names, import_conduit)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 141, in get_rpms_to_copy_by_name
    for unit in units:
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/existing.py", line 65, in get_existing_units
    for result in search_method(criteria):
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/unit_import.py", line 145, in get_destination_units
    return mixins.do_get_repo_units(self.dest_repo_id, criteria, ImporterConduitException)
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/mixins.py", line 617, in do_get_repo_units
    u = common_utils.to_plugin_associated_unit(unit, type_defs[type_id])
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/_common.py", line 92, in to_plugin_associated_unit
    unit_key[k] = pulp_unit['metadata'].pop(k)
ImporterConduitException: u'epoch'
2013-06-13 14:34:02,599 pulp.server.dispatch.task:ERROR: Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association.py", line 276, in associate_from_repo
    call_config, units=transfer_units)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/importer.py", line 68, in import_units
    return associate.associate(source_repo, dest_repo, import_conduit, config, units)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 84, in associate
    names_to_copy = get_rpms_to_copy_by_name(rpm_names, import_conduit)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/associate.py", line 141, in get_rpms_to_copy_by_name
    for unit in units:
  File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/existing.py", line 65, in get_existing_units
    for result in search_method(criteria):
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/unit_import.py", line 145, in get_destination_units
    return mixins.do_get_repo_units(self.dest_repo_id, criteria, ImporterConduitException)
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/mixins.py", line 617, in do_get_repo_units
    u = common_utils.to_plugin_associated_unit(unit, type_defs[type_id])
  File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/_common.py", line 92, in to_plugin_associated_unit
    unit_key[k] = pulp_unit['metadata'].pop(k)
PulpExecutionException: Pulp exception occurred: PulpExecutionException
2013-06-13 14:34:02,600 pulp.server.dispatch.task:INFO: FAILURE: Task 54cf6b3f-3d2b-43ea-ac0a-8e57c27da880: CallRequest: RepoUnitAssociationManager.associate_from_repo('pulp', u'pulp-copy', import_config_override=****, criteria=****)
Comment 1 Barnaby Court 2013-09-11 09:21:16 EDT
Preethi, I tested this using both pulp-2.2 and master branches with the following script and was unable to duplicate the issue.  Can you verify?

pulp-admin rpm repo create --repo-id pulp --feed http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-19/x86_64/
pulp-admin rpm repo sync run --repo-id pulp
pulp-admin rpm repo create --repo-id pulp-copy 
pulp-admin rpm repo copy rpm --from-repo-id pulp --to-repo-id pulp-copy --str-eq "name=pulp-server" --recursive
pulp-admin rpm repo copy group --from-repo-id pulp --to-repo-id pulp-copy --str-eq "id=pulp-server" --recursive 



:: Results of Script ::

[bcourt@bcourt ci]$ pulp-admin rpm repo create --repo-id pulp --feed http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-19/x86_64/
Successfully created repository [pulp]

[bcourt@bcourt ci]$ pulp-admin rpm repo sync run --repo-id pulp
+----------------------------------------------------------------------+
                    Synchronizing Repository [pulp]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       42/42 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 42/42 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTPS
[-]
... completed

[bcourt@bcourt ci]$ pulp-admin rpm repo create --repo-id pulp-copy 
Successfully created repository [pulp-copy]

[bcourt@bcourt ci]$ pulp-admin rpm repo copy rpm --from-repo-id pulp --to-repo-id pulp-copy --str-eq "name=pulp-server" --recursive
This command may be exited via ctrl+c without affecting the request.

[-]
Waiting to begin...

[-]
Running...

Units:
  pulp-server-2.3.0-0.8.alpha.fc19-noarch

[bcourt@bcourt ci]$ pulp-admin rpm repo copy group --from-repo-id pulp --to-repo-id pulp-copy --str-eq "id=pulp-server" --recursive
This command may be exited via ctrl+c without affecting the request.

[-]
Waiting to begin...

[-]
Running...

Units:
  pulp-server

[bcourt@bcourt ci]$
Comment 2 Jeff Ortel 2013-09-12 18:35:58 EDT
build: 2.3.0-0.10.alpha
Comment 3 Preethi Thomas 2013-09-19 16:14:01 EDT
verified that this has been fixed
[root@pulp-v2-server ~]# rpm -qa pulp-server
pulp-server-2.3.0-0.14.alpha.el6.noarch
[root@pulp-v2-server ~]# 

[root@pulp-v2-server ~]# pulp-admin rpm repo create --repo-id pulp --feed http://repos.fedorapeople.org/repos/pulp/pulp/v2/testing/fedora-19/x86_64/
Successfully created repository [pulp]

[root@pulp-v2-server ~]# pulp-admin rpm repo sync --repo-id pulp
Usage: pulp-admin [SUB_SECTION, ..] COMMAND
Description: run, schedule, or view the status of sync tasks

Available Sections:
  schedules - manage repository sync schedules

Available Commands:
  run    - triggers an immediate sync of a repository
  status - displays the status of a repository's sync tasks
[root@pulp-v2-server ~]# pulp-admin rpm repo sync run --repo-id pulp
+----------------------------------------------------------------------+
                    Synchronizing Repository [pulp]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[\]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       42/42 items
Delta RPMs: 0/0 items

... completed

Downloading distribution files...
[==================================================] 100%
Distributions: 0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 42/42 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTPS
[-]
... completed

[root@pulp-v2-server ~]# pulp-admin rpm repo create --repo-id pulp-copy
Successfully created repository [pulp-copy]

[root@pulp-v2-server ~]#  pulp-admin rpm repo copy rpm  --from-repo-id pulp --to-repo-id pulp-copy --str-eq "name=pulp-server" --recursive
This command may be exited via ctrl+c without affecting the request.

[-]
Waiting to begin...

[-]
Running...

Units:
  m2crypto-0.21.1.pulp-8.fc19-x86_64
  mod_wsgi-3.4-1.pulp.fc19-x86_64
  pulp-server-2.3.0-0.14.alpha.fc19-noarch
  python-gofer-0.76-1.fc19-noarch
  python-isodate-0.5.0-1.pulp.fc19-noarch
  python-nectar-1.1.1-1.fc19-noarch
  python-oauth2-1.5.170-3.pulp.fc19-noarch
  python-okaara-1.0.32-1.fc19-noarch
  python-qpid-0.7.946106-2.fc19-noarch
  python-webpy-0.32-9.fc19-noarch

[root@pulp-v2-server ~]# pulp-admin rpm repo copy group --from-repo-id pulp --to-repo-id pulp-copy --str-eq "id=pulp-server" --recursive 
This command may be exited via ctrl+c without affecting the request.

[-]
Waiting to begin...

[-]
Running...

Units:
 package_group:
  pulp-server
 rpm:
  createrepo-0.9.9-21.2.pulp.fc19-noarch
  pulp-puppet-plugins-2.3.0-0.14.alpha.fc19-noarch
  pulp-rpm-plugins-2.3.0-0.14.alpha.fc19-noarch
  pulp-selinux-2.3.0-0.14.alpha.fc19-noarch
  python-rhsm-1.8.0-1.pulp.fc19-x86_64
  python-semantic-version-2.2.0-3.fc19-noarch

[root@pulp-v2-server ~]#
Comment 4 Preethi Thomas 2013-12-09 09:31:36 EST
Pulp 2.3 released.

Note You need to log in before you can comment on or make changes to this bug.