Bug 854238

Summary: resyncing a repo after creating a package category for the repo fails.
Product: [Retired] Pulp Reporter: Preethi Thomas <pthomas>
Component: user-experienceAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED CURRENTRELEASE QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 2.0.6CC: syeghiay
Target Milestone: ---Keywords: Triaged
Target Release: Sprint 40   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-09 17:08:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Preethi Thomas 2012-09-04 13:06:40 UTC
Description of problem:
Repo sync fails for a feeded repo, if you resync after creating a package category, 

Version-Release number of selected component (if applicable):
[root@preethi-el6-pulp ~]# rpm -q pulp-rpm-server
pulp-rpm-server-2.0.2-1.el6.noarch


How reproducible:


Steps to Reproduce:
1. create a repo with a feed
2. sync the repo
3. create a category in the repo
4. Ryn a repo sync on the repo
  
Actual results:
[root@preethi-el6-pulp ~]# pulp-admin repo create --repo-id zoo --feed http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoo]

[root@preethi-el6-pulp ~]# pulp-admin repo sync run --repo-id zoo 
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[/]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items
Tree Files: 0/0 items
Files:      0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 0/0 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[/]
... completed

Publishing repository over HTTP
[-]
... skipped

Publishing repository over HTTPS
[-]
... completed

Successfully synchronized repository

[root@preethi-el6-pulp ~]# pulp-admin repo uploads category --repo-id zoo -i mammals --name mammals --description mammals -v
+----------------------------------------------------------------------+
                       Package Category Creation
+----------------------------------------------------------------------+

Package Category Details:
  Id:                      mammals
  Repo Id:                 zoo
  Description:             mammals
  Display Order:           0
  Name:                    mammals
  Packagegroupids:         None
  Translated Description:  
  Translated Name:         

Starting upload of selected packages. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Importing into the repository...
... completed

Deleting the upload request...
... completed

[root@preethi-el6-pulp ~]# pulp-admin repo units package-category --repo-id zoo 
Created:       2012-09-04T09:04:12
Id:            5045fc4c196f454f3600007c
Metadata:      
  Description:            None
  Display Order:          99
  Id:                     all
  Name:                   all
  Packagegroupids:        mammals, birds
  Repo Id:                zoo
  Translated Description: 
  Translated Name:        
Owner Id:      yum_importer
Owner Type:    importer
Repo Id:       zoo
Unit Id:       c1ad15f6-9733-44ff-b669-bc0f100e3de6
Unit Type Id:  package_category
Updated:       2012-09-04T09:04:12


Created:       2012-09-04T09:05:05
Id:            5045fc81196f454f3600008d
Metadata:      
  Description:            mammals
  Display Order:          0
  Id:                     mammals
  Name:                   mammals
  Packagegroupids:        None
  Repo Id:                zoo
  Translated Description: 
  Translated Name:        
Owner Id:      SYSTEM
Owner Type:    user
Repo Id:       zoo
Unit Id:       e952d672-ca9d-4203-bc04-f2f5725ce08a
Unit Type Id:  package_category
Updated:       2012-09-04T09:05:05


[root@preethi-el6-pulp ~]# pulp-admin repo sync run --repo-id zoo +----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items
Tree Files: 0/0 items
Files:      0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

[-]
Waiting to begin
Publishing packages...
[==================================================] 100%
Packages: 0/0 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Error during repository synchronization

[u'PulpExecutionException: Pulp exception occurred: PulpExecutionException\n']



Expected results:


Additional info:


pulp.log

2012-09-04 09:05:24,700 20278:140587751667456: pulp.server.managers.repo.publish:ERROR: publish:153 Exception caught from plugin during publish for repo [zoo]
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 139, in _do_publish
    publish_report = distributor_instance.publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/pulp/plugins/distributors/yum_distributor/distributor.py", line 436, in publish_repo
    groups_xml_path = comps_util.write_comps_xml(repo, existing_groups, existing_cats)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 399, in write_comps_xml
    comps_xml = form_comps_xml_from_units(existing_groups, existing_cats)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 176, in form_comps_xml_from_units
    yum_categories = map(unit_to_yum_category, categories)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 134, in unit_to_yum_category
    return dict_to_yum_category(cat_unit.metadata, category_id=cat_unit.unit_key["id"])
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 159, in dict_to_yum_category
    for groupid in obj['packagegroupids']:
TypeError: 'NoneType' object is not iterable
2012-09-04 09:05:24,701 20278:140587751667456: pulp.server.dispatch.task:ERROR: task:133 Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 130, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 127, in publish
    result = self._do_publish(repo, distributor_id, distributor_instance, transfer_repo, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/publish.py", line 139, in _do_publish
    publish_report = distributor_instance.publish_repo(transfer_repo, conduit, call_config)
  File "/usr/lib/pulp/plugins/distributors/yum_distributor/distributor.py", line 436, in publish_repo
    groups_xml_path = comps_util.write_comps_xml(repo, existing_groups, existing_cats)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 399, in write_comps_xml
    comps_xml = form_comps_xml_from_units(existing_groups, existing_cats)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 176, in form_comps_xml_from_units
    yum_categories = map(unit_to_yum_category, categories)
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 134, in unit_to_yum_category
    return dict_to_yum_category(cat_unit.metadata, category_id=cat_unit.unit_key["id"])
  File "/usr/lib/python2.6/site-packages/pulp_rpm/yum_plugin/comps_util.py", line 159, in dict_to_yum_category
    for groupid in obj['packagegroupids']:
PulpExecutionException: Pulp exception occurred: PulpExecutionException
2012-09-04 09:05:24,702 20278:140587751667456: pulp.server.dispatch.task:INFO: task:163 Task 2ea3108a-f691-11e1-96c7-525400c5886c: CallRequest: RepoPublishManager.publish(u'zoo', u'yum_distributor', distributor_instance=<yum_distributor.distributor.YumDistributor object at 0x7fdd5b491790>, distributor_config={}) FAILED

Comment 1 Pradeep Kilambi 2012-09-12 15:58:53 UTC
fixed! commit 68b6fcf2ef27ac73c641e914c7a779baf489f4d8

Comment 2 Jeff Ortel 2012-09-21 18:12:57 UTC
build: 0.328

Comment 3 Preethi Thomas 2012-09-26 19:29:11 UTC
verified
[root@pulp-master ~]# rpm -q pulp-rpm-server
pulp-rpm-server-0.0.328-1.fc17.noarch
[root@pulp-master ~]# 


[root@pulp-master ~]# pulp-admin repo create --repo-id zoo --feed http://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/zoo/
Successfully created repository [zoo]

[root@pulp-master ~]# pulp-admin repo sync run --repo-id zoo 
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[-]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       32/32 items
Delta RPMs: 0/0 items
Tree Files: 0/0 items
Files:      0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

Publishing packages...
[==================================================] 100%
Packages: 0/0 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[|]
... completed

Publishing repository over HTTP
[-]
... skipped

Publishing repository over HTTPS
[-]
... completed

Successfully synchronized repository

[root@pulp-master ~]# pulp-admin repo uploads category --repo-id zoo -i mammals --name mammals --description mammals -v
+----------------------------------------------------------------------+
                          Puppet Module Upload
+----------------------------------------------------------------------+

Extracting necessary metdata for each request...
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing upload
... completed

Starting upload of selected packages. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Importing into the repository...
... completed

Deleting the upload request...
... completed

[root@pulp-master ~]# pulp-admin repo units package-category --repo-id zoo 
Description:             None
Display Order:           99
Id:                      all
Name:                    all
Packagegroupids:         mammals, birds
Repo Id:                 zoo
Translated Description:  
Translated Name:         


Description:             mammals
Display Order:           0
Id:                      mammals
Name:                    mammals
Packagegroupids:         None
Repo Id:                 zoo
Translated Description:  
Translated Name:         


[root@pulp-master ~]# pulp-admin repo units package-category --repo-id zoo  --details
Created:       2012-09-26T23:07:18Z
Id:            50635266edfc06774d000232
Metadata:      
  Description:            None
  Display Order:          99
  Id:                     all
  Name:                   all
  Packagegroupids:        mammals, birds
  Repo Id:                zoo
  Translated Description: 
  Translated Name:        
Owner Id:      yum_importer
Owner Type:    importer
Repo Id:       zoo
Unit Id:       a43c233c-0459-40d0-9819-52e712775cbd
Unit Type Id:  package_category
Updated:       2012-09-26T23:07:18Z


Created:       2012-09-26T23:26:15Z
Id:            506356d7edfc06774d000247
Metadata:      
  Description:            mammals
  Display Order:          0
  Id:                     mammals
  Name:                   mammals
  Packagegroupids:        None
  Repo Id:                zoo
  Translated Description: 
  Translated Name:        
Owner Id:      admin
Owner Type:    user
Repo Id:       zoo
Unit Id:       14089e70-e353-4c42-9b20-826380d38724
Unit Type Id:  package_category
Updated:       2012-09-26T23:26:15Z


[root@pulp-master ~]# pulp-admin repo sync run --repo-id zoo 
+----------------------------------------------------------------------+
                     Synchronizing Repository [zoo]
+----------------------------------------------------------------------+

This command may be exited by pressing ctrl+c without affecting the actual
operation on the server.

Downloading metadata...
[|]
... completed

Downloading repository content...
[==================================================] 100%
RPMs:       0/0 items
Delta RPMs: 0/0 items
Tree Files: 0/0 items
Files:      0/0 items
... completed

Importing errata...
[-]
... completed

Importing package groups/categories...
[-]
... completed

[-]
Waiting to begin
Publishing packages...
[==================================================] 100%
Packages: 0/0 items
... completed

Publishing distributions...
[==================================================] 100%
Distributions: 0/0 items
... completed

Generating metadata
[\]
... completed

Publishing repository over HTTP
[-]
... skipped

Publishing repository over HTTPS
[-]
... completed

Successfully synchronized repository

[root@pulp-master ~]#

Comment 4 Preethi Thomas 2013-01-09 17:08:05 UTC
Pulp v2.0 released