Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 843800 (katello-dotpackage)

Summary: Syncronization raises an exception when package have a different name structure
Product: Red Hat Satellite Reporter: Marcelo Moreira de Mello <mmello>
Component: InstallationAssignee: Lukas Zapletal <lzap>
Status: CLOSED CURRENTRELEASE QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: high    
Version: NightlyCC: bkearney, dgoodwin, dkaylor, jmontleo, mmccune, mmello, xdmoon
Target Milestone: UnspecifiedKeywords: Regression, Triaged
Target Release: Unused   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 853356 (view as bug list) Environment:
Last Closed: 2014-01-21 19:54:41 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 853356    

Description Marcelo Moreira de Mello 2012-07-27 11:54:42 UTC 
Description of problem:

Syncronization raises an exception when package have a different name structure. 


After entering the repository for F16 (http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/), when finishing the sync process, it raises the exception below:

==> pulp/grinder.log <==
2012-07-27 03:08:16,728 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:138 1 threads are active. 1 items left to be fetched
2012-07-27 03:08:16,728 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:329 WorkerThread deleting ActiveObject
2012-07-27 03:08:16,732 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:344 Thread ending
2012-07-27 03:08:17,219 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:234 All threads have finished.
2012-07-27 03:08:17,270 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:253 ParallelFetch: 3113 items successfully processed, 3113 downloaded, 0 items had errors
2012-07-27 03:08:17,271 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:258 Transferred [564667992] bytes of [tree_file]
2012-07-27 03:08:17,271 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:258 Transferred [3455015673] bytes of [rpm]
2012-07-27 03:08:17,272 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:259 Transferred [4019683665] total bytes in 2242.64618111 seconds
2012-07-27 03:08:17,272 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:246 Finalizing metadata, moving /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr//repodata.new to /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr//repodata
2012-07-27 03:08:17,392 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:182 Cleaning any orphaned packages..
2012-07-27 03:08:18,491 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:192 Processed <>,<http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/> with <0> items in [2223] seconds. Report: 3113 successes, 3113 downloads, 0 errors
2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:203 Grinder stopping
2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked
2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed
2012-07-27 03:08:18,493 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-20, stopped daemon 140280267269888)>> to stop
2012-07-27 03:08:18,493 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked
2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed
2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-21, stopped daemon 140280275662592)>> to stop
2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked
2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed
2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-22, stopped daemon 140280250484480)>> to stop
2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked
2012-07-27 03:08:18,496 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed
2012-07-27 03:08:18,496 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-23, stopped daemon 140280242091776)>> to stop

==> pulp/pulp.log <==
2012-07-27 03:08:18,497 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:793 YumSynchronizer reported 3113 successes, 3113 downloads, 0 errors
2012-07-27 03:10:08,403 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:856 checksum type for repo id mmello_ORG-Fedora_16_x86_64-fedora_ufpr is sha256
2012-07-27 03:10:08,413 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:839 Running createrepo, this may take a few minutes to complete.
2012-07-27 03:10:08,417 1020:140280258877184: pulp.server.util:INFO: util:507 metadata found; taking backup.
2012-07-27 03:10:09,050 1020:140280258877184: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml', '--update', '/var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr']
2012-07-27 03:15:08,206 1020:140280258877184: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr finished
2012-07-27 03:15:08,226 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:843 Createrepo finished in 299.809221983 seconds
2012-07-27 03:15:08,307 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:477 Created a distributionID ks-Fedora-Fedora-16-x86_64
2012-07-27 03:15:08,331 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:422 No metadata for 'File Sync' present; no files to import to repo..
2012-07-27 03:15:08,340 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:385 Loading comps group info from: /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml
2012-07-27 03:15:08,704 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:412 Skipping errata imports from sync process
2012-07-27 03:15:08,981 1020:140280258877184: pulp.server.tasking.task:ERROR: task:472 Task failed: Task 3c3fe154-d7ac-11e1-83db-525400623320: _sync(mmello_ORG-Fedora_16_x86_64-fedora_ufpr, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f95a4041650>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f9584023110>>)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/tasking/task.py", line 418, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content
    added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/synchronizers.py", line 415, in import_metadata
    self.repo_api.collection.save(repo, safe=True)
  File "/usr/lib/python2.7/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 237, in save
    manipulate, safe, _check_keys=True, **kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 411, in update
    _check_keys, self.__uuid_subtype), safe)
InvalidDocument: key 'openoffice.org-voikko' must not contain '.'



Version-Release number of selected component (if applicable):
katello-0.2.50-1.fc16.noarch
pulp-1.1.11-1.fc16.noarch



How reproducible:
100%


Steps to Reproduce:
1. install a f16 box
2. set the repository for http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/
3. sync the channel
  
Actual results:

  sync raises an error


Expected results:

  sync repo as expected


Additional info:
Comment 1 Marcelo Moreira de Mello 2012-07-27 11:58:52 UTC 

[root@katello ~]# find / -iname "*openoffice.org-vo*"
/var/lib/pulp/packages/openoffice.org-voikko
/var/lib/pulp/packages/openoffice.org-voikko/3.1.2/5.fc16/x86_64/eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b/openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm
/var/lib/pulp/repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/Packages/openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm
[root@katello ~]# rpm -q katello pulp
katello-0.2.50-1.fc16.noarch
pulp-1.1.11-1.fc16.noarch
[root@katello ~]#
Comment 2 Marcelo Moreira de Mello 2012-07-27 12:00:06 UTC 
[root@katello ~]# rpm -q pymongo
pymongo-2.1.1-1.fc16.x86_64
Comment 3 Marcelo Moreira de Mello 2012-07-31 20:53:42 UTC 

  To be able to sync the repository for F16, you must downgrade the pymongo version to pymongo-1.11-1.fc16.x86_64. 

   # yum downgrade pymongo python-bson

   # rpm -q pymongo python-bson
   pymongo-1.11-1.fc16.x86_64
   python-bson-1.11-1.fc16.x86_64


   Afterwards, restart katello services and sync again. 


=> pulp/pulp.log <==
2012-07-31 17:45:48,759 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:856 checksum type for repo id mmelloORG-Fedora_16_x86_64-fedora_ufpr is sha256
2012-07-31 17:45:48,798 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:839 Running createrepo, this may take a few minutes to complete.
2012-07-31 17:45:48,798 1093:140043599451904: pulp.server.util:INFO: util:507 metadata found; taking backup.
2012-07-31 17:45:49,070 1093:140043599451904: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml', '--update', '/var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr']
2012-07-31 17:49:28,474 1093:140043599451904: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr finished
2012-07-31 17:49:28,478 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:843 Createrepo finished in 219.679879904 seconds
2012-07-31 17:49:28,514 1093:140043599451904: pulp.server.api.distribution:INFO: distribution:50 Distribution with id ks-Fedora-Fedora-16-x86_64 already exists
2012-07-31 17:49:28,548 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:422 No metadata for 'File Sync' present; no files to import to repo..
2012-07-31 17:49:28,557 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:385 Loading comps group info from: /var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml
2012-07-31 17:49:28,873 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:412 Skipping errata imports from sync process
2012-07-31 17:49:29,185 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:286 Sync on mmelloORG-Fedora_16_x86_64-fedora_ufpr returned 3107 packages, 0 errata in 537.659252882 seconds
2012-07-31 17:49:29,525 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:297 0 old packages to process, 3107 new packages to process
2012-07-31 17:49:29,531 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:299 Removing 0 old packages
2012-07-31 17:49:29,964 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:305 Adding 3107 new packages
2012-07-31 17:49:30,806 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:314 Examining 0 errata from repo mmelloORG-Fedora_16_x86_64-fedora_ufpr
2012-07-31 17:49:31,292 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:318 Removing 0 old errata from repo mmelloORG-Fedora_16_x86_64-fedora_ufpr
2012-07-31 17:49:32,512 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:327 Adding 0 new errata to repo mmelloORG-Fedora_16_x86_64-fedora_ufpr
2012-07-31 17:49:32,958 1093:140043599451904: pulp.server.tasking.task:INFO: task:454 Task succeeded: Task f870a2c2-db4f-11e1-bbfd-525400d00a36: _sync(mmelloORG-Fedora_16_x86_64-fedora_ufpr, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f5eb162c290>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f5eb162c910>>)

 webUI will display the sync complete message.
Comment 4 Ivan Necas 2012-08-31 09:05:48 UTC 
Note: the same issue is when using EPEL
Comment 6 Mike McCune 2012-09-11 23:40:54 UTC 
On EL6 I tried with the pymongo from EPEL and it synced fine:

$ rpm -q redhat-release-server-6Server pymongo pulp
redhat-release-server-6Server-6.3.0.3.el6.x86_64
pymongo-2.1.1-1.el6.x86_64
pulp-1.1.11-1.el6.noarch

$ pulp-admin content list --repoid=ACME_Corporation-Random-random-2 |grep open
openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm,eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b

no errors in grinder.log or webUI
Comment 7 Marcelo Moreira de Mello 2012-09-12 20:51:06 UTC 
(In reply to comment #6)
> On EL6 I tried with the pymongo from EPEL and it synced fine:
> 
> $ rpm -q redhat-release-server-6Server pymongo pulp
> redhat-release-server-6Server-6.3.0.3.el6.x86_64
> pymongo-2.1.1-1.el6.x86_64
> pulp-1.1.11-1.el6.noarch
> 
> $ pulp-admin content list --repoid=ACME_Corporation-Random-random-2 |grep
> open
> openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm,
> eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b
> 
> no errors in grinder.log or webUI

 Hello Mike, 

   Are you able to sync it using the if you Katello box is a F16?

   I hit this issue this morning again and had to downgrade the pymongo to pymongo-1.11-1.fc16.x86_64 to be able to sync the package. 

   Thanks. 
mmello
Comment 8 Mike McCune 2012-11-28 16:15:19 UTC 
*** Bug 879679 has been marked as a duplicate of this bug. ***
Comment 9 David Kaylor 2012-12-03 15:09:44 UTC 
The same issue occurs when syncing the "Red Hat Enterprise Linux 6 Server - Optional RPMs x86_64 6.3" repository:

key 'openoffice.org-langpack-en' must not contain '.'
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content
    added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 416, in import_metadata
    self.repo_api.collection.save(repo, safe=True)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 237, in save
    manipulate, safe, _check_keys=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 411, in update
    _check_keys, self.__uuid_subtype), safe)
InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'

The Katello server is also RHEL 6.3 and the pymongo package installed is pymongo-2.1.1-1.el6.x86_64.
Comment 10 Bryan Kearney 2014-01-21 19:07:38 UTC 
Moving to Sat6 to be tracked there. Upstream bugs are moving to redmine.