+++ This bug was initially created as a clone of Bug #843800 +++ Description of problem: Syncronization raises an exception when package have a different name structure. After entering the repository for F16 (http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/), when finishing the sync process, it raises the exception below: ==> pulp/grinder.log <== 2012-07-27 03:08:16,728 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:138 1 threads are active. 1 items left to be fetched 2012-07-27 03:08:16,728 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:329 WorkerThread deleting ActiveObject 2012-07-27 03:08:16,732 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:344 Thread ending 2012-07-27 03:08:17,219 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:234 All threads have finished. 2012-07-27 03:08:17,270 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:253 ParallelFetch: 3113 items successfully processed, 3113 downloaded, 0 items had errors 2012-07-27 03:08:17,271 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:258 Transferred [564667992] bytes of [tree_file] 2012-07-27 03:08:17,271 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:258 Transferred [3455015673] bytes of [rpm] 2012-07-27 03:08:17,272 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:259 Transferred [4019683665] total bytes in 2242.64618111 seconds 2012-07-27 03:08:17,272 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:246 Finalizing metadata, moving /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr//repodata.new to /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr//repodata 2012-07-27 03:08:17,392 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:182 Cleaning any orphaned packages.. 2012-07-27 03:08:18,491 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:192 Processed <>,<http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/> with <0> items in [2223] seconds. Report: 3113 successes, 3113 downloads, 0 errors 2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:203 Grinder stopping 2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,493 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-20, stopped daemon 140280267269888)>> to stop 2012-07-27 03:08:18,493 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-21, stopped daemon 140280275662592)>> to stop 2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-22, stopped daemon 140280250484480)>> to stop 2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,496 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,496 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-23, stopped daemon 140280242091776)>> to stop ==> pulp/pulp.log <== 2012-07-27 03:08:18,497 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:793 YumSynchronizer reported 3113 successes, 3113 downloads, 0 errors 2012-07-27 03:10:08,403 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:856 checksum type for repo id mmello_ORG-Fedora_16_x86_64-fedora_ufpr is sha256 2012-07-27 03:10:08,413 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:839 Running createrepo, this may take a few minutes to complete. 2012-07-27 03:10:08,417 1020:140280258877184: pulp.server.util:INFO: util:507 metadata found; taking backup. 2012-07-27 03:10:09,050 1020:140280258877184: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml', '--update', '/var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr'] 2012-07-27 03:15:08,206 1020:140280258877184: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr finished 2012-07-27 03:15:08,226 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:843 Createrepo finished in 299.809221983 seconds 2012-07-27 03:15:08,307 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:477 Created a distributionID ks-Fedora-Fedora-16-x86_64 2012-07-27 03:15:08,331 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:422 No metadata for 'File Sync' present; no files to import to repo.. 2012-07-27 03:15:08,340 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:385 Loading comps group info from: /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml 2012-07-27 03:15:08,704 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:412 Skipping errata imports from sync process 2012-07-27 03:15:08,981 1020:140280258877184: pulp.server.tasking.task:ERROR: task:472 Task failed: Task 3c3fe154-d7ac-11e1-83db-525400623320: _sync(mmello_ORG-Fedora_16_x86_64-fedora_ufpr, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f95a4041650>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f9584023110>>) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/pulp/server/tasking/task.py", line 418, in run result = self.callable(*self.args, **self.kwargs) File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync progress_callback, synchronizer, max_speed, threads) File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict) File "/usr/lib/python2.7/site-packages/pulp/server/api/synchronizers.py", line 415, in import_metadata self.repo_api.collection.save(repo, safe=True) File "/usr/lib/python2.7/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 237, in save manipulate, safe, _check_keys=True, **kwargs) File "/usr/lib/python2.7/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 411, in update _check_keys, self.__uuid_subtype), safe) InvalidDocument: key 'openoffice.org-voikko' must not contain '.' Version-Release number of selected component (if applicable): katello-0.2.50-1.fc16.noarch pulp-1.1.11-1.fc16.noarch How reproducible: 100% Steps to Reproduce: 1. install a f16 box 2. set the repository for http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/ 3. sync the channel Actual results: sync raises an error Expected results: sync repo as expected Additional info: --- Additional comment from mmello on 2012-07-27 07:58:52 EDT --- [root@katello ~]# find / -iname "*openoffice.org-vo*" /var/lib/pulp/packages/openoffice.org-voikko /var/lib/pulp/packages/openoffice.org-voikko/3.1.2/5.fc16/x86_64/eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b/openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm /var/lib/pulp/repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/Packages/openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm [root@katello ~]# rpm -q katello pulp katello-0.2.50-1.fc16.noarch pulp-1.1.11-1.fc16.noarch [root@katello ~]# --- Additional comment from mmello on 2012-07-27 08:00:06 EDT --- [root@katello ~]# rpm -q pymongo pymongo-2.1.1-1.fc16.x86_64 --- Additional comment from mmello on 2012-07-31 16:53:42 EDT --- To be able to sync the repository for F16, you must downgrade the pymongo version to pymongo-1.11-1.fc16.x86_64. # yum downgrade pymongo python-bson # rpm -q pymongo python-bson pymongo-1.11-1.fc16.x86_64 python-bson-1.11-1.fc16.x86_64 Afterwards, restart katello services and sync again. => pulp/pulp.log <== 2012-07-31 17:45:48,759 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:856 checksum type for repo id mmelloORG-Fedora_16_x86_64-fedora_ufpr is sha256 2012-07-31 17:45:48,798 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:839 Running createrepo, this may take a few minutes to complete. 2012-07-31 17:45:48,798 1093:140043599451904: pulp.server.util:INFO: util:507 metadata found; taking backup. 2012-07-31 17:45:49,070 1093:140043599451904: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml', '--update', '/var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr'] 2012-07-31 17:49:28,474 1093:140043599451904: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr finished 2012-07-31 17:49:28,478 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:843 Createrepo finished in 219.679879904 seconds 2012-07-31 17:49:28,514 1093:140043599451904: pulp.server.api.distribution:INFO: distribution:50 Distribution with id ks-Fedora-Fedora-16-x86_64 already exists 2012-07-31 17:49:28,548 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:422 No metadata for 'File Sync' present; no files to import to repo.. 2012-07-31 17:49:28,557 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:385 Loading comps group info from: /var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml 2012-07-31 17:49:28,873 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:412 Skipping errata imports from sync process 2012-07-31 17:49:29,185 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:286 Sync on mmelloORG-Fedora_16_x86_64-fedora_ufpr returned 3107 packages, 0 errata in 537.659252882 seconds 2012-07-31 17:49:29,525 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:297 0 old packages to process, 3107 new packages to process 2012-07-31 17:49:29,531 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:299 Removing 0 old packages 2012-07-31 17:49:29,964 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:305 Adding 3107 new packages 2012-07-31 17:49:30,806 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:314 Examining 0 errata from repo mmelloORG-Fedora_16_x86_64-fedora_ufpr 2012-07-31 17:49:31,292 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:318 Removing 0 old errata from repo mmelloORG-Fedora_16_x86_64-fedora_ufpr 2012-07-31 17:49:32,512 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:327 Adding 0 new errata to repo mmelloORG-Fedora_16_x86_64-fedora_ufpr 2012-07-31 17:49:32,958 1093:140043599451904: pulp.server.tasking.task:INFO: task:454 Task succeeded: Task f870a2c2-db4f-11e1-bbfd-525400d00a36: _sync(mmelloORG-Fedora_16_x86_64-fedora_ufpr, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f5eb162c290>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f5eb162c910>>) webUI will display the sync complete message.
# COMMENT Confirm: doing on recent Katello with RHEL 6.3 installed sync of "Red Hat Enterprise Linux 6 Server - Optional RPMs x86_64 6Server" from CDN failing with: ---- 2012-08-30 18:05:57,586 8236:140146697553664: pulp.server.api.synchronizers:INFO: synchronizers:408 updateinfo is found in repomd.xml, it's path is /var/lib/pulp//repos/CFSE_QE_Team/Library/content/dist/rhel/server/6/6Server/x86_64/optional/os/repodata/6a0e57bdbf4bc4f9299040afd43733b387ad434639f74ac29b55ea491d8d90c8-updateinfo.xml.gz 2012-08-30 18:06:00,108 8236:140146697553664: pulp.server.tasking.task:ERROR: task:472 Task failed: Task 36736819-f2ad-11e1-a746-002713b84c93: _sync(CFSE_QE_Team-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_-_Optional_RPMs_x86_64_6Server, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f76641d9cd0>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f76641d91d0>>) Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run result = self.callable(*self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync progress_callback, synchronizer, max_speed, threads) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict) File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 416, in import_metadata self.repo_api.collection.save(repo, safe=True) File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 237, in save manipulate, safe, _check_keys=True, **kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 411, in update _check_keys, self.__uuid_subtype), safe) InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.' 2012-08-31 01:00:01,292 8236:140146315880192: pulp.server.tasking.task:INFO: task:454 Task succeeded: Task 102db0f5-f2a9-11e1-b591-002713b84c93: cull_history(, ) ---- Version of components: ----------------------------------- candlepin-0.7.7.1-1.el6.noarch katello-glue-candlepin-1.1.8-1.el6.noarch pulp-common-1.1.12-1.el6.noarch katello-selinux-1.1.1-1.el6.noarch qpid-cpp-client-0.14-16.el6.x86_64 qpid-cpp-client-ssl-0.14-16.el6.x86_64 python-oauth2-1.5.170-2.pulp.el6.noarch pulp-selinux-server-1.1.12-1.el6.noarch katello-qpid-client-key-pair-1.0-1.noarch qpid-cpp-server-ssl-0.14-16.el6.x86_64 python-qpid-0.14-8.el6.noarch katello-agent-1.1.2-1.el6.noarch katello-certs-tools-1.1.8-1.el6.noarch katello-glue-foreman-1.1.8-1.el6.noarch m2crypto-0.21.1.pulp-7.el6.x86_64 pulp-1.1.12-1.el6.noarch katello-all-1.1.8-1.el6.noarch katello-repos-1.1.2-1.el6.noarch qpid-cpp-server-0.14-16.el6.x86_64 katello-common-1.1.8-1.el6.noarch katello-configure-1.1.6-1.el6.noarch katello-candlepin-cert-key-pair-1.0-1.noarch katello-qpid-broker-key-pair-1.0-1.noarch candlepin-selinux-0.7.7.1-1.el6.noarch katello-glue-pulp-1.1.8-1.el6.noarch candlepin-tomcat6-0.7.7.1-1.el6.noarch katello-1.1.8-1.el6.noarch katello-cli-common-1.1.4-1.git.12.9f57f86.el6.noarch katello-cli-1.1.4-1.git.12.9f57f86.el6.noarch mod_wsgi-3.3-3.pulp.el6.x86_64
I have there installed: pymongo-2.1.1-1.el6.x86_64 python-bson-2.1.1-1.el6.x86_64
According to pulp FAQ [1], the recommended approach is to downgrade pymongo to version 1.11. [1] https://fedorahosted.org/pulp/wiki/UGFAQ#rpmname
# COMMENT verification scenario will include 2 parts: syncing RHEL6 Optional channel from CDN http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/ - as custom repo sync
# VERIFIED did the sync of both repos (above). all successfully completed. checked on version: --- python-qpid-0.7.946106-1.el6.noarch candlepin-cert-consumer-brno-cfse.usersys.redhat.com-1.0-1.noarch qpid-cpp-client-0.14-16.el6.x86_64 candlepin-0.7.8-1.el6cf.noarch katello-glue-pulp-1.1.12-7.el6cf.noarch qpid-cpp-server-ssl-0.14-16.el6.x86_64 katello-candlepin-cert-key-pair-1.0-1.noarch mod_wsgi-3.3-3.pulp.el6.x86_64 python-isodate-0.4.4-4.pulp.el6.noarch katello-cli-common-1.1.8-1.git.2.a0908e7.el6.noarch katello-cli-1.1.8-1.git.2.a0908e7.el6.noarch qpid-cpp-client-ssl-0.14-16.el6.x86_64 python-oauth2-1.5.170-2.pulp.el6.noarch pulp-common-1.1.12-1.el6cf.noarch pulp-selinux-server-1.1.12-1.el6cf.noarch m2crypto-0.21.1.pulp-7.el6.x86_64 candlepin-selinux-0.7.8-1.el6cf.noarch katello-common-1.1.12-7.el6cf.noarch katello-glue-candlepin-1.1.12-7.el6cf.noarch katello-1.1.12-7.el6cf.noarch katello-all-1.1.12-7.el6cf.noarch katello-qpid-broker-key-pair-1.0-1.noarch katello-certs-tools-1.1.8-1.el6cf.noarch katello-repos-1.1.2-1.el6.noarch katello-agent-1.1.2-1.el6.noarch katello-configure-1.1.9-3.el6cf.noarch qpid-cpp-server-0.14-16.el6.x86_64 pulp-1.1.12-1.el6cf.noarch katello-selinux-1.1.1-1.el6cf.noarch candlepin-tomcat6-0.7.8-1.el6cf.noarch katello-qpid-client-key-pair-1.0-1.noarch
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2012-1543.html
getting rid of 6.0.0 version since that doesn't exist