Description of problem: Syncronization raises an exception when package have a different name structure. After entering the repository for F16 (http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/), when finishing the sync process, it raises the exception below: ==> pulp/grinder.log <== 2012-07-27 03:08:16,728 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:138 1 threads are active. 1 items left to be fetched 2012-07-27 03:08:16,728 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:329 WorkerThread deleting ActiveObject 2012-07-27 03:08:16,732 1020:140280250484480: grinder.ParallelFetch:INFO: ParallelFetch:344 Thread ending 2012-07-27 03:08:17,219 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:234 All threads have finished. 2012-07-27 03:08:17,270 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:253 ParallelFetch: 3113 items successfully processed, 3113 downloaded, 0 items had errors 2012-07-27 03:08:17,271 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:258 Transferred [564667992] bytes of [tree_file] 2012-07-27 03:08:17,271 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:258 Transferred [3455015673] bytes of [rpm] 2012-07-27 03:08:17,272 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:259 Transferred [4019683665] total bytes in 2242.64618111 seconds 2012-07-27 03:08:17,272 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:246 Finalizing metadata, moving /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr//repodata.new to /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr//repodata 2012-07-27 03:08:17,392 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:182 Cleaning any orphaned packages.. 2012-07-27 03:08:18,491 1020:140280258877184: grinder.RepoFetch:INFO: RepoFetch:192 Processed <>,<http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/> with <0> items in [2223] seconds. Report: 3113 successes, 3113 downloads, 0 errors 2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:203 Grinder stopping 2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,492 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,493 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-20, stopped daemon 140280267269888)>> to stop 2012-07-27 03:08:18,493 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-21, stopped daemon 140280275662592)>> to stop 2012-07-27 03:08:18,494 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-22, stopped daemon 140280250484480)>> to stop 2012-07-27 03:08:18,495 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:287 stop() invoked 2012-07-27 03:08:18,496 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:297 stop() completed 2012-07-27 03:08:18,496 1020:140280258877184: grinder.ParallelFetch:INFO: ParallelFetch:207 Told thread <<WorkerThread(Thread-23, stopped daemon 140280242091776)>> to stop ==> pulp/pulp.log <== 2012-07-27 03:08:18,497 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:793 YumSynchronizer reported 3113 successes, 3113 downloads, 0 errors 2012-07-27 03:10:08,403 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:856 checksum type for repo id mmello_ORG-Fedora_16_x86_64-fedora_ufpr is sha256 2012-07-27 03:10:08,413 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:839 Running createrepo, this may take a few minutes to complete. 2012-07-27 03:10:08,417 1020:140280258877184: pulp.server.util:INFO: util:507 metadata found; taking backup. 2012-07-27 03:10:09,050 1020:140280258877184: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml', '--update', '/var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr'] 2012-07-27 03:15:08,206 1020:140280258877184: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr finished 2012-07-27 03:15:08,226 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:843 Createrepo finished in 299.809221983 seconds 2012-07-27 03:15:08,307 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:477 Created a distributionID ks-Fedora-Fedora-16-x86_64 2012-07-27 03:15:08,331 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:422 No metadata for 'File Sync' present; no files to import to repo.. 2012-07-27 03:15:08,340 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:385 Loading comps group info from: /var/lib/pulp//repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml 2012-07-27 03:15:08,704 1020:140280258877184: pulp.server.api.synchronizers:INFO: synchronizers:412 Skipping errata imports from sync process 2012-07-27 03:15:08,981 1020:140280258877184: pulp.server.tasking.task:ERROR: task:472 Task failed: Task 3c3fe154-d7ac-11e1-83db-525400623320: _sync(mmello_ORG-Fedora_16_x86_64-fedora_ufpr, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f95a4041650>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f9584023110>>) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/pulp/server/tasking/task.py", line 418, in run result = self.callable(*self.args, **self.kwargs) File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync progress_callback, synchronizer, max_speed, threads) File "/usr/lib/python2.7/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict) File "/usr/lib/python2.7/site-packages/pulp/server/api/synchronizers.py", line 415, in import_metadata self.repo_api.collection.save(repo, safe=True) File "/usr/lib/python2.7/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 237, in save manipulate, safe, _check_keys=True, **kwargs) File "/usr/lib/python2.7/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.7/site-packages/pymongo/collection.py", line 411, in update _check_keys, self.__uuid_subtype), safe) InvalidDocument: key 'openoffice.org-voikko' must not contain '.' Version-Release number of selected component (if applicable): katello-0.2.50-1.fc16.noarch pulp-1.1.11-1.fc16.noarch How reproducible: 100% Steps to Reproduce: 1. install a f16 box 2. set the repository for http://fedora.c3sl.ufpr.br/linux/releases/16/Fedora/x86_64/os/ 3. sync the channel Actual results: sync raises an error Expected results: sync repo as expected Additional info:
[root@katello ~]# find / -iname "*openoffice.org-vo*" /var/lib/pulp/packages/openoffice.org-voikko /var/lib/pulp/packages/openoffice.org-voikko/3.1.2/5.fc16/x86_64/eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b/openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm /var/lib/pulp/repos/mmello_ORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/Packages/openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm [root@katello ~]# rpm -q katello pulp katello-0.2.50-1.fc16.noarch pulp-1.1.11-1.fc16.noarch [root@katello ~]#
[root@katello ~]# rpm -q pymongo pymongo-2.1.1-1.fc16.x86_64
To be able to sync the repository for F16, you must downgrade the pymongo version to pymongo-1.11-1.fc16.x86_64. # yum downgrade pymongo python-bson # rpm -q pymongo python-bson pymongo-1.11-1.fc16.x86_64 python-bson-1.11-1.fc16.x86_64 Afterwards, restart katello services and sync again. => pulp/pulp.log <== 2012-07-31 17:45:48,759 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:856 checksum type for repo id mmelloORG-Fedora_16_x86_64-fedora_ufpr is sha256 2012-07-31 17:45:48,798 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:839 Running createrepo, this may take a few minutes to complete. 2012-07-31 17:45:48,798 1093:140043599451904: pulp.server.util:INFO: util:507 metadata found; taking backup. 2012-07-31 17:45:49,070 1093:140043599451904: pulp.server.util:INFO: util:489 started repo metadata update: ['createrepo', '--database', '--checksum', 'sha256', '-g', '/var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml', '--update', '/var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr'] 2012-07-31 17:49:28,474 1093:140043599451904: pulp.server.util:INFO: util:542 createrepo on /var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr finished 2012-07-31 17:49:28,478 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:843 Createrepo finished in 219.679879904 seconds 2012-07-31 17:49:28,514 1093:140043599451904: pulp.server.api.distribution:INFO: distribution:50 Distribution with id ks-Fedora-Fedora-16-x86_64 already exists 2012-07-31 17:49:28,548 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:422 No metadata for 'File Sync' present; no files to import to repo.. 2012-07-31 17:49:28,557 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:385 Loading comps group info from: /var/lib/pulp//repos/mmelloORG/Library/custom/Fedora_16_x86_64/fedora_ufpr/repodata/d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-d7017d949b6f30725d20ee3ef45d9fa03d603464b9d411e056af2fddf5ceb608-Fedora-16-comps.xml 2012-07-31 17:49:28,873 1093:140043599451904: pulp.server.api.synchronizers:INFO: synchronizers:412 Skipping errata imports from sync process 2012-07-31 17:49:29,185 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:286 Sync on mmelloORG-Fedora_16_x86_64-fedora_ufpr returned 3107 packages, 0 errata in 537.659252882 seconds 2012-07-31 17:49:29,525 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:297 0 old packages to process, 3107 new packages to process 2012-07-31 17:49:29,531 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:299 Removing 0 old packages 2012-07-31 17:49:29,964 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:305 Adding 3107 new packages 2012-07-31 17:49:30,806 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:314 Examining 0 errata from repo mmelloORG-Fedora_16_x86_64-fedora_ufpr 2012-07-31 17:49:31,292 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:318 Removing 0 old errata from repo mmelloORG-Fedora_16_x86_64-fedora_ufpr 2012-07-31 17:49:32,512 1093:140043599451904: pulp.server.api.repo_sync:INFO: repo_sync:327 Adding 0 new errata to repo mmelloORG-Fedora_16_x86_64-fedora_ufpr 2012-07-31 17:49:32,958 1093:140043599451904: pulp.server.tasking.task:INFO: task:454 Task succeeded: Task f870a2c2-db4f-11e1-bbfd-525400d00a36: _sync(mmelloORG-Fedora_16_x86_64-fedora_ufpr, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f5eb162c290>, skip={}, max_speed=None, threads=4, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f5eb162c910>>) webUI will display the sync complete message.
Note: the same issue is when using EPEL
On EL6 I tried with the pymongo from EPEL and it synced fine: $ rpm -q redhat-release-server-6Server pymongo pulp redhat-release-server-6Server-6.3.0.3.el6.x86_64 pymongo-2.1.1-1.el6.x86_64 pulp-1.1.11-1.el6.noarch $ pulp-admin content list --repoid=ACME_Corporation-Random-random-2 |grep open openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm,eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b no errors in grinder.log or webUI
(In reply to comment #6) > On EL6 I tried with the pymongo from EPEL and it synced fine: > > $ rpm -q redhat-release-server-6Server pymongo pulp > redhat-release-server-6Server-6.3.0.3.el6.x86_64 > pymongo-2.1.1-1.el6.x86_64 > pulp-1.1.11-1.el6.noarch > > $ pulp-admin content list --repoid=ACME_Corporation-Random-random-2 |grep > open > openoffice.org-voikko-3.1.2-5.fc16.x86_64.rpm, > eb154dbedf52c544e783e0af679cc16d1ef290107900efd606951474c6c1533b > > no errors in grinder.log or webUI Hello Mike, Are you able to sync it using the if you Katello box is a F16? I hit this issue this morning again and had to downgrade the pymongo to pymongo-1.11-1.fc16.x86_64 to be able to sync the package. Thanks. mmello
*** Bug 879679 has been marked as a duplicate of this bug. ***
The same issue occurs when syncing the "Red Hat Enterprise Linux 6 Server - Optional RPMs x86_64 6.3" repository: key 'openoffice.org-langpack-en' must not contain '.' Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run result = self.callable(*self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync progress_callback, synchronizer, max_speed, threads) File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict) File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 416, in import_metadata self.repo_api.collection.save(repo, safe=True) File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 237, in save manipulate, safe, _check_keys=True, **kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry return method(*args, **kwargs) File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 411, in update _check_keys, self.__uuid_subtype), safe) InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.' The Katello server is also RHEL 6.3 and the pymongo package installed is pymongo-2.1.1-1.el6.x86_64.
Moving to Sat6 to be tracked there. Upstream bugs are moving to redmine.