Bug 818640 - mongodb interaction seems to hate package names that contain a '.' such as openoffice.org rpms.
mongodb interaction seems to hate package names that contain a '.' such as op...
Product: Pulp
Classification: Community
Component: user-experience (Show other bugs)
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Pradeep Kilambi
Preethi Thomas
Depends On:
Blocks: 825312
  Show dependency treegraph
Reported: 2012-05-03 11:38 EDT by Jim Perrin
Modified: 2013-09-09 12:30 EDT (History)
2 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 825312 (view as bug list)
Last Closed: 2012-05-25 10:13:56 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Jim Perrin 2012-05-03 11:38:50 EDT
Description of problem:
When syncing the rhel6-optional repository, the sync fails with:
InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.pulp-admin repo create --id live-rhel6-opt --feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/optional/os --feed_ca=mine --feed_key=mine --feed_cert=mine --relativepath=/live/rhel6/optional
2.pulp-admin repo sync --id live-rhel6-opt
3.pulp-admin repo status --id live-rhel6-opt
Actual results:
fails with InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'

Expected results:
repository syncs successfully. 

Additional info:
2012-05-03 10:20:31,167 22480:140181933876992: pulp.server.tasking.task:ERROR: task:472 Task failed: Task 8be02882-952f-11e1-ba57-bc305ba1aebf: _sync(live-rhel6-opt, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f7eb403ea50>, skip={}, max_speed=None, threads=None, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f7eb403e510>>)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content
    added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 415, in import_metadata
    self.repo_api.collection.save(repo, safe=True)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 237, in save
    manipulate, safe, _check_keys=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 411, in update
    _check_keys, self.__uuid_subtype), safe)
InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'
Comment 1 Pradeep Kilambi 2012-05-03 16:48:20 EDT
Interesting, I was able to reproduce the above error with mongodb-2.0.2 and pymongo-2.1; I then downgraded to mongodb-1.8 and pymongo-1.11 and the sync works fine.

Before downgrade my mongo rpms are,
$ rpm -qa |grep mongo

after downgrade,
$ rpm -qa|grep mongo

$ pulp-admin repo sync -F --id=el_optional
Sync for repository el_optional started
Sync: Finished
4374/4374 new items downloaded
0/4374 existing items processed

Item Details: 
RPMs: 4374/4374
Comment 2 Jim Perrin 2012-05-03 17:08:05 EDT
That would  (in theory) put the blame somewhat squarely on pymongo.

On my currently affected rhel6 system I have:

[root@evolution conf.d]# rpm -qa '*mongo*'

All are at or lower than your downgraded versions except pymongo.
Comment 3 Pradeep Kilambi 2012-05-04 07:59:43 EDT
yea the issue is with newer versions of pymongo. version 2.1 enforces the '.' check in key. But the older versions dint. The original issue causing this seems to be when we import packagegroups into mongo during syncs. 

IMO we should enforce pulp v1 to use pymongo < 2.1 and do the packagegroup imports correctly in v2.
Comment 4 Jim Perrin 2012-05-04 09:14:13 EDT
Rolling back to an older pymongo does indeed resolve it here. 

I'm not certain I like the idea of requiring a rollback for users if they've been keeping current, as it may cause repo conflicts between epel and pulp.
Comment 5 Pradeep Kilambi 2012-05-04 15:27:44 EDT
Yea, we decided the best approach is to put an FAQ on pulp wiki giving instructions on how to downgrade pymongo in case user sees this error. This will avoid the issue where user is already on newer version and pulp upgrade conflicts and fails.

In v2, we'll account for this scenario and design the imports accordingly so user wont hit this issue.

Comment 6 Preethi Thomas 2012-05-15 14:07:20 EDT
document updated
Comment 7 Preethi Thomas 2012-05-25 10:13:56 EDT
Pulp v1.1 Release
Comment 8 Preethi Thomas 2012-05-25 11:55:50 EDT
As per comment https://bugzilla.redhat.com/show_bug.cgi?id=818640#c5 moving to won't fix for 1.1

Will open a new one for 2.0

Note You need to log in before you can comment on or make changes to this bug.