Bug 818640

Summary: mongodb interaction seems to hate package names that contain a '.' such as openoffice.org rpms.
Product: [Retired] Pulp Reporter: Jim Perrin <jperrin>
Component: user-experienceAssignee: Pradeep Kilambi <pkilambi>
Status: CLOSED WONTFIX QA Contact: Preethi Thomas <pthomas>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: jortel, pkilambi
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 825312 (view as bug list) Environment:
Last Closed: 2012-05-25 14:13:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 825312    

Description Jim Perrin 2012-05-03 15:38:50 UTC
Description of problem:
When syncing the rhel6-optional repository, the sync fails with:
InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'

Version-Release number of selected component (if applicable):
pulp-1.1.7-1.el6.noarch
mongodb-1.8.2-2.el6.x86_64


How reproducible:
Always

Steps to Reproduce:
1.pulp-admin repo create --id live-rhel6-opt --feed=https://cdn.redhat.com/content/dist/rhel/server/6/6Server/x86_64/optional/os --feed_ca=mine --feed_key=mine --feed_cert=mine --relativepath=/live/rhel6/optional
2.pulp-admin repo sync --id live-rhel6-opt
3.pulp-admin repo status --id live-rhel6-opt
  
Actual results:
fails with InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'

Expected results:
repository syncs successfully. 

Additional info:
2012-05-03 10:20:31,167 22480:140181933876992: pulp.server.tasking.task:ERROR: task:472 Task failed: Task 8be02882-952f-11e1-ba57-bc305ba1aebf: _sync(live-rhel6-opt, synchronizer=<pulp.server.api.synchronizers.YumSynchronizer object at 0x7f7eb403ea50>, skip={}, max_speed=None, threads=None, progress_callback=<bound method RepoSyncTask.progress_callback of <pulp.server.api.repo_sync_task.RepoSyncTask object at 0x7f7eb403e510>>)
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/tasking/task.py", line 418, in run
    result = self.callable(*self.args, **self.kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 283, in _sync
    progress_callback, synchronizer, max_speed, threads)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/repo_sync.py", line 379, in fetch_content
    added_errataids = synchronizer.import_metadata(repo_dir, repo_id, skip_dict)
  File "/usr/lib/python2.6/site-packages/pulp/server/api/synchronizers.py", line 415, in import_metadata
    self.repo_api.collection.save(repo, safe=True)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 237, in save
    manipulate, safe, _check_keys=True, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 80, in retry
    return method(*args, **kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 411, in update
    _check_keys, self.__uuid_subtype), safe)
InvalidDocument: key 'openoffice.org-langpack-en' must not contain '.'

Comment 1 Pradeep Kilambi 2012-05-03 20:48:20 UTC
Interesting, I was able to reproduce the above error with mongodb-2.0.2 and pymongo-2.1; I then downgraded to mongodb-1.8 and pymongo-1.11 and the sync works fine.

Before downgrade my mongo rpms are,
$ rpm -qa |grep mongo
libmongodb-2.0.2-8.fc16.x86_64
mongodb-server-2.0.2-8.fc16.x86_64
mongodb-2.0.2-8.fc16.x86_64
pymongo-2.1.1-1.fc16.x86_64



after downgrade,
$ rpm -qa|grep mongo
libmongodb-1.8.2-9.fc16.x86_64
pymongo-1.11-1.fc16.x86_64
mongodb-1.8.2-9.fc16.x86_64
mongodb-server-1.8.2-9.fc16.x86_64



$ pulp-admin repo sync -F --id=el_optional
Sync for repository el_optional started
Sync: Finished
4374/4374 new items downloaded
0/4374 existing items processed

Item Details: 
RPMs: 4374/4374

Comment 2 Jim Perrin 2012-05-03 21:08:05 UTC
That would  (in theory) put the blame somewhat squarely on pymongo.

On my currently affected rhel6 system I have:

[root@evolution conf.d]# rpm -qa '*mongo*'
libmongodb-1.8.2-2.el6.x86_64
pymongo-2.1.1-1.el6.x86_64
mongodb-server-1.8.2-2.el6.x86_64
mongodb-1.8.2-2.el6.x86_64


All are at or lower than your downgraded versions except pymongo.

Comment 3 Pradeep Kilambi 2012-05-04 11:59:43 UTC
yea the issue is with newer versions of pymongo. version 2.1 enforces the '.' check in key. But the older versions dint. The original issue causing this seems to be when we import packagegroups into mongo during syncs. 

IMO we should enforce pulp v1 to use pymongo < 2.1 and do the packagegroup imports correctly in v2.

Comment 4 Jim Perrin 2012-05-04 13:14:13 UTC
Rolling back to an older pymongo does indeed resolve it here. 

I'm not certain I like the idea of requiring a rollback for users if they've been keeping current, as it may cause repo conflicts between epel and pulp.

Comment 5 Pradeep Kilambi 2012-05-04 19:27:44 UTC
Yea, we decided the best approach is to put an FAQ on pulp wiki giving instructions on how to downgrade pymongo in case user sees this error. This will avoid the issue where user is already on newer version and pulp upgrade conflicts and fails.

In v2, we'll account for this scenario and design the imports accordingly so user wont hit this issue.

https://fedorahosted.org/pulp/wiki/UGFAQ#rpmname

Comment 6 Preethi Thomas 2012-05-15 18:07:20 UTC
verified
document updated
https://fedorahosted.org/pulp/wiki/UGFAQ#rpmname

Comment 7 Preethi Thomas 2012-05-25 14:13:56 UTC
Pulp v1.1 Release

Comment 8 Preethi Thomas 2012-05-25 15:55:50 UTC
As per comment https://bugzilla.redhat.com/show_bug.cgi?id=818640#c5 moving to won't fix for 1.1

Will open a new one for 2.0