Bug 649118
| Summary: | [RFE] Support for running mongo DB on separate instance | ||
|---|---|---|---|
| Product: | [Retired] Pulp | Reporter: | Todd Sanders <tsanders> |
| Component: | z_other | Assignee: | John Matthews <jmatthew> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Preethi Thomas <pthomas> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | unspecified | CC: | jmatthew |
| Target Milestone: | --- | Keywords: | FutureFeature, ReleaseNotes, Triaged |
| Target Release: | Sprint 20 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Enhancement | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2011-08-15 15:14:41 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 647488, 673012 | ||
Fixed with below two commits http://git.fedorahosted.org/git/?p=pulp.git;a=commit;h=72fd8b8cd32a8856e1d92adedbb13cfe3766fc95 http://git.fedorahosted.org/git/?p=pulp.git;a=commit;h=dd6ddc90fd57f253dfa5b352a38917d584376992 Will move this to MODIFIED after I've had a chance to do more testing with multiple mongo databases. I verified on Fedora 14 pulp was able to use pymongo 1.9 and connect to a replica set. Followed the setup instructions here: https://fedorahosted.org/pulp/wiki/DBFailover To configure pulp to use a specific mongodb server edit: /etc/pulp/pulp.conf [database] seeds = localhost Change localhost to be a mongodb URI as defined here: http://www.mongodb.org/display/DOCS/Connections mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]] Fixed in build 0.117. [database] # automatically upgrade the database when the data model changes auto_upgrade: true name: _database # comma separated list of URIs to pass to pymongo seeds: 10.16.79.220 [root@10 ~]# service mongod stop Stopping mongod: [ OK ] [root@10 ~]# [root@10 ~]# [root@10 ~]# [root@10 ~]# [root@10 ~]# [root@10 ~]# mongo 10.16.79.220 MongoDB shell version: 1.6.4 connecting to: 10.16.79.220/test > exit bye [root@10 ~]# pulp-admin repo list error: operation failed: Authorization failed. Check your username and password or your certificate [root@10 ~]# service mongod status mongod is stopped [root@10 ~]# o -bash: o: command not found [root@10 ~]# [root@10 ~]# [root@10 ~]# [root@10 ~]# pulp-admin -u admin -p admin repo create --id=test --name=pulp --feed=yum:http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-14/x86_64/ Successfully created repository [ test ] [root@10 ~]# pulp-admin -u admin -p admin repo sync --id=test -F Sync for repository test started You can safely CTRL+C this current command and it will continue [==================================================] 100% (10 of 10 pkgs) 10/10 new items downloaded 0/10 existing items verified Sync: Finished [root@10 ~]# pulp-admin repo list error: operation failed: Authorization failed. Check your username and password or your certificate [root@10 ~]# pulp-admin -u admin -p admin repo list +------------------------------------------+ List of Available Repositories +------------------------------------------+ Id test Name pulp FeedURL http://repos.fedorapeople.org/repos/pulp/pulp/testing/fedora-14/x86_64/ FeedType yum Arch noarch Sync Schedule None Packages 10 Files 0 Distributions None Publish True Clones [] Groups None For the DB Failover to work we need to upgrade pymongo to 1.9.x. Currently we are using python-pymongo-1.6.x. This upgrade requires more work on our build bot to handle the differences between Fedora/RHEL. Plan to address in Sprint 20 Upgraded to pymongo 1.9 http://git.fedorahosted.org/git/?p=pulp.git;a=commit;h=25f89653e6b6537a9961640ca16dbcdfed54cb8c [ReleaseNotes] Upgrades of pulp require a manual step to address a name change to "pymongo". Pulp had a dependency on a custom packaged version of pymongo called python-pymongo. We have changed to using pymongo from Fedora/EPEL. The manual step required prior to upgrade is: "sudo rpm -e --nodeps python-pymongo" Fixed in 0.136.
[root@preethi ~]# rpm -q pulp
pulp-0.0.181-1.fc14.noarch
[database]
# automatically upgrade the database when the data model changes
name: pulp_database
# comma separated list of URIs to pass to pymongo
#seeds: localhost
seeds: kalpana.usersys.redhat.com
mongodb://admin:admin@]kalpana.usersys.redhat.com
# Uncomment the below section with appropriate values, to use an external ldap for
# user authentication instead of pulp.
#[ldap]
#uri: ldap://localhost
#base: dc=localhost
[root@preethi ~]# vi /etc/pulp/pulp.conf
[root@preethi ~]# service pulp-server restart
Stopping httpd: [ OK ]
Stopping Qpid AMQP daemon: [ OK ]
Starting mongod: [ OK ]
Starting Qpid AMQP daemon: [ OK ]
Starting httpd: [ OK ]
[root@preethi ~]# pulp-admin repo list
No repositories available to list
[root@preethi ~]#
[root@preethi ~]#
[root@preethi ~]# pulp-admin -u admin -p admin repo create --id=test --name=pulp
Successfully created repository [ test ]
[root@preethi ~]# pulp-admin repo list
+------------------------------------------+
List of Available Repositories
+------------------------------------------+
Id test
Name pulp
Feed URL None
Feed Type None
Feed Certs CA:No Cert:No Key:No
Consumer Certs CA:No Cert:No Key:No
Architecture noarch
Sync Schedule None
Packages 0
Files 0
Distributions None
Publish True
Clones []
Groups None
Filters []
Notes {}
[root@preethi ~]# pulp-admin -u admin -p admin repo delete --id=test
Successful deleted repository [ test ]
[root@preethi ~]# pulp-admin repo list
No repositories available to list
[root@preethi ~]#
Closing with current community release pulp-0.0.223 |
Description of RFE: Currently, we assume mongo will be run on localhost at the standard port (27017). I want to be able to support multiple mongo DBs running on separate instances or user-defined ports. This will require changes to both the pulp configuration (pulp.conf), plus the use/config of the pymongo driver. Suggestions: 1. /etc/pulp/pulp.conf: Add attribute for DB seeds. [database] # automatically upgrade the database when the data model changes auto_upgrade: true # mongo DB seeds: db1-server:27017,db2-server:27018,db3-server:27019 2. /src/pulp/server/db/connection.py def _initialize(): """ Initialize the connection pool and top-level database for pulp. """ global _connection, _database try: _connection = pymongo.Connection(config.config.get('database', 'seeds')) _database = _connection._database _database.add_son_manipulator(NamespaceInjector()) _database.add_son_manipulator(AutoReference(_database)) except Exception: _log.critical('Database initialization failed') _connection = None _database = None raise