Bug 692969 - pulp should handle pymongo error
Summary: pulp should handle pymongo error
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: unspecified
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ---
: Sprint 23
Assignee: Jason Connor
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-04-01 20:20 UTC by dgao
Modified: 2014-03-31 01:39 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-07-15 21:59:07 UTC


Attachments (Terms of Use)

Description dgao 2011-04-01 20:20:12 UTC
[root@pulp-nightly ~]# service mongod stop
Stopping mongod:                                           [  OK  ]
[root@pulp-nightly ~]# pulp-admin -u admin -p admin repo list
error: operation failed: AutoReconnect: could not find master/primary

Comment 1 dgao 2011-04-06 15:50:12 UTC
The pulp server should really catch this error, that way the client should not see this error.

Comment 2 Jason Connor 2011-05-12 17:41:21 UTC
Pulp now retries on the AutoReconnect exception for a configurable number of tries.

Comment 3 Jeff Ortel 2011-05-12 18:45:39 UTC
build: 0.176

Comment 4 dgao 2011-06-07 21:46:19 UTC
[root@pulp-nightly ~]# service mongod stop
Stopping mongod:                                           [  OK  ]
[root@pulp-nightly ~]# pulp-admin -u admin -p admin auth login
error: operation failed: AutoReconnect: could not find master/primary


pulp.log
2011-06-07 18:23:05,866 [ERROR][Dummy-5] report_error() @ base.py:57 - Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/base.py", line 53, in report_error
    return method(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pulp/server/webservices/controllers/base.py", line 94, in _auth_decorator
    user = check_username_password(username, password)
  File "/usr/lib/python2.7/site-packages/pulp/server/auth/authentication.py", line 123, in check_username_password
    return _check_username_password_local(username, password)
  File "/usr/lib/python2.7/site-packages/pulp/server/auth/authentication.py", line 98, in _check_username_password_local
    user = _user_api.user(username)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/user.py", line 94, in user
    users = self.users({'login': login}, fields)
  File "/usr/lib/python2.7/site-packages/pulp/server/api/user.py", line 87, in users
    users = list(self.collection.find(spec=spec, fields=fields))
  File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 601, in next
    if len(self.__data) or self._refresh():
  File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 564, in _refresh
    self.__query_spec(), self.__fields))
  File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 521, in __send_message
    **kwargs)
  File "/usr/lib64/python2.7/site-packages/pymongo/connection.py", line 694, in _send_message_with_response
    sock = self.__socket()
  File "/usr/lib64/python2.7/site-packages/pymongo/connection.py", line 542, in __socket
    sock = self.__pool.socket()
  File "/usr/lib64/python2.7/site-packages/pymongo/connection.py", line 160, in socket
    self.sock = (pid, self.socket_factory())
  File "/usr/lib64/python2.7/site-packages/pymongo/connection.py", line 518, in __connect
    host, port = self.__find_master()
  File "/usr/lib64/python2.7/site-packages/pymongo/connection.py", line 509, in __find_master
    raise AutoReconnect("could not find master/primary")
AutoReconnect: could not find master/primary

Comment 5 Sachin Ghai 2011-07-05 09:59:00 UTC
Facing same issue with new build ( pulp 0.201 and rhui-tools 2.0.34).

The pulp server was idle from last 2 days and got following error in /var/log/httpd/ssl_error_log

[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111] mod_wsgi (pid=3320): Target WSGI script '/srv/pulp/webservices.wsgi' cannot be loaded as Python module.
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111] mod_wsgi (pid=3320): Exception occurred processing WSGI script '/srv/pulp/webservices.wsgi'.
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111] Traceback (most recent call last):
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/srv/pulp/webservices.wsgi", line 17, in <module>
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     from pulp.server.webservices.application import wsgi_application
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 29, in <module>
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     connection.initialize()
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 47, in initialize
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     _connection = pymongo.Connection(seeds)
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib64/python2.6/site-packages/pymongo/connection.py", line 305, in __init__
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     self.__find_master()
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib64/python2.6/site-packages/pymongo/connection.py", line 509, in __find_master
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     raise AutoReconnect("could not find master/primary")
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111] AutoReconnect: could not find master/primary
[root@dhcp201-111 ~]# tail -f /var/log/httpd/ssl_error_log
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     from pulp.server.webservices.application import wsgi_application
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 29, in <module>
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     connection.initialize()
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib/python2.6/site-packages/pulp/server/db/connection.py", line 47, in initialize
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     _connection = pymongo.Connection(seeds)
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib64/python2.6/site-packages/pymongo/connection.py", line 305, in __init__
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     self.__find_master()
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]   File "/usr/lib64/python2.6/site-packages/pymongo/connection.py", line 509, in __find_master
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111]     raise AutoReconnect("could not find master/primary")
[Tue Jul 05 13:15:57 2011] [error] [client 10.65.201.111] AutoReconnect: could not find master/primary


pulp.log:
========

2011-07-05 13:15:57,931 3320:140344735811344: pulp.server.db.connection:INFO: connection:46 Attempting Database connection with seeds = localhost
2011-07-05 13:15:57,932 3320:140344735811344: pulp.server.db.connection:CRITICAL: connection:53 Database initialization failed
2011-07-05 13:19:04,283 4063:140071028119520: pulp.server.async:INFO: async:333 Task reply handler, started.
2011-07-05 13:19:04,284 4063:140070669629200: gofer.messaging.broker:INFO: broker:100 connecting:
{dhcp201-111.englab.pnq.redhat.com:5674}:
transport=SSL
host=dhcp201-111.englab.pnq.redhat.com
port=5674
cacert=/etc/pki/rhua/qpid-ca.crt
clientcert=/etc/pki/rhua/qpid-client.crt


[root@dhcp201-111 Packages]# service mongod status
mongod dead but subsys locked


Note You need to log in before you can comment on or make changes to this bug.