Bug 812661

Summary: 'nova floating-ip-list' hangs
Product: [Fedora] Fedora Reporter: Cole Robinson <crobinso>
Component: openstack-novaAssignee: Mark McLoughlin <markmc>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 17CC: akscram, alexander.sakhnov, asalkeld, bfilippov, breu, jonathansteffan, markmc, matt_domsch, mlvov, p, rbryant, rkukura
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-04-18 14:26:43 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Cole Robinson 2012-04-15 19:37:08 UTC 
'nova floating-ip-list' hangs and eventually times out. I don't think this is a novaclient issue though. This causes dashboard 'access + security' screen to hang. 100% reproducible here. I haven't done any manual floating IP config so it should be in stock state after RPM install.

Only thing of note is in the nova API logs:

2012-04-15 15:30:07 INFO nova.api.openstack.wsgi [req-e6cf77cc-b796-4578-84f7-37ba4c66aa29 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] GET http://localhost:8774/v1.1/a9ae463a4fe8474d8709cfe98b3d4dac/os-floating-ips
2012-04-15 15:30:07 INFO nova.rpc.impl_qpid [req-e6cf77cc-b796-4578-84f7-37ba4c66aa29 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] Connected to AMQP server on localhost:5672
2012-04-15 15:31:07 ERROR nova.rpc.impl_qpid [req-e6cf77cc-b796-4578-84f7-37ba4c66aa29 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] Timed out waiting for RPC response: None
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid Traceback (most recent call last):
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 359, in ensure
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid     return method(*args, **kwargs)
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 408, in _consume
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid     nxt_receiver = self.session.next_receiver(timeout=timeout)
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid   File "<string>", line 6, in next_receiver
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 651, in next_receiver
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid     raise Empty
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid Empty: None
2012-04-15 15:31:07 TRACE nova.rpc.impl_qpid 
2012-04-15 15:31:07 ERROR nova.api.openstack [req-e6cf77cc-b796-4578-84f7-37ba4c66aa29 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] Caught error: Timeout while waiting on RPC response.
2012-04-15 15:31:07 TRACE nova.api.openstack Traceback (most recent call last):
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/__init__.py", line 41, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     return req.get_response(self.application)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1086, in get_response
2012-04-15 15:31:07 TRACE nova.api.openstack     application, catch_exc_info=False)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/request.py", line 1055, in call_application
2012-04-15 15:31:07 TRACE nova.api.openstack     app_iter = application(self.environ, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/keystone/middleware/auth_token.py", line 176, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     return self.app(env, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     return resp(environ, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     return resp(environ, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     return resp(environ, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/routes/middleware.py", line 131, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     response = self.app(environ, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 159, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     return resp(environ, start_response)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 147, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     resp = self.call_func(req, *args, **self.kwargs)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/webob/dec.py", line 210, in call_func
2012-04-15 15:31:07 TRACE nova.api.openstack     return self.func(req, *args, **kwargs)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 803, in __call__
2012-04-15 15:31:07 TRACE nova.api.openstack     content_type, body, accept)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 851, in _process_stack
2012-04-15 15:31:07 TRACE nova.api.openstack     action_result = self.dispatch(meth, request, action_args)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py", line 926, in dispatch
2012-04-15 15:31:07 TRACE nova.api.openstack     return method(req=request, **action_args)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/contrib/floating_ips.py", line 137, in index
2012-04-15 15:31:07 TRACE nova.api.openstack     floating_ips = self.network_api.get_floating_ips_by_project(context)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 92, in get_floating_ips_by_project
2012-04-15 15:31:07 TRACE nova.api.openstack     {'method': 'get_floating_ips_by_project'})
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/__init__.py", line 68, in call
2012-04-15 15:31:07 TRACE nova.api.openstack     return _get_impl().call(context, topic, msg, timeout)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 521, in call
2012-04-15 15:31:07 TRACE nova.api.openstack     return rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 338, in call
2012-04-15 15:31:07 TRACE nova.api.openstack     rv = list(rv)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 299, in __iter__
2012-04-15 15:31:07 TRACE nova.api.openstack     self._iterator.next()
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 417, in iterconsume
2012-04-15 15:31:07 TRACE nova.api.openstack     yield self.ensure(_error_callback, _consume)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 363, in ensure
2012-04-15 15:31:07 TRACE nova.api.openstack     error_callback(e)
2012-04-15 15:31:07 TRACE nova.api.openstack   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 402, in _error_callback
2012-04-15 15:31:07 TRACE nova.api.openstack     raise rpc_common.Timeout()
2012-04-15 15:31:07 TRACE nova.api.openstack Timeout: Timeout while waiting on RPC response.
2012-04-15 15:31:07 TRACE nova.api.openstack 
2012-04-15 15:31:07 INFO nova.api.openstack [req-e6cf77cc-b796-4578-84f7-37ba4c66aa29 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] http://localhost:8774/v1.1/a9ae463a4fe8474d8709cfe98b3d4dac/os-floating-ips returned with HTTP 500
Comment 1 Cole Robinson 2012-04-18 14:26:43 UTC 
Actually this is just another side effect of a hung nova-network

*** This bug has been marked as a duplicate of bug 812677 ***