Attachment 577590 Details for Bug 812677 – compute log snippet

compute log snippet

compute-snippet.log (text/x-log), 10.99 KB, created by Cole Robinson on 2012-04-15 22:01:25 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Cole Robinson

Created: 2012-04-15 22:01:25 UTC

Size: 10.99 KB

patch

obsolete

>2012-04-15 17:55:44 ERROR nova.rpc.impl_qpid [req-a05569e1-312f-4735-8abf-bce386680275 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] Timed out waiting for RPC response: None
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid Traceback (most recent call last):
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 359, in ensure
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid     return method(*args, **kwargs)
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 408, in _consume
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid     nxt_receiver = self.session.next_receiver(timeout=timeout)
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid   File "<string>", line 6, in next_receiver
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 651, in next_receiver
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid     raise Empty
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid Empty: None
>2012-04-15 17:55:44 TRACE nova.rpc.impl_qpid 
>2012-04-15 17:55:44 ERROR nova.compute.manager [req-a05569e1-312f-4735-8abf-bce386680275 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] [instance: 4966131d-b879-410b-b4f8-08342b0ca357] Instance failed network setup
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357] Traceback (most recent call last):
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 559, in _allocate_network
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     requested_networks=requested_networks)
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 170, in allocate_for_instance
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     'args': args})
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/__init__.py", line 68, in call
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     return _get_impl().call(context, topic, msg, timeout)
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 521, in call
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     return rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 338, in call
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     rv = list(rv)
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 299, in __iter__
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     self._iterator.next()
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 417, in iterconsume
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     yield self.ensure(_error_callback, _consume)
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 363, in ensure
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     error_callback(e)
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 402, in _error_callback
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357]     raise rpc_common.Timeout()
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357] Timeout: Timeout while waiting on RPC response.
>2012-04-15 17:55:44 TRACE nova.compute.manager [instance: 4966131d-b879-410b-b4f8-08342b0ca357] 
>2012-04-15 17:55:44 ERROR nova.rpc.amqp [req-a05569e1-312f-4735-8abf-bce386680275 5d59c8790fbe406cb11605bb82fe959b a9ae463a4fe8474d8709cfe98b3d4dac] Exception during message handling
>2012-04-15 17:55:44 TRACE nova.rpc.amqp Traceback (most recent call last):
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 252, in _process_data
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     rval = node_func(context=ctxt, **node_args)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/exception.py", line 114, in wrapped
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     return f(*args, **kw)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 177, in decorated_function
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     sys.exc_info())
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     self.gen.next()
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 171, in decorated_function
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     return function(self, context, instance_uuid, *args, **kwargs)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 651, in run_instance
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     do_run_instance()
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/utils.py", line 948, in inner
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     retval = f(*args, **kwargs)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 650, in do_run_instance
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     self._run_instance(context, instance_uuid, **kwargs)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 451, in _run_instance
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     self._set_instance_error_state(context, instance_uuid)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib64/python2.7/contextlib.py", line 24, in __exit__
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     self.gen.next()
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 424, in _run_instance
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     requested_networks)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 559, in _allocate_network
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     requested_networks=requested_networks)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/network/api.py", line 170, in allocate_for_instance
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     'args': args})
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages2012-04-15 17:55:44 TRACE nova.rpc.amqp     return _get_impl().call(context, topic, msg, timeout)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 521, in call
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     return rpc_amqp.call(context, topic, msg, timeout, Connection.pool)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 338, in call
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     rv = list(rv)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/amqp.py", line 299, in __iter__
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     self._iterator.next()
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 417, in iterconsume
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     yield self.ensure(_error_callback, _consume)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 363, in ensure
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     error_callback(e)
>2012-04-15 17:55:44 TRACE nova.rpc.amqp   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 402, in _error_callback
>2012-04-15 17:55:44 TRACE nova.rpc.amqp     raise rpc_common.Timeout()
>2012-04-15 17:55:44 TRACE nova.rpc.amqp Timeout: Timeout while waiting on RPC response.
>2012-04-15 17:55:44 TRACE nova.rpc.amqp 
>2012-04-15 17:55:56 ERROR nova.rpc.impl_qpid [-] Timed out waiting for RPC response: None
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid Traceback (most recent call last):
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 359, in ensure
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid     return method(*args, **kwargs)
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/nova/rpc/impl_qpid.py", line 408, in _consume
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid     nxt_receiver = self.session.next_receiver(timeout=timeout)
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid   File "<string>", line 6, in next_receiver
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid   File "/usr/lib/python2.7/site-packages/qpid/messaging/endpoints.py", line 651, in next_receiver
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid     raise Empty
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid Empty: None
>2012-04-15 17:55:56 TRACE nova.rpc.impl_qpid 
>2012-04-15 17:55:56 DEBUG nova.manager [-] Skipping ComputeManager._run_image_cache_manager_pass, 38 ticks left until next run from (pid=26712) periodic_tasks /usr/lib/python2.7/site-packages/nova/manager.py:147
>2012-04-15 17:55:56 DEBUG nova.manager [-] Running periodic task ComputeManager._reclaim_queued_deletes from (pid=26712) periodic_tasks /usr/lib/python2.7/site-packages/nova/manager.py:152
>2012-04-15 17:55:56 DEBUG nova.compute.manager [-] FLAGS.reclaim_instance_interval <= 0, skipping... from (pid=26712) _reclaim_queued_deletes /usr/lib/python2.7/site-packages/nova/compute/manager.py:2380
>2012-04-15 17:55:56 DEBUG nova.manager [-] Running periodic task ComputeManager._report_driver_status from (pid=26712) periodic_tasks /usr/lib/python2.7/site-packages/nova/manager.py:152
>2012-04-15 17:55:56 INFO nova.compute.manager [-] Updating host status
>2012-04-15 17:55:56 DEBUG nova.virt.libvirt.connection [-] Updating host stats from (pid=26712) update_status /usr/lib/python2.7/site-packages/nova/virt/libvirt/connection.py:2471
>2012-04-15 17:55:58 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_unconfirmed_resizes from (pid=26712) periodic_tasks /usr/lib/python2.7/site-packages/nova/manager.py:152
>

Actions: View

Attachments on bug 812677: 577590